Processes and erroneous input values

Is there an explanation anywhere of the various processes available when creating feeds?

I have a data feed that sometimes gives an erroneous figure. Is there a simple way to ignore it? I wonder if the rate process could be used, but does ‘skip next’ skip the next figure rather than the current one?

The green line is the RH scale and is simply an on/off 1/0 indicator - probably more difficult to check. The other inputs are all temperature (LH scale) which occasionally give a 0.

No. But IMO there should be.

There is a brief description of each process provided when you select any process and some examples of using the most basic processes given in the user guide. But there is nowhere that gives indepth information, examples or tips and tricks. I suggested this was needed sometime ago and got shot down in flames. So I keep my own documentation which is a PITA to maintain single handed as I do not know when changes get slipped in and/or find time to test everything so it’s a sort of liquid work in progress set of notes really.

The “skip next” processes skip the next process, so if you had

  1. skip next if < 5
  2. log to feed
  3. some other process

then any value under 5 would cause the “skip next” to skip logging that value to the feed and carry on from the “other process” (if there is one).

So yes that could do what you need if a zero value is totally out of normal range so that you could filter it as such.

Assuming a regular input interval and a matching fixed interval feed, it would result in a null datapoint so the graph would ignore rather than display a “0”. Or with a phptimeseries feed the datapoint just wouldn’t exist (not null nor 0).

1 Like

Missed that or I’d have added some support. Perhaps the environment has changed.

What does the ‘rate’ process do? What I am looking for is an absolute difference greater than 1 (temperature will not move that quickly). I’m not clear from the summary, especially if the correct value is negative.

I’m not sure you can do what you want efficiently with what processes we have.

You may be able to achieve it if you have additional supporting feeds and a pretty complex process list, eg the “rate of change” which basically would determine the difference so you could use that to trigger a logic process, but yes, I think you would need separate sub-chains of processes to handle positive and negative increments, eg if neg multiply by -1 before hitting the logic “if greater than 1” process. The problem with complex processing is that there is no way of holding values to reuse later in the processlist other than in a feed, so it gets quite bulky to do simple tasks.

What is a Helios? Can you fix the issue at source?

Will the Helio temperature ever go to 0? It would be much easier to use “0” or a fixed limit to do what you want, despite being significantly less desirable.

It is my MHRV unit. I am tapping into the MODBUS so I’m not sure why I get the occasional bad reading. I have had to replace one of the temperature sensors already so it might be a sign the others are on the way out. However, it will not run properly without them! And yes, they will definitely at some time go to zero (and below)!

I think I will publish and process through node red rather than publish direct. Only just realised there was an issue!


How are you tapping in? Is it possible that the reading is not “0” but a failed reading being presented as “0”?

Do you poll each value separately or poll for a block of registers?

The better way is to poll for a block and if it fails to complete, abort and move on to the next loop. When you poll for individual values there is a lot of unnecessary to’ing and fro’ing that increases traffic and requires a fast turn around and time outs, so it’s easier to fail than with a slower single read.

I’m doing a lot of Modbus coding at the moment and I’ve experimented with several approaches on 8 different devices, it’s quicker, more reliable and (more so in the case of energy monitors) more accurate to batch read as the reading are all for the exact same point in time.

Even if you do go to nodered, I would still be interested in knowing more about this as my Modbus project is potentially going to be for emonhub. As i say, I have 8 devices, but I’m trying to make a robust solution so any “modbus issues” are useful to me.

TBH I have no idea how it works!

I found a library that someone had written to interface to the modbus. I bought a USB interface module and plugged it in. It does what I ask although I am sure I could use a more general MODBUS solution and I would like to be able to send commands.

Part of the problem I think is that there is already a controller and slave so I think I might be muscling my way into that!

It is all posted on github GitHub - borpin/heliospy: Python script to communicate with a Helios EC500 Pro RS485 bus The python was one of the first things I did and is horrible!

I’d like to be able to control the fan speed but not worked out how.

Do you have a manual or link to the modbus register details for the mvhr unit? Are you sure it’s Modbus and not just rs485 and a manufacturer specific protocol? I didn’t look too long at the heliospy code, but I didn’t notice any Python modbus modules being imported, maybe they do it without a library, I didn’t look too deep.

Oh! Yes if it is Modbus there should only be one master on the bus.

I’ll dig it out tonight. You are right, I just assumed (sic) it was MODBUS.

@pb66, I have dropped the comms file into the github repository. The original link I got it from is long gone :smile:

[edit] - just realised I had already put it there!!! It is the PDF.

I just have a brief look through the code and wonder if this line

might benefit from being telegram=[None,None,None,None,None,None] (or even just telegram=[None]*6 is neater?)

so that if the string of tests does happen to allow a bad read through, it would return None not 0 when it hits L154 return telegram[4] as telegram[4] could still be 0, eg the crc for a all zero packet can be 0 so you could be dependent on just the sender and receiver values causing it to fail.

This isn’t Modbus, which I guess is a good thing if you have multiple masters. Since there are references to multiple masters and both a sender and recipient id in the payload, it sounds like multi-master is fine.

1 Like

Thanks Paul. I’ll give it a go. I need to make the main routine more robust so time I had a look at it.

Hi @pb66 ,

Having worked out how to do logging, I have an output of the data via MQTT and there is a null in the JSON string for bad data.

Can you tell me what the default behaviour for Emoncms is for a null input value? I though it was ignored and not logged but I may be wrong. If I am, I need to add in a step in the feed process to skip if null.