Graphs and deltas

Emoncms
1

Hi all,

I just recently installed my emonPi and I could use some help with the “delta” feature of graphs. For now the only node I have is a custom arduino-based node that reports water and fuel consumption over the radio.
The node reports the total index to date (and restarts at zero whenever the arduino is restarted). It sends data at most once every 30s, and only if the index has changed.

I have those two feeds:

feeds
feeds1291×255 26 KB

For the inputs I simply used “log to feed” and PHPFINA Fixed in 30s

What I am trying to do is create graphs that show hourly and/or daily fuel and water consumption. I tried quite a few settings in the Graph view but I am not getting anywhere… mostly because I don’t understand precisely what the expectations are w.r.t. the underlying data.
Here are two sample graphs: the default one for “water”, and the same graph after selecting “delta” and reloading; a couple bars do show a correct delta, but most of them apparently show “index - 0”.

I assume that missing values are a problem - how would you recommend that I set up my inputs, feed and graphs to be able to create nice graphs showing consumption over arbitrary periods (e.g. anywhere from per minute to per month)?
Is there somewhere a detailed explanation of how the graph options (“Type”, the number next to “Type”, “Fix” checkbox, “Delta”, “DP”…) are used to drive the processing?

I read about the Wh accumulator, however
a) I am reporting liters (water) and milliliters (fuel) here, not too sure how the spike filtering will behave here?
b) The comment mentions “daily kWh data” - will I be able to create deltas for arbitrary intervals from that feed?

If needed I can modify the Arduino program to avoid gaps and send the index at a fixed period; I can’t prevent the restart to 0 when the arduino is reset though.

Any suggestion welcome!
Franck

Default graph:

graph1
graph11046×737 93.7 KB

With “deltas” selected:

graph2
graph21028×782 92.2 KB

Sample raw water data (arduino in debug mode → period may not be 30s):

1546691400, null
1546692300, null
1546693200, null
1546694100, 746
1546695000, null
1546695900, 774
1546696800, 774
1546697700, null
1546698600, 807
1546699500, 822
1546700400, null
1546701300, null
1546702200, 825
1546703100, null
2

Taking this step by step, how exactly does the “log to feed” setting of an input work? Say I simplify the node so that it sends its data every 30s, always, so I’d have
0s 350
30s 350
60s 352
90s 358
120s 358

If I configure the input as Wh accumulator, “PHPFINA fixed in” “120s” what will happen? Will the feed process the most recently received input value when it hits the 2mn mark?

Thanks!
Franck

3

Correct!

That is a better solution, if you look at the example data you posted eg

1546699500, 822
1546700400, null
1546701300, null
1546702200, 825

We can tell that 3 units were used over the 15mins between 1546701300 and 1546702200 (12 unit/hr for 15mins) following no activity for 30mins. BUT, in emoncms graphs this will appear as 3 units used over the 45mins between 1546699500 and 1546702200 (4 units/hour for 45mins) due to the nulls in place of zeros. Since the zero activity is just as informative as the non-zero values, and they make no impact on the file size on disc, there seems little point in not sending the zero readings unless you are using a really tight gsm data plan to send the data (and even then it might be worth considering). The data

1546699500, 822
1546700400, 0
1546701300, 0
1546702200, 825

would graph as expected.

4

Thanks Paul,

I will give that a try.

5

I had a similar sort of issue (not exact as it was temperature reporting) Processes and erroneous input values and my solution was not to update the feed if the input was null by using a skip if null. Depending on the feed type, Emoncms will then pad the data but that seems to be better than feeding it nulls. I think there is a difference between a value that is null and a padding null.

I wonder if Emoncms needs a generic accumulator process for this sort of data recording? Might be worth raising an issue on GitHub. I doubt it would be too difficult to do (beyond me though :smile:) . All the accumulator process does (in effect) is store the last value in cache (using REDIS) so if the supplying data is reset to zero, the process just adds the next feed value onto the cached value (there is a bit more to it than that over simplification).

6

I definitely agree that a generic processor for index-type inputs would be useful, if only for usability… I am still getting up to speed with the environment, I may look into that at some point since I intend to have quite a few pulse-type inputs.