Hi i have a portable EVSE to charge my EV that measures energy and delivers data to my local emoncms install only when it is connected and charging at home.
When i finish charging before midnight, the next day all kWh/d feeds are not updated to zero, making all visualizations wrong.
The same thing happened on my solar measurement system since it only produces valid data when there’s sun . A valid workaround is to keep posting data - all zeroes, just to make emoncms happy.
But on my portable EVSE i have no way to keep sending data when it is not physically present or off. I suppose i could do a artificial feed on Node-RED to push zeroes but that would not be very nice.
How can emoncms handle non continuous data feeds like the examples above?
You do not need to send continuous data to achieve what you describe. the issue is that since no data arrives after midnight, the kWh/d feeds (or any feeds) do not get updated since there is no input to trigger the kWh/d process to say “this new data is not the same day as the last so lets start a new count rather than accumulating it on yesterdays total” simply sendin one zero data point shortly after midnight would fix the issue you describe.
This issue would not exist if you were using the “OEM recommended” method of NOT using kWh/d feeds and instead use the kWh total to “graph” daily totals, however I have to agree that method falls short in that it doesn’t allow you to display today’s total which is by far the most useful aspect of a daily report IMO so a simple hack to send a single datapoint after midnight is your best bet.
Hi paul, thank you for your clear explanation. I will have to list every “mobile” data feed i have and artificially send a update at midnight to refresh the kWh/d feeds.
You shouldn’t need to do that, You can send a single zero value to it’s own input and the “zero” all the kWh’d feeds in that inputs processing no need to do them all individually and you can write to one feed from multiple inputs no problem, it just gets erratic when using live fixed interval feeds. Since the kWh/d feeds are phptimeseries there is no issue and even any fixed interval feeds in this instance would not be an issue as no data is being sent to them at night to cause conflicts.
No, create a single new input to be updated once at midnight that posts zeros to all the existing kWh/d feeds in parrallel to the existing inputs that work normally,
One “zeroing” input that lists all the kWh/d feeds to be reset at midnight, those same feeds will get updated as usual by the original inputs, except at midnight.
I do not think a “null” will do what you want, it will in effect have no impact on the running total as it is “null” you do actually want “0”
No special processes required, for example lets say you have 4 inputs with process lists like so
node 1 : input1
log to feed “ct1_power”
power to kWh “ct1_total”
power to kWh/d “ct1_daily”
node 1 : input2
log to feed “ct2_power”
power to kWh “ct2_total”
power to kWh/d “ct2_daily”
node 2 : input1
log to feed “ctA_power”
power to kWh “ctA_total”
power to kWh/d “ctA_daily”
node 2 : input2
log to feed “ctB_power”
power to kWh “ctB_total”
power to kWh/d “ctB_daily”
add a new input (only updated shortly after midnight) and add all the existing daily feeds that require resetting to it like so
node 99 : Input1
power to kWh/d “ct1_daily”
power to kWh/d “ct2_daily”
power to kWh/d “ctA_daily”
power to kWh/d “ctB_daily”
in fact you probably do not even need to use the “power to kWh/d” process, using a “log to feed” will probably work too and has the benefit of not checking if it’s a new day so time inaccuracies between different machines cannot cause an issue eg
node 99 Input1
log to feed “ct1_daily”
log to feed “ct2_daily”
log to feed “ctA_daily”
log to feed “ctB_daily”
plus if you were to use a reset to zero like so
node 99 Input1
reset to zero
log to feed “ct1_daily”
log to feed “ct2_daily”
log to feed “ctA_daily”
log to feed “ctB_daily”
it won’t even matter what value you send to that input as it will always write a zero to each of the feeds, all it needs is the processing triggered by a new value, any value.
not knowing what existing processing you have or what or where the data originates so I cannot be more specific but hopefully this will demonstrate my suggestion enough for you to adapt it if needed.
Hi paul, that did the trick, thanks for your clear explanaion. It works really well with cron triggering the update. I will try to use schedule + a stable feed to do the same in my local emoncms copy to keep this independent of external systems.
Now on the same subject a different question, i am finding that my power usage of my async energy meter is unusable.
The graphs are shown below and represents output from a energy sensor that posts data only when my EV is charging:
1s top graph (kWh/d) - it’s fine.
2nd from the top (Power) - a real mess.
Zooming a particular area with two charging sessions reveals the issue, emoncms draws a line between the last charge and the begining of the next charge. However zooming in the individual periods each start and end with Power = zero - it is deliberated in my implementation of the energy monitor.
Is this expected? It looks like emoncms is performing some kind of “averaging” before drawing the graphs, depending on the selected zoom.
I think what you are describing is normal behaviour, emoncms records data points and if you were using 1kW an hour ago and the next (hourly) data point said you are using 2kW the graphs would just show a steady increase from 1kW to 2kW over that hour because the graph is just rendering a line between the 2 known datapoints, it has no clue about what happened in between them.
True, however, as you can see on the bottom graphs every session starts and ends with a zero to enforce a straight line with P=0W between both sessions. Notice the strong line coming from zero and ending at zero
My guess is that zooming out the graph makes the zeroes disappear from that view, the graph will draw the “guess line” with whatever values are displayed making it unusable to read and worse with wrong kWh estimates over those periods.
Specifically:
On that period there were two charges, 10,28kWh + 1,13kWh = 11,41kWh total energy.
If you zoom the graph to fit both periods the estimate shows a wrong value of 56.33kWh
The first strange thing is: it’s the same input, the end of the first charge has a ending ZERO. PHPFINA feed doesn’t consider it but PHPTIMESERIES clearly shows otherwise.
Now the case would be simple if there was no ZERO power reading at the begining of the next charge, the graph would draw a line between these two points and that’s is. But zooming in that particular area:
The main cause of discrepancies when using phpfina is that the data only gets saved at the next update after the fixed interval has run, so not every update gets stored.It looks like the trailing zero may have been missed on the phpfina.
I assume that’s a typo and you mean “if there was a ZERO power reading” ?
The problem with the “on-demand” type posting is even the phptimeseries will not have a leading 0 unless your node can predict when the next non-zero value is going to occur and send a 0 before the first non-zero value. That wouldn’t be so difficult in some instances, does your node have some sort of “if greater than zero send data” so it doesn’t send when dormant? If so do you provide timestamps in your post url? if so could you edit that aforementioned test to be some thing like (psuedo code)
get new_value
if new_value > zero
time_now = time()
if last_value == 0
send 0 value with (time_now -10secs) timestamp
send new_value with time_now timestamp
last_value = new_value
emoncms will just see this as getting the “leading 0 value” a little late and the following update on time and record both datapoints (assuming the time subtracted is equal to or greater than the fixed interval if using phpfina).
[edit] Also adding soething like (psuedo code)
If new_value == 0 and last value != 0
time_now = time()
send 0 value with with time_now timestamp
last_value = 0
will also ensure that trailing zero happens if you haven’t got that covered already
These leading and trailing messages are not properly timed in relation to the periodic updates, so thank you for referring that. Actually these actions of sending zeroes exist because i was running into the same problem i want to solve now. And it actually worked with emoncms for some time.
I think i can do some stunts with Node-RED, i’ll report back.
Hi, i have two workarounds for this async data reportings, i’m not truly happy with any of them, however, i’m incapable of understanding how the graph engine work, and all it’s consequences, so i have to use them as they exist and believe it’s the best possible implementation:
1 - Use Node-RED to artificially inject zeroes every time the input is inactive.
2 - When creating graphics select the option “Skip missing data”.
