Emoncms daily, weekly, monthly kWh and averaging

Submitted by TrystanLea on Sun, 24/04/2016 - 17:10

I’ve been working on improving emoncms support for timezone correct daily, weekly & monthly kWh data and also fully calculated average (mean) temperature, humidity, power etc.

Its now possible to access, view and export to CSV these values from the new emoncms graph interface as a start.

This screenshot shows average daily power data calculated from a power feed overlayed against daily kWh as calculated from a cumulative kWh feed. Both approaches arrive at the same values (apart from a few small differences if there are outages). But it illustrates that the calculation of daily mean is aligned correctly.

The request type can be selected from the type drop down menu and includes types:

Interval - specify a periodic fixed interval at which to request datapoints i.e a data point every 3600 seconds.
Daily - daily data correct to timezone set in user profile
Weekly - weekly data correct to timezone set in user profile
Monthly - monthly data correct to timezone set in user profile
To return the average for the period selected the average tickbox needs to be ticked, otherwise the value at the timestamp of the start of each interval will be returned - which is what is needed in the case of calculating kWh data from cumulative kWh feeds.

On request the averages are calculated on the fly in the background, the result is cached so that next time an average is requested the whole feed does not have to be processed. Because of the way this works at the moment ‘reload’ needs to be clicked twice once to register the feed for averaging and second to display the result. I will be working on a way to make this a more seamless experience.

While most of the implementation for this is currently working on the emoncms master branch the part that registers the feed for background processing of averages is not yet complete and so the averaging will only work up to 7 days there while on emoncms.org averages covering the full feed can be returned. There are a couple of things left to do to fully complete this but for anyone keen to test its present state this is now available on emoncms.org and the master branch for testing.

1 Like

Eric - where did you find this info on the OEM site? I’ve been looking for a process that can do averages. Jon

original post from yesterday on live forum

Should have linked back, sorrry Jon

I must be spending too much time on the new forum - I completely missed this on the old forum! Thank you!

1 Like

I need your help.
When I flag the average column, in order to obtain a daily average graph, the graphs disappear and also all the values in CSV format and the statistical information in the row of that particular feed. Where is the mistake?

If you click the W for Weekly or M for Monthly does the image re-appear?
13 AM copy

 

Can you include a screen shot of the entire graph page? (you should now have privileges to post an image)

Ok, here my problem.
I plot a graph with my hourly temperatures bey flagging the feed: it works.

Then I select “daily” and reload the graph: temperatures at 00.00 a.m. for each day are plotted (not daily averages!).

If I flag “average”, graph and statistical values disappear.
It only reappear if I undo the “average” flag.

I add only one thing: if I Oper CSV values, I see that at the begging of my data series there are a lot of missing values (null). I don’t know if this is the cause of the problem.

Nobody able to help me?

Mario - sorry for disappearing. I’ve been trying the same thing and got equally odd things (chart disappears). And I did get averages that didn’t make sense. I did see the order you click on things does change the output (i.e., clicking Averages first versus change Type to Daily)

So I grabbed the raw data and calculated my own averages for the day. Now I know the right number!

Anyway the bottom line is I could not determine what was going wrong. Trystan @TrystanLea can you assist?

In my case, if I select Average, graphs disappear, not depending by type (daily, monthly, fixed…), they always disappear

do you start with “D” for Daily, “W” for weekly? or “M” for monthly (upper left buttons)? I just want to make sure we are starting the same way…

Jon, it does not mind the order I select fields. There is only one constant: when I flag “average” and then “reload”, graph disappears. Always, any case or type has been previously selected

Have you tried clicking reload twice?

What type of feed are you using this on? I believe it only works for phpfina.

Since the data is calculated on the fly, are you leaving it long enough to crunch the numbers and present a graph?

Yes, I click “reload” twice, I use phpfina, I wait one -two minutes, I think it is sufficient for a hourly time series of five days…

I remember you, graphs disappear and statistical values under graph go to zero

I wonder if I am the he only one with this problem…

I have the same issue but I am not sure of the fix. Next time the graph goes blank, try clicking on the “M” for month.
17 PM

Does the display come back?

Hello, could you see if there are any errors returned in the request to the feed/average API using developer tools?
Do averages of smaller time frames work? hourly or perhaps 6 hours?

I tried to get your suggestions, but now this is what appears to me on emoncms page…
Where are the average option and the statistical values under the graph?
Now, if you select a certain time interval only single data at that particular interval are plotted, no average values at all…