Community
OpenEnergyMonitor

Community

Could not do basic feed API query


(Josenivaldo Benito Junior) #1

Hi,

I am playing with my feeds in Emoncms.org but I could not make a feed API query to work. As per feed API help page, I can reach List.json for a list of my feeds. This works OK! However, same help page says I can reach

https://emoncms.org/feed/data.json?id=xxxxxx&start=1507647600&end=1508179408&interval=10

ID xxxxx -> my KWh feed,
Start and End UNIX timestamp (oct-10-2017 and oct-16-2017)

The answer is:

[]

But the feed claims have data since Aug-18-2017, why I can´t ask for data in the period above?

Remarks:
-> In fact I only need two datapoints for this feed, the one in the start and one for the end timestamp. Is there a better way to get it?
-> if I put wrong timestamps the Emoncms.org complains as expected.

Sorry for this dumb question but I could not spot what I am doing wrong.

Thanks and best regards,
Benito


(Paul) #2

Hi Benito

You are not the first, nor will you be the last to get caught out by this. The API expects the unix time stamps to be in milliseconds, so add 3 zeros to the start and end time stamps and it should work as originally expected, the interval, however is still expected in seconds.


(Josenivaldo Benito Junior) #3

thanks a lot, it works now. Shame on me !


(George (aka Costas the coder)) #4

@pb66 any idea why the API provides an extra data point at 1507647610000 with an end time of 1507647600000 at 10s intervals?


(Paul) #5

What feed type are you using George? I suspect phpfina, I do not have much experience with that feed type

Taking a quick peek at the code I would expect it is due to the “less than or equal to” test at L319 of the phpfina engine (assuming that’s the engine you are using)

since the $time variable is incremented at the beginning of each loop rather than at the end, each loop is processing the next incremented $time so the “or equal to” results in an additional iteration of $end + $interval.

You could try editing that line to read

        while($time<$end)

however I do not know what (if any) possible ramifications that might have elsewhere if other parts rely on that error.


(Paul) #6

Not at all, we have all done it. It is not obvious or intuitive to use unixtime in millis, especially as the additional resolution has no effect, the default/maximum resolution in emoncms is whole seconds, the extra zero’s just get removed.


(Josenivaldo Benito Junior) #7

Yes, it is PHPFINA. I can see the same behavior, one extra data point after $end period.


(Paul) #8

Looking at the commit history it looks like there were changes made in Mar '15 that might have knocked this code out of wack. Prior to that the $time var was incremented after reading the data point value from the data file,


(George (aka Costas the coder)) #9

Yes that’s the one.

I guess @TrystanLea will need to fix the bug on the EmonCMS.org server.


(Trystan Lea) #10

Thanks for spotting @pb66 @GeorgeB , Il have to spend some time double checking this to ensure there are no knock on effects but I take note, thanks.


(Paul) #11

Thanks @Trystan, It’s the same on both local and emoncms.org so I have opened an issue on both repos as a reminder


(Clive Duffy) #12

All,

I submitted the following pull request last night as the PHPFina get_data_dmy() was not returning the last full data point and also noticed the additional data points from get_data() detailed above.

Using an example of a monthly interval, I also found that where the data started to be collected within that interval that this interval is overlooked. e.g. month data starting 1st July, but the data was first recorded 15th July, is a null data point. An equivalent happens if requested end is after the last captured data point.

I propose that the get_data*() and get_average*() functions are expanded to include a ‘partial’ option, meaning that when requested it will return these first/ last entries and if appropriate could even place them against the datetime for that the start/ end, 15th July with the data point for the example above.

I’m willing to write something up, but wanted confirmation and thoughts before starting. We could also resolve Paul’s request as the same time.

Clive.


Email report complains has not enough data, while it has
(Paul) #13

Is this possibly related? Email report complains has not enough data, while it has