Error fetching values from emoncms

Phani_Dutt · 2016-11-24T19:10:41.766Z

Hello,

I am getting the error while fetching the values from the emoncms when providing the below link:

http:///emoncms/feed/data.json?id=4&start=1480012183&end=1480012198.

When I am providing the above link and try to fetch the values from this it shows an error message that is “{“success”:false,“message”:“request end time before start time”}”

I have a requirement to fetch the energy values for the start and the end unix time stamps as well as the time stamps also.Can you please help me with that.

Thank you.

pb66 · 2016-11-24T22:23:58.615Z

Try using unix millisecond timestamps eg

http:///emoncms/feed/data.json?id=4&start=1480012183000&end=1480012198000

Phani_Dutt · 2016-11-25T12:24:53.761Z

Thank you Paul( pb66). Your answer worked for me

But I have a doubt why we have to take the millisecond unitime values for the values that we obtained?

Thank you.

pb66 · 2016-11-25T12:42:58.906Z

No problem!

Phani_Dutt:

But I have a doubt why we have to take the millisecond unitime values for the values that we obtained?

I totally agree, I’m not sure why they need to be milliseconds.

Phani_Dutt · 2016-11-25T13:08:28.392Z

pb66:

http:///emoncms/feed/data.json?id=4&start=1480012183000&end=1480012198000

The answer that you povided is correct but we want to take the exact energy value for the unixstart time and end time that is [1480012183000=energy1 and 1480012198000=energy2].

But the link provides the values as shown below:

[[1480012183000,33698808],[1480012193000,33705836],[1480012203000,33707248]] for the link provided by you above.

Please can you help me with that.

Thank you

pb66 · 2016-11-25T13:32:30.092Z

You can only specify a start point and end point. The api will return the data points between those times. To my knowledge, there is no api to tell you the value at any specific time.

The main reason behind this is the data is updated at specific points in time and effectively dumb about what happened inbetween.

If you are measuring 2kW at 2pm and 4kW at 4pm, that does not necessarily mean it was 3kW at 3pm. in fact it could have been 0kW from 2.01pm until 3.59pm or it could have been 100kW from 2.30pm until 3.30pm, there is no globally safe way of “guessing” the value at 3pm.

Even with accumulative values eg 9kWh at 9am and 11kWh at 11am, you cannot assume it was 10kWh at 10am, it may have shot up to 11kWh a minuite after 9am and sat there for 2 hours or not moved for 2 hours and then shot up just before 11am, no way of telling.

So the best the data can tell you is what is was at the datapoints closest to your time for you to decide. If you are using fixed interval feeds you may be able to force a “near as damn it” value e.g. if you are using a 10s fixed interval you could ask for all datapoints betwenn 5secs before and 5 secs after your chosen time and that should return the one datapoint closest to your time +/- 5secs and repeat that for the second “energy2” datapoint.

Bill.Thomson · 2016-11-25T17:41:32.683Z

pb66:

I totally agree, I’m not sure why they need to be milliseconds.

Isn’t that because javascript keeps time in milliseconds?

pb66 · 2016-11-26T16:28:16.410Z

That may well be the underlying reason but some scaling could make the api more user friendly.

The inputs and feeds use whole seconds, I think the data returned can only be to whole second granularity despite being returned in milliseconds, just seems like a whole lot of extra zeros going to and fro that add little more than confusion.

Bill.Thomson · 2016-11-26T16:31:02.935Z

pb66:

just seems like a whole lot of extra zeros going to and fro that add little more than confusion.

You’ve got that right!

Phani_Dutt · 2016-11-30T15:10:44.326Z

hello

@pb66 When I am giving this link it shows the energy values in the browser but when i try fetch it from a perl script it shows an empty brackets like this [].What is the error in that link. Can you please help me with that.
http://server/emoncms/feed/data.json?apikey=&id=26&start=1480518361000&end=1480518371000
Thank you

borpin · 2019-02-08T17:15:48.076Z

Phani_Dutt:

but when i try fetch it from a perl script it shows an empty brackets like this

I know this is an old thread, but I had exactly the same issue and came across this answer; firstly the miliseconds and then the empty brackets.

I happened to try exactly the same URL again (as the empty bracket URL) and it returned data. I suspect it is the feedwriter buffer that is the issue if you are looking for very recent data.

Could anyone confirm this might be the reason? I noticed it the other day when looking at a graph where there was no recent data.

pb66 · 2019-02-08T20:35:14.034Z

If it is the feedwriter (which I don’t know for sure but would hope not, as the feeds are also held in redis for more recent data) as you suspect it should be hit or miss for every other api call as the feedwriter runs every _n_Secs (I think it’s 60 but I’m not 100%).

Can you try a browser or wget (or maybe less so curl) to see if it’s a perl only or a more global issue?

borpin · 2019-02-10T12:18:49.750Z

I’ll not get a chance today, but the way to test would be a quick python script to check for the last 10s every 10s; I think you may get empty arrays 5 times out of 6.

It was a simple browser call I used.

Um,

github.com

emoncms/emoncms/blob/69ad755fe6f1ff35ecc6dd673a1804162fd72d3d/default.settings.php#L52-L52

    
      
          ,'sleep' => 600          // Number of seconds to wait before write buffer to disk - user selectable option

On non-emonpi/EmonSD it is defaulted to 600!

That explains a lot!!!

Checking a graph, no figures appear in the graph that have not been written to disk (i.e. still in ~~feedwriter~~ redis buffer).

pb66 · 2019-02-10T15:00:29.934Z

borpin:

That explains a lot!!!

Indeed it does, I had no idea it was 10mins, wow!

That was seemingly changed back in Dec 2015, prior to that is was definitely 60s so I’m not entirely nuts, just a bit out of date

borpin:

but the way to test would be a quick python script to check for the last 10s every 10s; I think you may get empty arrays 5 times out of 6.

I don’t run anything with feedwriter enabled so i cannot test, that’s why I’m a bit hazy on it. Everything I know about the low-write stuff is learnt through helping users here and looking at the code and commit notes.

borpin · 2019-02-10T16:36:59.250Z

pb66:

I don’t run anything with feedwriter enabled

I’m only really running it so I can see what is going on and spot any issues!

pb66 · 2019-02-13T12:13:37.796Z

further to our comments above, I discovered that the emonpi remains at “60” but just the non-emonpi was changed to “600” in the commit I linked above.

github.com

emoncms/emoncms/blob/9.9.6/default.settings.php#L49-L53

    
      
          // Redis Low-write mode
          'redisbuffer'=>array(
              'enabled' => false      // If enabled is true, requires redis enabled and feedwriter service running
              ,'sleep' => 600          // Number of seconds to wait before write buffer to disk - user selectable option
          ),

github.com

emoncms/emoncms/blob/9.9.6/default.emonpi.settings.php#L49-L53

    
      
          'redisbuffer'=>array(
              'enabled' => true      // If enabled is true, requires redis enabled and feedwriter service running
              ,'sleep' => 60          // Number of seconds to wait before write buffer to disk - user selectable option
          ),