Emoncms.org API calls - Problem / delay / slow running emoncms.org?

Emoncms Emoncms.org
1

Hi all,

I use the API calls to pull 5 minute data from emoncms.org so that I post to PVOuput in my custom script.

I can see data is hitting my emoncms.org ok from my emonPi, but my API calls are bringing nothing back, ie [].

This has been working fine for weeks/months, but stopped working at around 10:10am this morning.

ie, this is the api call I make every 5 mins (with updated unix times each call)
http://emoncms.org/feed/data.json?id=374691&end=1552993501788&start=1552993201788&apikey=xxxxxxxx&interval=10

What is very weird is that if I call for the last 5 mins right now, it comes back with nothing.
But if I wait 10 mins and make the same API call the figures seem to be there ok. Like there is now a delay.

Any ideas why? Is there an issue on emoncms.org?

Thanks.

2

Sort of confirmed that its taking about 10 minutes for the figures to pull through.

So if I make the call straight away I get []

But after about 10 mins I get the figures through: [[1552994702000,783.94000244141],[1552994712000,793.57000732422],[1552994722000,805.34002685547],[1552994732000,811.76000976562],[1552994742000,803.20001220703],[1552994752000,795.71002197266],[1552994762000,790.35998535156],[1552994772000,785.01000976562],[1552994782000,785.01000976562],[1552994792000,786.08001708984],[1552994802000,781.79998779297],[1552994812000,775.38000488281],[1552994822000,771.09997558594],[1552994832000,767.89001464844],[1552994842000,762.53997802734],[1552994852000,752.90997314453],[1552994862000,743.28002929688],[1552994872000,736.85998535156],[1552994882000,721.88000488281],[1552994892000,710.10998535156],[1552994902000,699.40997314453],[1552994912000,684.42999267578],[1552994922000,672.65997314453],[1552994932000,652.33001708984],[1552994942000,640.55999755859],[1552994952000,626.65002441406],[1552994962000,610.59997558594],[1552994972000,596.69000244141],[1552994982000,579.57000732422],[1552994992000,560.30999755859],[1552995002000,546.40002441406],[1552995012000,535.70001220703]]

Certainly seems like something is on a go-slow on emoncms.org
If you hit refresh between 5 mins and 10 mins you can slowly see new numbers added to the api call result.

Like I said, this started about 10:10am this morning. Before that time, and for weeks/months the api call has been instant.

Hope this helps someone figure out what is happening.

Thanks.

3

Looks like normal service resumed around 11:45am?

4

Looks like there was problems through the night? I’ve got zero usage readings from the api between 2:30am and 6:00am.

Is there some slow running issue on the emoncms.org backend?

Is this the best place to report stuff for investigation?

Cheers.

5

It is the best place yes, we just need to give the right people a heads up with a mention. @TrystanLea, @Gwil, @glyn.hudson or @emrys is there any known issues/delays on the emoncms.org servers that you are aware of?

6

Fab, thanks Paul.

If anyone needs any further info just ask and will try to help.

7

Thanks @Zarch I can confirm there was a slow down on one of the queues between midnight and just after 6am this morning. I realise I had not balanced the queues for a while and that coincided with additional load on that queue, perhaps a large historic data upload. It must have tipped the balance and it started to build up. I have now rebalanced the queues.

My apologies for the resulting data gap on the api that you saw.

1 Like
8

Good stuff, thanks for the update.

Was that work also the reason for the blip between 10:10 and 11:45 yesterday?

I’m wondering if I need to amend my code to be less ‘do it now’ and maybe work 5 or 10 mins in the past to allow for any slight delays/blips? LOL, more coding work! :slight_smile:

9

Yes it was the same reason. One option could be to have the script set its start time to the last valid timestamp of data received, that way it would catch up if this happens again? Ideally you wouldn’t need to of course, the rebalancing of the queues should keep it running on time for now.

1 Like
10

Were there more issues between 09:25 and 11:15 this morning? (3rd April)?

Looks like my script picked up no power and solar figures between those times from the API?

11

Why not just pass the data from the EmonPi (assuming you are logging locally as well as remotely)?

12

Yes, I may have to move to this model… I could run the script on the emonPi directly and pull local data, but think i’d prefer spinning up a new VM instead, keep everything separate… just need to get my VM box working first… on the look out for a cheap Intel NUC. :slight_smile: