API query result format changed? 9:45am 17/3/22

Emoncms
1

Hi all,

I’ve got a script that has been running for years that pulls my usage every 5 mins from emoncms.

Has been working flawlessly until 9:45 today.

Uses a simple call to get data points.
http://emoncms.org/feed/data.json?id=999999&apikey=xxxxx&start=1647510301871&end=1647510601871&interval=10

Pre 9:45am
[[1647510001000,1879],[1647510011000,1879],[1647510021000,1883],[1647510031000,1896],[1647510041000,1895],[1647510051000,1891],[1647510061000,1894],[1647510071000,1900],[1647510081000,1887],[1647510091000,1903],[1647510101000,1896],[1647510111000,1900],[1647510121000,1900],[1647510131000,1902],[1647510141000,1904],[1647510151000,1900],[1647510161000,1906],[1647510171000,1922],[1647510181000,1922],[1647510191000,1920],[1647510201000,1921],[1647510211000,1933],[1647510221000,1936],[1647510231000,1940],[1647510241000,1928],[1647510251000,1929],[1647510261000,1935],[1647510271000,1929],[1647510281000,1932],[1647510291000,1951],[1647510301000,1954]]

Post 9:45am
[[1647510301000,2266.9599609375],[1647510311000,2264.820068359375],[1647510321000,2279.800048828125],[1647510331000,2273.3798828125],[1647510341000,2276.590087890625],[1647510351000,2277.659912109375],[1647510361000,2271.239990234375],[1647510371000,2275.52001953125],[1647510381000,2272.31005859375],[1647510391000,2274.449951171875],[1647510401000,2274.449951171875],[1647510411000,2288.360107421875],[1647510421000,2277.659912109375],[1647510431000,2294.780029296875],[1647510441000,2301.199951171875],[1647510451000,2300.1298828125],[1647510461000,2296.919921875],[1647510471000,2290.5],[1647510481000,2297.989990234375],[1647510491000,2321.530029296875],[1647510501000,2303.340087890625],[1647510511000,2316.179931640625],[1647510521000,2321.530029296875],[1647510531000,2306.550048828125],[1647510541000,2283.010009765625],[1647510551000,2306.550048828125],[1647510561000,2305.47998046875],[1647510571000,2335.43994140625],[1647510581000,2315.110107421875],[1647510591000,2327.949951171875],[1647510601000,2330.090087890625]]

What where integer/whole values are now not.
Has something changed behind the scenes?

My code doesn’t like this change… :joy:

1 Like
2

Possibly related to EmonCMS.org being updated to version 11 this morning.

2 Likes
3

That’s strange, will see if I can make sense of why this is

1 Like
4

Thanks @TrystanLea
I’ve just wrapped a bunch of my code with int() so I’m hopefully back up and running for a bit!! :joy:

1 Like
5

It’s quite bizzare as I cant see anything that would affect rounding like this, as far as I was aware it was always returning data with lots of decimal places, but there must be something subtle that Im missing. Will keep looking, good to hear that you have sorted it on your end in the meantime.

6

A bit of an update @TrystanLea A bit of my mistake.

You’re right, the API call has been supplying float numbers in the past.
It just seems some of my feeds come back as either integer or float, hence I got confused whilst troubleshooting.

So as you were there, no problem.

BUT, I have found something else that does seem new. I’m getting some null results returned.

ie
[[1647510301000,2266.9599609375],[1647510311000,2264.820068359375],[1647510321000,2279.800048828125],[1647510331000,2273.3798828125],[1647510341000,2276.590087890625],[1647510351000,2277.659912109375],[1647510361000,2271.239990234375],[1647510371000,2275.52001953125],[1647510381000,2272.31005859375],[1647510391000,2274.449951171875],[1647510401000,2274.449951171875],[1647510411000,2288.360107421875],[1647510421000,2277.659912109375],[1647510431000,2294.780029296875],[1647510441000,2301.199951171875],[1647510451000,2300.1298828125],[1647510461000,2296.919921875],[1647510471000,2290.5],[1647510481000,2297.989990234375],[1647510491000,2321.530029296875],[1647510501000,2303.340087890625],[1647510511000,2316.179931640625],[1647510521000,2321.530029296875],[1647510531000,2306.550048828125],[1647510541000,2283.010009765625],[1647510551000,2306.550048828125],[1647510561000,2305.47998046875],[1647510571000,null],[1647510581000,null],[1647510591000,null],[1647510601000,null]]

It’s this that is causing my script the problems and this looks new behaviour since around 9:45.
I can’t find any reference to ‘null’ results in the last 3 days leading up to 9:45.

I’m happy to send the API calls via DM if they would help?

Cheers, Mick

7

Aha, ok, if you enter add &skipmissing=1 to the URL it will get rid of that. I realise now that I’ve changed the default behaviour to including missing values for the timestamps range that is requested. I will look at that a bit closer to see if the old default can be reinstated.

1 Like
8

Thanks @TrystanLea
Will make that change. :+1:

1 Like
9

My PV output upload (pulls data from emoncms, post to PVOutput) stopped working for the same reason.

Adding skipmissing to the headers resolved it.

Just need to work out how I go back and fetch 18 days of missing data.

10

Sorry about that @greentangerine how did it end up being 18 days missing?

11

I only noticed today that the cronjob that runs every five minutes on the emonpi stopped working on the 4th March.

The data should still be there on emoncms, just hasn’t been uploaded to PVOutput. I’ve also learnt how to code Python to be more resilient to NoneType exceptions (you wouldn’t think that coding has been my paid job for 34 years!)

I guess I just need to copy my script and modify it to query emoncms for the missing interval as a one-off. Will get around to that over the weekend.

1 Like