EMONCMS API JSON: return feed data across daylight saving time switch

Hi,

I am running a local EMONCMS on a EMONPI and use JSON to extract feed data using the command:
https://localIPaddr/feed/data.json?id=0&start=UNIXTIME_MILLISECONDS&end=UNIXTIME_MILLISECONDS&interval=1800
to extract half hourly power consumption.

This works fine, except when the start and end UNIXTIME are crossing the changeover from/to daylight saving time (DST).
When gaining a DST hour, the extra two half hours are not extracted.
When losing a DST hour, the missing hour is still extracted as if the day had 24 hours.

However, when using my local EMONCMS web front end, the missing and extra hour are correctly reflected in the graphs by showing 50 half hour periods and 46 respectively.

Is there a way to extract the feed data with the correct number of half hour periods when crossing the DST change over?

Many thanks in advance for sharing your insights.

Are you saying that sometimes you don’t get (end-start)/interval+1 data points?

My take is that there is no concept of daylight savings time with UNIXTIME and there’s no notion of local time with UNIXTIME. It’s based on UTC. My understanding is that Emoncms timestamps everything with UNIXTIME.

Here on the US east coast, we will change from EDT to EST at 2am on Nov 4, 2018. Here are the UNIXTIMEs and corresponding local date/time.

1541304000 Sunday, November 4, 2018 12:00:00 AM [GMT-04:00]
1541307600 Sunday, November 4, 2018 1:00:00 AM [GMT-04:00]
1541311200 Sunday, November 4, 2018 1:00:00 AM [GMT-05:00]
1541314800 Sunday, November 4, 2018 2:00:00 AM [GMT-05:00]
1541318400 Sunday, November 4, 2018 3:00:00 AM [GMT-05:00]

1541394000 Monday, November 5, 2018 12:00:00 AM GMT-05:00

Note that UNIXTIME spans 90,000 seconds or 25 hours. Try using the UNIX time converter to develop the sequence for March 11 when the clocks were set ahead. Throughout it all, electric power is logged with UNIXTIME that advances second by second regardless of the games we play with our local clocks.

It’s not that the (end-start)/interval + 1 is missing.
When I retrospectively extract a whole day, I don’t get 23/25 hours, but always 24 when DST changes.

Are you saying that the difference between the UNIXTIME for midnight on the previous day and midnight on the time-change day is always 3600*24 = 86,400?

Yes, it is always 86,400, or should I say ‘appears’ to be always 86,400.

Thanks to your explanations about UNIXTIME not being aware of the games we play with our local clocks, I started to understand what is probably causing the problem - at my end of course: When I converted the UNIXTIME of the extract, I always applied the same TimeOffset taken via the EMONCMS API. That of course adjusts itself automatically when the time-change comes, but when the extract looks at data before and after the time-change, the TimeOffset is of course wrong for UNIXTIMEs before the time-change.

Hence, I think I will have to make sure my code is using different TimeOffsets for before and after the time-change. To be honest, I never looked closely into this because it always worked correctly until I looked at those days when the time does change.

I will try that and let you know what I find. Many thanks for pointing me in the right direction.

Remember, only the clocks go back, we do not go back in time…

" Time is an illusion. Lunchtime doubly so."

Thank you everyone for pointing me in the right direction.
Now that I understand how UNIXTIME works, it’s quite obvious.
And yes, once I adjusted the time offset in my extract code to match the actual offset before and after the change of time, the extract delivers the expected 23 and 25 hours for those two days in a year.