Having trouble with dates and times in data export (is the options dialogue in BST while the backend in UTC?)

Hi
I do a lot of data exporting from emoncms (both online and locally).

A recent problem is that if I ask data to start at midnight, I actually get data from 11pm the previous day. I think that the front-end export is using BST, while the backend is still on GMT/UTC?

Can you add a check box (“Tick to use GMT.UTC”) to make the front end specify local time or UTC time, please.

emonPi is working awesome otherwise :slight_smile:
Many Thanks.

Oh there are big discussions on GitHub about timezones :slight_smile:

When you say midnight, do you mean midnight local time? How do you specify the time in the request?

Have you set your timezone in the user profile?

Internally, emonCMS uses UNIX time - i.e. number of seconds since the epoch.

2 posts were split to a new topic: EVESE and Demandshaper time issue. Timezone?

Hi,

Thanks for the reply.
My timezone is UTC.

Even when I ask to export the data in Unix time, it still gives me an export file that starts one hour earlier. For instance 00:00 on Thursday morning gives me a file that starts at 23:00 Wed night.

As I mentioned it feels like the front end time/date chooser is using BST but the data is in UTC.

Thanks

Can you answer this? There are many interfaces to export data through.

00:00+01:00 (BST) is the same as 23:00+00:00 (UTC) - it is the same number of seconds since the epoch.

Thanks for the reply. This is how I usually do it.

As there is no TZ here, I would expect it to use the user timezone.

Which is why it is doing exactly what you ask it to :grin:.

Set your user timezone correctly (just click on it - not very clear you can do that) and you might get what you expect.

Hi,
My timezone is UTC, and I set the clock on the interface to start from 2am. But it gives me data from 1am UTC?

Your profile says Cardiff so the local time (TZ) to you is UTC+1

It looks as if the export is using the browser TZ. Why not set the user TZ correctly? If you do, what do you get?

This just demonstrates the problem with any system when you display times without a TZ. The string is meaningless without it.

What timezone is on your user profile?

Screenshot 2020-04-20 at 16.56.57

I am still reading the threads but I agree that an indication of the time zone in question is needed. Perhaps time/date data export could be three columns ‘EpochTime/TZ time/Feed’. At least this way the EpochTime is always attached.

Everything is working as expected.

  • UI is in your computer TZ. eg. BST (UTC+1)
  • Your profile is in UTC TZ.

1- Export converted to YMDhms comes from profile TZ hence you select in UI TZ and have export data in profile TZ.
2 - Export with no conversion comes always in UTC seconds + offset on the UI. This offset field is automatically filled with the seconds difference for you profile TZ to UTC.

UTC is not a physical location, clocks do not change in UTC.
Why is your profile in UTC?

1 Like

'cos UTC sounds cool :wink:

I see what you are saying that the browser javascript time setting and my profile on the server time setting are out of sync. Hence one asks for data at 2am and the other correctly sends data from 1am. Nothing wrong there, except the human in the middle. I changed my profile now to London time.

Could be worth adding a local TZ time setting to the dialog box for the Feed.Download dialog box also just like you did for the graphs (although that’s more work ;-(.

Thanks again for the explanation.

1 Like

That isn’t obvious to a user of the UI and there is no means of selecting if the export is converted or not.

As I said, the export should indicate it is UTC by including the timezone as per ISO 8601. Without it, it is deemed to be local time.

Unfortunately, the date/time the feed is collected at is related to the user timezone not the sensor generating the data. You may find there is an hours gap in your data, depending on how it is collected.

1 Like