Extracting data from emoncms

mcockerell · 2023-07-24T13:47:03.997Z

I’m trying to extract data from emoncms to reconcile against my smart meter readings.
At the moment I am attempting to do this via the CSV export on the feed interface, but seem unable to achieve what I want:

I want to extract data values at 8:00 am every day for a month.
I can set this up in the input boxes as, for example, from 01/06/2023 08:00:00 to 30/06/2023 08:00:00, period ‘Daily’ but the data returned is always at midnight (00:00:00) - the same thing happens if I try to extract the CSV data using the Graph interface.
Is this a design limitation of the interface, or am I doing something wrong? Is the only way to get the data that I need to set the interval to hourly and post-process the CSV output to discard all the unwanted data?

[Update] I’ve now found the page for the Feed API again and am able to achieve what I want this way, which will probably be easier for me as I can script the extract. However, it seems to require the date in mm/dd/yyyy format, although it returns dd/mm/yyyy format - this seems a little inconsistent.

Thanks,
Martin

PS I was also looking for details on the PHPFina and PHPTimeSeries datasets, and the links in the Emoncms Core Concepts documentation point to the old ‘Learn’ pages - I’ve been unable to find the new location in the Documentation pages.

alexandrecuer · 2023-08-11T17:45:00.324Z

mcockerell:

PS I was also looking for details on the PHPFina and PHPTimeSeries datasets

I think they are here

github.com

//github.com/emoncms/emoncms/tree/master/docs/timeseries

Robert.Wall · 2023-08-11T19:29:00.634Z

mcockerell:

I’ve been unable to find the new location in the Documentation pages.

@Gwil Have these been left out by accident or design?

Gwil · 2023-08-12T13:00:11.390Z

@TrystanLea intentionally moved some of the more in-depth EmonCMS documentation that was in Learn to the EmonCMS GitHub repository.

We will need to fix those broken links.

Robert.Wall · 2023-08-12T13:08:44.642Z

Gwil:

We will need to fix those broken links.

My problem was, I couldn’t find even a broken link. Hence the question.

mcockerell · 2023-08-12T16:02:07.328Z

Here: https://docs.openenergymonitor.org/emoncms/coreconcepts.html#inputs-devices-input-processing-and-feeds

2023-08-12 16_57_10-Emoncms Core Concepts — OpenEnergyMonitor 0.0.1 documentation — Mozilla Firefox693×90 8.71 KB

Clicking ‘PHPFina’ or ‘PHPTimeSeries’ displays ’ Learn has now moved to OpenEnergyMonitor: Docs’ but the link doesn’t take you to the relevant section - if not broken then not very helpful

borpin · 2023-08-12T18:41:34.070Z

Gwil:

intentionally moved some of the more in-depth EmonCMS documentation that was in Learn to the EmonCMS GitHub repository.

So not part of the Documentation? If so that is utterly daft.

Gwil · 2023-08-12T21:41:53.024Z

Moving it from Learn pre-dated the current Docs site.

Most sections of the current Docs is sourced from the Docs folder in the relevant GitHub repository. It might just have been missed out in error. I will take a look.

TrystanLea · 2023-08-16T07:19:02.262Z

Yes it would be good to move this back into docs.

borpin · 2023-08-16T11:52:03.613Z

mcockerell:

I can script the extract. However, it seems to require the date in mm/dd/yyyy format, although it returns dd/mm/yyyy format

If scripting, I suggest using UNIX format as it excludes things like DST issues.

Check your locale settings as I’d expect the input to be determined by that.

Again, getting the output as a Unix format stops errors in DST and Timezone.