Hi,
I’ve set up my own server to host emoncms, and I’m sending data to it from an emonpi.
I’ve modified the emonpi firmware so that it takes a measurement every 0.5s (sounds mad, good reason tho), and the batch of data gets send up to emoncms very nicely every 10s or so, I can see the json string with the expected timestamps in the config view. The data is stored in 4 feeds (realtime, phptimeseries).
However, when I go to download the data (through the emoncms csv export) the csv file is in 1s intervals. I cycled a load at measured intervals to confirm that there is actually only 1 datapoint/s, so it’s not a formatting issue. When I deselected the ‘Excel’ format select box, the csv file only has 1 row it in.
Can anyone help me see how to get the full-resolution data out please? In effect I want to set an ‘interval’ value of less than 1s.
Thanks,
Martin
I think you’re going to struggle with using emoncms and data at that frequency. Although emonhub passes the data to emoncms as a unixtime float, the emoncms input immediately casts that timestamp to an integer, so only whole seconds are used.
The type of feed you are using also records time in whole seconds as an 32bit integer too.
Even at 1 second intervals you may well get some very erratic calculations and processing since the timestamps are cast to integers you will find that 1495067945.000 and 1495067945.999 are treated as the same point in time where as 1495067945.999 and 1495067946.001 would be considered 1 full second apart.
You may want to look at InfluxDB. It has the capability to ingest large quantities of data at high speed at nanosecond precision. It would handle your 2 s/s measurement rate easily. It uses an SQL-like language, so is easy to use if you’re at all familiar with SQL. Install/setup is easy too. Runs on the RPi as well as x86 boxes, and Docker too. (and no, I’m not affiliated in any way, with InfluxData. I just use the product and am very satisfied with it)
Thank you both for the advise, I can make do with 1/s for now and then I’m going to look at InfluxDB. It is new to me so thank you for that tip.