Have questions now about retrieving data…
(1) how do I decode my log data from backups or an old SD card with an emosOS image so I can copy them to text files for analysis?
(2) Is there a way to download all the data from the online hosted emoncms database without using the graphical interface? I have a lot of points, and extracting the data a month at a time is tedious…
Thank you for any help.
Hello @SteffanCook, downloading data from emoncms.org or another hosted instance is easy if you have a local Pi running the latest emonSD. You can either use the Emoncms sync module available from the setup menu on the emoncms installation running on the Pi or the backup script detailed here https://github.com/emoncms/emoncms/blob/master/docs/Backup.md.
Hi @TrystanLea.
Thanks for your message. I have been trying out the various ways of importing backup data this morning (detailed Doing a 'Full Update' to the emonPi caused it to 'brick').
What I really wanted though was a way to turn those .DAT files (in the backups) into readable CSV data. I don’t want to use the graphical interface module on emonCMS to pull out the data, I want to get the data from the backup. THe graphical interface is a bit cumbersome (I could kill those calendar UI) and it tends to be limited to a few thousand data sets, and I have tens of thousand size data sets to get out so that means many attempts.
Any straightforward way to decode those .DAT files available?
Cheers. Diolch
Steffan
This page gives an overview of how the data is stored in the phpfina format:
https://learn.openenergymonitor.org/electricity-monitoring/timeseries/Fixed-interval
Its a while since I tried it, but I think this script will read out the contents of a phpfina file as space separated:
https://github.com/emoncms/usefulscripts/blob/master/enginereaders/phpfina.php
You will need to set the directory and feedid at the top before running the script from command line with php:
$ php phpfina.php@TrystanLea - the phpfina.php file does work and it does print out the data values from the .dat files.
For ease of use, could you add some code that puts those $time and $value results into a text file of the same name in the same folder. (sadly I don’t know enough code to write this)
Thanks for any help.