Hey,
Sometimes emoncms stops logging data. Either something goes haywire with the box itself and it stops recording, or the network connection goes down and emoncms.org stops receiving. So, I have all the data, but sometimes it’s split over the two instances (local or emoncms.org). I would love to be able to sync the two, even if that means downloading everything and writing a script to overwrite the NaNs.
Is there an easier way?
Bruce
What version? What hardware?
Can you click on the button Copy as Markdown next to Server Information on the Admin page and paste into a reply here please (no further formatting required).
I’m sure it’s just benign stuff caused by my own development. Restarting the wrong daemon without restarting others breaks the connection and if I don’t notice for a few hours I lose a bunch of data. In this case I’m only interested in correcting the existing gaps in my logs.
However, here is the result of your request:
Model :- Raspberry Pi 2 Model B Rev 1.1 - 1GB (Sony UK)
Serial num. :- 9C7F0B26
Temperature :- 34.70°C - 35.8°C
emonpiRelease :- - File-system :- read-write
Oh, OK. yes possibly. There is the sync module, but I’m not really clear on what it does exactly @TrystanLea shoudl be able to advise.
Thanks for the pointer. I tried the sync module but the feed-list API returns this:
<b>Notice</b>: Trying to get property 'host' of non-object in <b>/home/pi/sync/sync-module/sync_controller.php</b> on line <b>51</b><br />
<br />
<b>Notice</b>: Trying to get property 'apikey_read' of non-object in <b>/home/pi/sync/sync-module/sync_controller.php</b> on line <b>51</b><br />
<br />
<b>Warning</b>: file_get_contents(/feed/listwithmeta.json?apikey=): failed to open stream: No such file or directory in <b>/home/pi/sync/sync-module/sync_controller.php</b> on line <b>51</b><br />
<br />
<b>Warning</b>: Invalid argument supplied for foreach() in <b>/home/pi/sync/sync-module/sync_controller.php</b> on line <b>83</b><br />
{"solar_power":{"local":{"exists":true,"id":2,"tag":"","engine":"5","datatype":"1","start_time":1452338220,"interval":10,"npoints":12903083},"remote":{"exists":false,"start_time":"","interval":"","npoints":""}},"vrms":{"local":{"exists":true,"id":3,"tag":"","engine":"5","datatype":"1","start_time":1452338220,"interval":10,"npoints":12903083},"remote":{"exists":false,"start_time":"","interval":"","npoints":""}},"solar_energy":{"local":{"exists":true,"id":6,"tag":"","engine":"5","datatype":"1","start_time":1452338720,"interval":10,"npoints":12903033},"remote":{"exists":false,"start_time":"","interval":"","npoints":""}},"temperature":{"local":{"exists":true,"id":9,"tag":"","engine":"5","datatype":"1","start_time":1452338760,"interval":10,"npoints":12903029},"remote":{"exists":false,"start_time":"","interval":"","npoints":""}},"house_power":{"local":{"exists":true,"id":12,"tag":"","engine":"5","datatype":"1","start_time":1452431490,"interval":10,"npoints":12893756},"remote":{"exists":false,"start_time":"","interval":"","npoints":""}},"house_energy":{"local":{"exists":true,"id":13,"tag":"","engine":"5","datatype":"1","start_time":1452431490,"interval":10,"npoints":12893756},"remote":{"exists":false,"start_time":"","interval":"","npoints":""}},"grid_power":{"local":{"exists":true,"id":14,"tag":"","engine":"5","datatype":"1","start_time":1452431890,"interval":10,"npoints":12893716},"remote":{"exists":false,"start_time":"","interval":"","npoints":""}},"grid_energy":{"local":{"exists":true,"id":15,"tag":"","engine":"5","datatype":"1","start_time":1452431910,"interval":10,"npoints":12893714},"remote":{"exists":false,"start_time":"","interval":"","npoints":""}},"livingroom_raw_temperature":{"local":{"exists":true,"id":26,"tag":"","engine":"5","datatype":"1","start_time":1480886240,"interval":10,"npoints":10048280},"remote":{"exists":false,"start_time":"","interval":"","npoints":""}},"boiler_raw_temperature":{"local":{"exists":true,"id":27,"tag":"","engine":"5","datatype":"1","start_time":1480886280,"interval":10,"npoints":6507101},"remote":{"exists":false,"start_time":"","interval":"","npoints":""}},"livingroom_temperature":{"local":{"exists":true,"id":28,"tag":"","engine":"5","datatype":"1","start_time":1480886310,"interval":10,"npoints":2562650},"remote":{"exists":false,"start_time":"","interval":"","npoints":""}},"boiler_temperature":{"local":{"exists":true,"id":29,"tag":"","engine":"5","datatype":"1","start_time":1480886320,"interval":10,"npoints":2562867},"remote":{"exists":false,"start_time":"","interval":"","npoints":""}},"bedroom_temperature":{"local":{"exists":true,"id":33,"tag":"","engine":"5","datatype":"1","start_time":1481843100,"interval":60,"npoints":1658764},"remote":{"exists":false,"start_time":"","interval":"","npoints":""}},"bedroom_temperature_external":{"local":{"exists":true,"id":34,"tag":"","engine":"5","datatype":"1","start_time":1481843100,"interval":60,"npoints":1658764},"remote":{"exists":false,"start_time":"","interval":"","npoints":""}},"bedroom_humidity":{"local":{"exists":true,"id":35,"tag":"","engine":"5","datatype":"1","start_time":1481843160,"interval":60,"npoints":1658763},"remote":{"exists":false,"start_time":"","interval":"","npoints":""}},"emonth_battery":{"local":{"exists":true,"id":36,"tag":"","engine":"5","datatype":"1","start_time":1481843160,"interval":60,"npoints":1658763},"remote":{"exists":false,"start_time":"","interval":"","npoints":""}},"living_room_temperature_raw_2":{"local":{"exists":true,"id":37,"tag":"","engine":"5","datatype":"1","start_time":1504821080,"interval":10,"npoints":4163962},"remote":{"exists":false,"start_time":"","interval":"","npoints":""}},"air_pressure":{"local":{"exists":true,"id":40,"tag":"","engine":"5","datatype":"1","start_time":1504821510,"interval":10,"npoints":7654753},"remote":{"exists":false,"start_time":"","interval":"","npoints":""}},"emonth5:rssi":{"local":{"exists":true,"id":41,"tag":"","engine":"5","datatype":"1","start_time":1505829150,"interval":10,"npoints":7553979},"remote":{"exists":false,"start_time":"","interval":"","npoints":""}},"dew_point":{"local":{"exists":true,"id":44,"tag":"","engine":"5","datatype":"1","start_time":1509614580,"interval":60,"npoints":1195906},"remote":{"exists":false,"start_time":"","interval":"","npoints":""}},"kitchen_cput0":{"local":{"exists":true,"id":45,"tag":"Node kitchen","engine":"5","datatype":"1","start_time":1522530640,"interval":10,"npoints":2342665},"remote":{"exists":false,"start_time":"","interval":"","npoints":""}},"lounge_cput0":{"local":{"exists":true,"id":46,"tag":"Node lounge","engine":"5","datatype":"1","start_time":1522530660,"interval":10,"npoints":5883838},"remote":{"exists":false,"start_time":"","interval":"","npoints":""}},"emonpi_cput0":{"local":{"exists":true,"id":49,"tag":"Node emonpi","engine":"5","datatype":"1","start_time":1522531190,"interval":10,"npoints":5883785},"remote":{"exists":false,"start_time":"","interval":"","npoints":""}},"fridge_temperature":{"local":{"exists":true,"id":50,"tag":"Node kitchen","engine":"5","datatype":"1","start_time":1563059850,"interval":30,"npoints":610306},"remote":{"exists":false,"start_time":"","interval":"","npoints":""}},"fridge_humidity":{"local":{"exists":true,"id":51,"tag":"Node kitchen","engine":"5","datatype":"1","start_time":1563059940,"interval":30,"npoints":610303},"remote":{"exists":false,"start_time":"","interval":"","npoints":""}},"freezer_temperature":{"local":{"exists":true,"id":52,"tag":"Node kitchen","engine":"5","datatype":"1","start_time":1563059940,"interval":30,"npoints":610303},"remote":{"exists":false,"start_time":"","interval":"","npoints":""}},"freezer_humidity":{"local":{"exists":true,"id":53,"tag":"Node kitchen","engine":"5","datatype":"1","start_time":1563059970,"interval":30,"npoints":610302},"remote":{"exists":false,"start_time":"","interval":"","npoints":""}},"fridge_dew_point":{"local":{"exists":true,"id":54,"tag":"Node kitchen","engine":"5","datatype":"1","start_time":1563280650,"interval":30,"npoints":602946},"remote":{"exists":false,"start_time":"","interval":"","npoints":""}},"freezer_dew_point":{"local":{"exists":true,"id":55,"tag":"Node kitchen","engine":"5","datatype":"1","start_time":1563280680,"interval":30,"npoints":602945},"remote":{"exists":false,"start_time":"","interval":"","npoints":""}}}
and the page just retries forever because it can’t parse the JSON. Is this a known bug?
How did you install it?
I’m surprised at this path /home/pi/sync/sync-module/. @TrystanLea is the person who knows.
If you followed these instructions GitHub - emoncms/sync: In development: Download or upload emoncms feed data between local and remote emoncms server's it should work, but is not in line with the current filesystem layout. On an emonpi with the latest image, it will go into /opt/emoncms/modules.
I didn’t, it’s always been there.
That link doesn’t seem to have install instructions.
Sorry, I thought you had updated your image. I updated the link. Sorry I can’t help further it’ll need @TrystanLea.
Hello @bwduncan at present emoncms.org does not support uploading data using the sync module, it only supports downloading data. Have you considered using a remote access solution such as dataplicity for remote access to your local system? that way you don’t need to replicate everything in both places?
Remote Access - Guide | OpenEnergyMonitor
Hi Trystan,
That looks neat! I ended up setting up a wireguard VPN which I actually really like. I find it much more useful than I thought I would. Off-topic: KDE connect is so handy and can work anywhere with a VPN, samba browsing works seamlessly for the Mac user on my “LAN” when out and about…
The problem is that neither system has the full set of data. For example, if the Pi locks up because the SD card is getting old then it stops recording but it is still exporting the data to emoncms.org.
The format is pretty simple, I’ll just sync the feed from emoncms.org and write a bit of python to fill in the blanks. Thanks for your help!
Bruce
That’s an idea, if you use the sync module to download the feed data, you could then use the postprocess module mergefeeds process to combine both, it automatically fills in blanks in one feed if there is data available in the other. The output feed is an average of both feeds where data is present in both - but given that your feeds should be the same the output should be the same…
There’s a bit on the post process module here: Post Process module — OpenEnergyMonitor 0.0.1 documentation