Pushing data with intermittent cellular internet

I’m setting up a emonpi in a holiday cottage in Scotland where the internet connect is mobile broadband which can drop out occasionally. Naturally all sensors measurements are logged locally, but I’d like to keep the emoncms.org data in sync - this will provide the main user interface for visitors.

I’ve currently set the sensor inputs to be pushed to emoncms. But I’m just wondering what is the default behaviour in the case that there is an internet dropout? From the emoncms.org perspective are un-synced samples cached and uploaded later or discarded? I’ve looked through the emonhub.conf file and can’t see any buffer settings.

I was wondering about periodically syncing the feeds rather than the inputs. However I’m not sure if that can be set to sync periodically. I also noticed some issues that feeds that come down from the internet have a different name to the local feeds, so the two don’t merge together. I’m not overly concerned with this as I just want to push data from the emonpi to the cloud.

Any pointers would be gratefully received. I’d be happy to ssh onto the pi and setup a cron job if that is the best solution.

Thanks
Joe

Sorry I think this query should probably live in emonHub.

Anyhow, to answer my own question…

Looking at emonhub/emonhub_buffer.py at emon-pi · openenergymonitor/emonhub · GitHub and emonhub/EmonHubEmoncmsHTTPInterfacer.py at emon-pi · openenergymonitor/emonhub · GitHub it seems that the default emonhub buffer is 100000 samples. The self.flush method calls self._process_post which should return True if data is successfully sent to Emoncms. If False the data is stored in the buffer.

However, the buffer is circular so old samples will be overwritten if the upload continues to fail. I should be able to increase that buffer somehow…

I think the Sync module may do what you want so pass it to the local EmonPi then let it sync to emoncms.org.