For a long time now we have been running an automated incremental daily backup of all feed data on emoncms.org.
The backup data is stored on 3 external drives that are attached to each of the 3 storage servers that emoncms.org is running on. There is a daily process that copies over the new data recorded every day. This will allow relatively quick restoration of a storage server in the event of a failure, as it will be a matter of plugging the backup drive in to the new machine and copying the data across.
In order to double check that the backup’s are ok I’ve implemented a way to load up the backup data in the emoncms.org graph interface, if you wish to check up on your emoncms.org feed data backups this is how to do so:
To check your feed data backups:
- Open a feed in the emoncms graph view:
- Add ?backup=true to the URL:
You should now see the same data but without the most recent hours since the last daily backup ran:
Im aware that a better solution for quick restoration is to replicate each storage server continuously over a second machine in a way where its possible to switch between one or the other. We plan to do this once we implement the subscription charge which will help us afford the extra server costs.
Im also developing an easier way to backup emoncms.org to a local emoncms installation and even upload a local installation to emoncms.org. Its called the sync module: http://github.com/emoncms/sync. It works well for backup already but I have not yet enabled the upload part on emoncms.org