Emoncms backup to Dropbox

Paul · 2016-07-06T09:51:47.301Z

That’s good to hear!
Just remember to change the ownership of the folders/files to pi/pi after they’ve been extracted.

For emoncms folders/files -
sudo chown -R www-data:root /var/lib/{phpfiwa,phpfina,phptimeseries}

…and for node-red and mysql -
sudo chown -R pi:pi your_dir

Paul

PS - I’ve left a little utility to help download the archives from dropbox cloud in dropbox-archive/lib called dropShell.
To run it, ./dropShell.sh from within the lib directory.
You can then cd to backups
ls to display your cloud archives
and get an archive, which would then be downloaded to your lib folder.
I would recommend moving the archive to a different directory somewhere to extract it, to keep the ‘lib’ folder clear.
To extract the archive within that directory - sudo tar -zxvf archive_01_07_2016_0030.tar.gz

glyn.hudson · 2016-07-09T09:37:49.312Z

Nice work! @Paul Good to see this working.

What about making it into a emoncms module? So users could easily install and trigger backups from the Emoncms interface? What would be required? Do you think there would be demand for such a module?

Paul · 2016-07-10T20:53:04.846Z

glyn.hudson:

So users could easily install and trigger backups from the Emoncms interface?

My original concept was to develop a fit & forget module, which wasn’t reliant on user’s remembering to trigger a backup, it just backed everything up once a night, and weeded old backups, with no user intervention, so if a problem arose then data loss would be minimal.
That’s pretty much where the module is at this time.

glyn.hudson:

What about making it into a emoncms module?

With a little work I’m sure it’s viable.
It would be good to be able to be able to get it’s configuration from the emoncms settings.php, so there would be no duplicity - mysql database settings, file paths, etc. making installation quicker and avoiding errors.

glyn.hudson:

Do you think there would be demand for such a module?

There hasn’t been that much interest so far TBH. Maybe now you’ve almost eradicated SD card failures, data loss is not perceived to be a priority issue…

Paul

glyn.hudson · 2016-07-10T22:31:29.465Z

Paul:

It would be good to be able to be able to get it’s configuration from the emoncms settings.php, so

Good idea.

Paul:

Maybe now you’ve almost eradicated SD card failures, data loss is not perceived to be a priority issue.

Nothing lasts forever!

borpin · 2016-07-11T08:01:37.655Z

Paul:

There hasn’t been that much interest so far TBH. Maybe now you’ve almost eradicated SD card failures, data loss is not perceived to be a priority issue…

I think this is a great idea. Have not got around to using it but that is because I backup locally, but it is still one of the best backup strategies.

Paul · 2016-07-11T21:36:42.764Z

A post was split to a new topic: Restoring Dropbox-archive backups

Dave · 2016-07-13T14:13:59.218Z

Hi Paul
Fantastic work on the auto backup, have you thought on adding the ability to backup additional folders or files?

Regards
Dave

Paul · 2016-07-13T18:40:57.817Z

What additional folders or files did you have in mind?

Paul

Dave · 2016-07-13T23:34:28.975Z

Thought about backing up the modules folder as I’ve modded several of them to improve functionality.

Regards
Dave

Paul · 2016-07-14T11:52:12.850Z

It shouldn’t be too difficult. I’m away for a few days, and I’ll have a look when i get back.

Paul

Dave · 2016-07-14T18:09:51.757Z

Happy days, thanks Paul.

Paul · 2016-07-15T19:32:39.779Z

@Dave if you wish to also back up your emoncms Modules directory, add the following lines of code to the dropbox-archive/lib/sync.sh around line 19, just after the node-red section (presuming your ‘Modules’ path is the same);

cp -pr /var/www/emoncms/Modules $DIR/../$tdir```

Paul

Paul · 2016-08-13T19:23:53.687Z

I’ve just updated one of the dropbox-uploader scripts which includes a number of bug fixes, increased performance and more importantly enables compatibility with Dropbox’s latest (v2) api.

Also added a feature which if installed on a emonpi or emonpi image in the RO partition, the script puts the OS into RW, and returns it to RO after the script has completed.

To update; git pull from the dropbox-archive directory.

Paul

Paul · 2016-08-22T21:30:49.328Z

Thanks to a suggestion from @ian I’ve made a further change to the dropbox-archive script, which now creates backup archives which are also compatible with the emonpi backup function.

What this means is that dropbox-archive will automatically backup your emoncms data & setting files regularly in the background to Dropbox as normal, but should you wish to restore your emonpi, the archive can now be downloaded from Dropbox and imported into your emonpi by using the backup - restore function.

The script also creates a backup of your node-red flows, settings, etc, however node-red is not currently restored by the emonpi backup script, but can be easily restored manually.

Paul

Paul · 2016-08-24T09:49:48.921Z

@ian in a private email, you mentioned; “…Very easy as you can select the file in your Drop Box directory from within Emoncms.”
Were you able to download the remote dropbox archive from within emoncms?

Paul

ian · 2016-08-24T11:22:53.552Z

Hi Paul

Yes! In the Emoncms backup page there is a “choose file” button. Hit that and you can select any file in any directory on your PC including dropbox. I am on Windows, don’t know if Linux is the same. Having selected the file there is an “Import Backup” button, click that and the dropbox file is imported.

Ian

Paul · 2016-08-24T18:57:51.926Z

ian:

…and you can select any file in any directory on your PC including dropbox

But dropbox is a cloud based service, how are you accessing it?
… or are you using some sort of desktop app to first sync your dropbox (remote) files to your PC, and then loading them into emoncms from the local copy?

Paul

bidouilleur · 2016-08-24T19:22:57.711Z

for sure it is via a local directory that syncs with dropbox (and an app)
I use also dropbox. The app keeps in sync a local directory (usually called dropbox) and your file in the cloud

so it is easy to restore locally from a backup on dropbox. The backup it is instantly pushed down again in the local directory.

ian · 2016-08-25T08:50:06.126Z

Hi Paul

If you create a dropbox account using Windows an app is installed at the same time. A dropbox directory is created and then synced automatically with the cloud. Any files placed in the cloud appear in local device dropbox directories within seconds. There is an Android app that does the same thing.

Ian

bidouilleur · 2016-08-25T16:54:15.944Z

and on mac, iOS and even linux if I’m correct …