Backup module usb-import.sh script (beta)

One advantage of this usb import approach is that it can in certain cases be used to recover a corrupted SD card were a previous backup archive was not made - or is quite old. If the SD card mounts ok the script may run without trouble, but if the damage is greater, running the file system consistency check utility (fsck) maybe needed before doing an import.

Both Glyn and I had our home systems go down following the same power cut in our village last weekend. While my system booted there were issues with emonhub and redis resulting from SD card corruption. I was able to recover using the importer tool without running fsck. Glyn’s on the other hand would not boot initially - but after running fsck we were able to mount the old SD card and run the import script.

Here are the steps to do this:

1. Insert SD card in SD card reader and plug in to the emonPi/emonbase/raspberrypi running a fresh copy of the emonSD image.
2. Attempt to recover the SD card partitions e.g:

sudo fsck.ext4 /dev/sda2 (root OS partition)
sudo fsck.ext2 /dev/sda3 (data partition)

3. Once complete, run the importer tool as above.

1 Like

And you’ve now both got spike protection… :grinning:

1 Like

Worth adding a page on the docs for this I’d suggest.

1 Like

Actually, @TrystanLea a comprehensive guide on upgrading an SDCard would be fantastic. I cannot remember what is out there right now.

My current advice would be

You are running an older setup. My advice is to upgrade to the lastest version. emonSD-17Oct19 release

To do this;

1 Like

I’ve added the USB import method now to the backup module guide and to a new guide on updating and upgrading emonpi/emonbase systems.

Backup Guide:
Import / Backup - Guide | OpenEnergyMonitor

Update and Upgrade
Update & Upgrade - Guide | OpenEnergyMonitor

Thanks @borpin for your start on this, you will see I’ve made use of what you wrote here: Restore Data From SD Card (USB) · openenergymonitor/emonpi Wiki · GitHub

1 Like

@TrystanLea great. Could you add in a bookmark for the Upgrade section so a direct link can be created to it please?

Couple of things.

“Place the old SD card in a SD” I think it should be ‘an’ so “Place the old SD card in an SD”

“Create a temporary user on the new system if you have not done so already.”

This instruction could be confusing as to whether you mean a user on the OS or on the EmonCMS system. Perhaps

“From the EmonCMS login page, click register and create a temporary user. Once the import is complete the original user details will be used.”

and add

“5. Once the import is complete, log out and back into the EmonCMS page with the original user details”.

1 Like

Great will add those in, thanks @borpin

1 Like

Thanks @borpin I’ve made the changes you suggested. Apparently bookmarks work to any heading, you just need to type the full heading in lower case and with dashes:

Update & Upgrade - Guide | OpenEnergyMonitor

For some headings the link appears when you mouse over as in

image

I could not get the USB import to work so went to a ssh session and found my USB device was found as


I edited the backup/usb-import.sh at master · emoncms/backup · GitHub
usb=$(ls $disk_id | grep ‘usb-Generic_Mass-Storage’) to usb=$(ls $disk_id | grep ‘usb-Generic’)

1 Like

also they should be a little more checking, if you click import from USB without the usb the web interface dies with
’ Can’t connect to database, please verify credentials/configuration in settings.php
Error message: Unknown database ‘emoncms’

Thanks @Pukka, that’s useful to know, I did wonder how many varients on the grep search term for the usb-Generic… there might be. I’ve created a github issue to keep track of this usb-Generic vs usb-Generic_Mass-Storage detection · Issue #52 · emoncms/backup · GitHub.

1 Like

Yet another example of why new “features” need to be “wrung out” before getting released.

I was sure I’d tested this one, its now fixed, thanks!

I have just updated to emonSD-17Oct19 and get “USB drive not found”. The USB drive is:

OK I just read the code and I think I see that the master hasn’t been updated yet, pending Trystan’s GitHub issue investigation - completely understand why. I assume Paul created his own branch to get it working. I’m not a GitHub user so I might need some help to create my own mod :slight_smile:

If you just edit the file line 34 in the folder /opt/emoncms/modules/backup/

to read

usb=$(ls $disk_id | grep ‘usb-Generic’)

it should work.

Just note, you will need to undo this before you are able to update the backup module again.

Thank you Brian! After an initial attempt using Git, I went ahead and did exactly what you suggest about half an hour ago, and the import worked a treat :slight_smile:

However, I wouldn’t have known to undo it for future updates to work though - so double-thanks for that. I made a copy of the file first though :wink:

It would be nice to know how to have done it properly though, via Git…
I created my own fork, edited it and committed it back to my private repo. Now though, how would I apply the update on the EmonPi in order to test it? And then how would I revert back to the master Emoncms release afterwards?

If you want to play be my guest :grinning:

First an explanation of terms

  • upstream is the original emoncms repo you forked.
  • origin is your fork of the upstream
  • local is the repo on your machine

However because the original clone was done direct from the upstream, it calls itself origin in this case (git doesn’t make anything easy - trust me). We won’t bother creating the link though - it doesn’t matter here. It does matter (a bit) if you are doing development work.

First part right, create a fork. Ideally then create your own branch on that fork - if you subsequently want to create a PR (Pull Request) to submit something you have done to the main emoncms repo, this makes it much easier.

To use your repo, you first need to add it to your local machine in the same directory the original is in.

git remote add myrepo https://github.com/YOUR_NAME/backup.git

The ‘myrepo’ can be any name but not one of the 3 above (they have special meaning) git remote -v should look similar to this.

$ git remote -v
myrepo  https://github.com/borpin/backup.git (fetch)
myrepo  https://github.com/borpin/backup.git (push)
origin  https://github.com/emoncms/backup.git (fetch)
origin  https://github.com/emoncms/backup.git (push)

Then the commands

git fetch myrepo
git checkout myrepo/BRANCH_NAME

You will get a message about detatched head - don’t worry about that.

git status

tells you the branch you are on and you are now using your code.

to go back simply do

git checkout origin/master

or

git checkout origin/stable

depending what branch you were on before you started messing about.

HTH

1 Like

Many thanks for taking the time and trouble to write that, Brian. I will master it, but I may need to go and have a little lie down for a bit first :wink:

2 Likes