Community
OpenEnergyMonitor

Community

Replacing the SD card with a USB flash drive in the emonBase

Hi all,

I know I read about this is the past but I can’t find the relevant posts anymore: to limit the risk of SD card failure I would like to replace the SD card in my emonBase with an SLC flash drive.
I assume that this will be easier than selectively moving just the data filesystem(s) to the flash drive (esp. in terms of future updates)?

I am running emonSD-30Oct18 (OEM pre-built) on a Raspberry Pi 3 Model B+ Rev 1.3 - 1 GB (Sony UK).

Are there any easy to follow instructions lying around?

Thanks!
Franck

Hi Franck,

This should help:

It makes reference to a USB Hard Drive, but the process is the same for either media
since both are connected to the Pi via USB.

Be sure to change the /etc/fstab file to reflect the correct location of the
/home/pi/data partition. i.e. on your USB drive vice the microSD card.

Thanks Bill,

This doesn’t look like a walk in the park :frowning:

Franck

Hello @frank102,

A bit can be set on the Raspberry Pi 3 to force it to always boot from USB mass storage. Understand that this sets a bit in the RPi and cannot be reversed.

See: How to boot from a USB mass storage device on a Raspberry Pi

USB boot is available on the Raspberry Pi 3B, 3B+, 3A+ and Raspberry Pi 2B v1.2 models only. (I assume also the RPi4)

Another ‘outdated’ but workable solution is described here this method still uses the SD card to boot the machine but is reversible.

I strongly suggest trying this method on a non-critical system first if you’re not comfortable with the procedure.

Best wishes,

I’m not sure the firmware does support this at launch. It is on their do very soon list I believe (as is net booting).

I’d use an old, small SDCard to get the kernel up and then run everything from the SLC.

It’s actually not that difficult (time consuming, more than anything else), but if one isn’t
comfortable working with the Linux command line, it can appear rather intimidating.

The change that Neil mentioned is the better route to take. While the change itself is
permanent, implementing it doesn’t restrict your Pi to booting only from a USB device.

The change enables the Pi to look at the USB ports to see if a bootable device is there,
and if not, the Pi will still boot from a microSD card.

I worked as a Unix sysadmin 20 years ago… but I am pretty out of touch with Linux specific boot sequence & filesystem or device handling :slight_smile:

Booting from USB first sounds good, I have a Raspberry 3B+ so it looks like I don’t even need to flip the USB bit. So the high-level process would be:

  • download the emonSD image
  • Use Etcher (or equivalent) to burn the image on the flash drive
  • Make the flash drive bootable (if that applies for Linux, I’ll investigate)
  • boot from the SD, insert the flash drive
  • … and not sure what to expect there: will I need to manually mount each partition from the flash drive?
  • modify the fstab on the flash drive… hoping I can figure out the right /dev/xxx device names to use.

Franck

If you want to boot directly from the USB drive, you need to “flip the bit.”

Yes
Yes
Yes, it needs to be bootable.
Once you’ve set the “boot from USB” bit, there’s no need for the SD card.
See the next line
Yes. Your fstab will look something like this:

/dev/sda1  /boot          vfat  defaults,noatime  0  0
/dev/sda2  /              ext4  defaults,noatime  0  1
/dev/sda3  /home/pi/data  ext4  defaults,noatime  0  2

Make sure you put a carriage return in the last line of the fstab file.

You’ll need to edit the /boot/cmdline.txt file too.
Change root=/dev/mmcblk0p2 to read root=/dev/sda2 and you should be good to go.

You may want to turn off the swapfile and use a swap partition.

More info on the entire process, here:

Not for a pi 3b+, from the rpi website link given earlier in the thread.

The Raspberry Pi 3+ is able to boot from USB without any changes, but the Raspberry Pi 3 requires the USB boot bit to be set in the OTP (one-time programmable) memory. If you are using a Raspberry Pi 3+, please go to the next section.

[Edit] if going down the “small sdcard” route, be aware the very latest “Buster” image has a 256MB boot partition, prevoius boot partitions have all been about 60MB. I have a couple of old 64MB cards I used for this previously.

[edit2] I’m looking forward to hearing what the position is with the Pi4, booting to a USB3 HDD/SSD should be pretty slick.

Thanks Bill, I’ll give this a try.

I am assuming that the Pi won’t boot from the flash until the fstab is fixed. I was planning to do that on the Pi after booting from the SD one last time, but it looks like I should be able to mount the ext4 partition on my Mac.

Franck

After you modify /boot/cmdline.txt with the new location of the root filesystem. It’ll start the boot
process from the SD card, then mount and run, the root filesystem from the USB drive even if the
root filesystem entry in /etc/fstab still points to the SD card. (that’s been my experience, YMMV)

That’s if the USB drive isn’t bootable, but contains a root filesystem.
Otherwise, if the USB device is bootable, it’ll boot from the USB device first.

Yep. Missed that part. shocked

Post edited.

And fast too.

Making progress, but I can’t get the Pi to boot from the flash drive :frowning:

I used Etcher to flash the drive with the emonSD-30Oct18 image: I don’t find any tools that would turn an already flashed drive into a bootable drive, it seems the image just is bootable or isn’t?

If I insert both SD and flash drive the Pi boots from the SD card.
If I insert the flash drive only the Pi won’t boot - the screen doesn’t even come up.

I have tried booting once with “program_usb_boot_mode=1” at the end of /boot/config.txt just in case (even this shouldn’t be needed on the 3B+) and that didn’t help.

Here is what I have so far:

[email protected]:~ $ lsblk
NAME        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda           8:0    1 28.9G  0 disk
├─sda1        8:1    1 43.2M  0 part
├─sda2        8:2    1  3.9G  0 part
└─sda3        8:3    1  3.3G  0 part
mmcblk0     179:0    0 14.5G  0 disk
├─mmcblk0p1 179:1    0 43.2M  0 part /boot
├─mmcblk0p2 179:2    0  3.9G  0 part /
└─mmcblk0p3 179:3    0 10.5G  0 part /home/pi/data

[email protected]:~ $ mkdir /sda1
[email protected]:~ $ sudo mount /dev/sda1 /sda1
[email protected]:~ $ mkdir /sda2
[email protected]:~ $ sudo mount /dev/sda2 /sda2

[email protected]:/sda1 $ sudo vi /sda1/cmdline.txt

/sda2/etc/fstab uses PARTUUID on that image, check that they match the new block devices

[email protected]:~ $ sudo blkid /dev/sda1
/dev/sda1: LABEL="boot" UUID="3725-1C05" TYPE="vfat" PARTUUID="50c2568e-01"

[email protected]:~ $ sudo blkid /dev/sda2
/dev/sda2: LABEL="rootfs" UUID="fd695ef5-f047-44bd-b159-2a78c53af20a" TYPE="ext4" PARTUUID="50c2568e-02"

[email protected]:~ $ sudo blkid /dev/sda3
/dev/sda3: UUID="9d452b0a-cb89-4ec2-a32c-6b85dcbcc1c2" TYPE="ext2" PARTUUID="50c2568e-03"

Franck

I don’t have any 3B+'s so cannot speak from experience, but since the bit doesn’t need setting and in the RPi guide there is mention of a certain image date as to when this works from, I suspect the PARTUUID may be needed. The “/dev/sad2” was a quick and dirty way of setting up a hdd as the PARTUUID can change when any changes are made to the partitions and so because the emonsd adds a 3rd partition, the “/dev/mmcblk0p2” address has continued to be used for the same reason.

If you boot to the sdcard, then mount the hdd, take note of the PARTUUID prefix (50c2568e ? ) and then edit both the cmdline.txt and fstab to use the PARTUUIDs and partition number ids instead you might have more joy.

The cmdline.txt should have something like root=PARTUUID=50c2568e-02 in place of the root=/dev/sda2 or root=/dev/mmcblk0p2 and the fstab will look something like

PARTUUID=50c2568e-01  /boot          vfat  defaults,noatime  0  0
PARTUUID=50c2568e-02  /              ext4  defaults,noatime  0  1
PARTUUID=50c2568e-03  /home/pi/data  ext2  defaults,noatime  0  2

This is far more robust but it can be a pain since you must be sure to update the cmdline.txt and fstab before rebooting if changes are made to the partitions.

I do not know for sure if this is the issue, but either way, this is the better method if you can manage the PARTUUID changes (or avoid them by not changing the hdd partitions) because one day you might leave another usb drive attached and reboot (power interruption?) and “sda2” might get assigned to another drive and therefore not boot.

[edit]

from the guide

Starting with the 2017-04-10 release of Raspbian you can install a working Raspbian system to a USB mass storage device by copying the operating system image directly onto your USB device, in the same way that you would for an SD card.

and from the image release notes

2017-04-10:
* Wolfram Mathematica updated to version 11.0.1
* Adobe Flash Player updated to version 25.0.0.127
* Use PARTUUID to support USB boot

Thanks Paul,

This all works like a charm in the end, with the proper power supply and the cmdline.txt change to use PARTUUID. Here is a summary of the instructions for anyone hitting this thread in the future:

The steps below were tested with Pi model 3B+ and the emonSD-30Oct18 image. For older models check out the OTP bit step explained higher up in this thread. The good news is that this is 100% reversible, if you run in trouble remove the USB drive, put the SD card back in and your original system is back.

  1. Download the emonSD image (Github)

  2. download Etcher

  3. Plug in your flash drive in your PC, and use Etcher to burn the image to the drive

  4. Make sure you have the proper power supply for your Pi. Also make sure you have the proper cable (fast charge / 2A min). Don’t skimp here, underpowering will cause issues…

  5. Plug-in the USB drive into the Pi

  6. If you want to resize your data partition, do it now using fsck & resize2fs

  7. Mount the root partition from the USB drive:

    [email protected]:~ $ mkdir /sda1
    [email protected]:~ $ sudo mount /dev/sda1 /sda1

  8. Edit cmdline.txt to replace the root device:
    [email protected]: sudo vi /sda1/cmdline.txt
    <replace root=/dev/sda2 with root=PARTUUID=50c2568e-02>

  9. Shutdown the pi, pull out the SD card, and start up the system

Additional notes:

  • If nothing comes out of the video port check your power supply
  • If you get a bunch of system-level messages (about device detection among others) but no “Welcome to Raspbian” prompt check your cmdline.txt
  • The emonpi/service-runner-update.sh that gets executed automatically on the first boot from the new partitions froze on me the first time around. I ran it manually the next time around.

Franck

1 Like

If you just want to edit the cmdline.txt alone, you can do that on your PC immediately after flashing and then just connect and boot straight to the usb drive, eliminating the steps with a sdcard.

In Ether there is a setting to auto-eject after flashing, it is on by default as a safety measure. I have this turned off so that I can drop a ssh and wpa_supplicant.conf file (and any edits etc) to the /boot partition and then manually eject the usb/sdcard once I’m ready.

Having gone this far i would recommend editing your fstab to use PARtUUIDs too

Good point about editing on the PC, I initially thought I would need to edit the fstab on the ext4 partition as well which is why I went that route.

The emonSD-30Oct18 image already uses them. I wondered how that was possible but they come from the partition table which is burnt in the image, so this shouldn’t need changing.

I am not completely out of the woods yet, the performance is dismal with the flash drive (5mn to log in) after I restored my data, though it could be the drive, or the update (I know run low-write 10.1.1), or a power problem, or a process gone wild…

Franck

The fstab should look like this on the Oct2018 image

i say “should” but we have no buildguide to reference for that image, previous images used this and we’ve been told the Oct2018 “is the same”.

Obviously those addresses will not work with your HDD so i assume you are saying they are already using PARTUUID ? If that is the case, then when you enlarge your partitions, which i assume you will do unless you have found yourself a really small hdd, the PARTUUID might well change.

Yes, they already changed the fstab to use PARTUUIDs.
I didn’t even think about enlarging the partitions, good point - I will check the uuids afterwards.

Franck