Various users have described moving the SD card root operating system image to an external HDD to better protect the SD card from corruption when subjected to uncontrolled power downs. Indeed, the requirement to maximise SD card life has spawned a great deal of effort on the part of the developers to create a low-write SD card image. This topic is intended to discuss the issues relating to moving away from the SD card to external USB HDDs, which are considered more robust to power failures
It is well known that SD cards are not ideal for data logging applications when numerous small size data writes are continuously required. Much discussion and very worthy efforts have been expended by the development team in creating a buffered low write SD card image. At the same time, they acknowledge that it is a sensible move to transfer the system to an external USB HDD. A script located at /var/www/emoncms/scripts/usb_hdd/usb_hdd has been provided for this purpose.
This post discusses the experience of using the script with a relatively slow early version miniature 1.8" USB HDD and poses some further questions raised.
Paul Reed has helpfully [described the use of the script] (emoncms/USB_HDD.md at stable · emoncms/emoncms · GitHub) which is based upon an article on the adafruit site.
On my first attempt to run this script, it failed to complete:
Target drive = /dev/sda
[start] Will create new ext4 filesystem on /dev/sda
[start] If there is data on /dev/sda, it will be lost.
Really proceed? (y)es / (n)o y
[dependencies] Installing gdisk, rsync, and parted.
Reading package lists… Done
Building dependency tree _
Reading state information… Done
parted is already the newest version.
rsync is already the newest version.
rsync set to manually installed.
The following NEW packages will be installed:
_ gdisk
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 193 kB of archives.
After this operation, 799 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 Index of /raspbian jessie/main gdisk armhf 0.8.10-2 [193 kB]
Fetched 193 kB in 0s (337 kB/s)
Selecting previously unselected package gdisk.
(Reading database … 49470 files and directories currently installed.)
Preparing to unpack …/gdisk_0.8.10-2_armhf.deb …
Unpacking gdisk (0.8.10-2) …
Processing triggers for man-db (2.7.0.2-5) …
Setting up gdisk (0.8.10-2) …
[fs create] Creating /dev/sda1
[fs create] Creating ext4 filesystem on /dev/sda1
mke2fs 1.42.12 (29-Aug-2014)
The file /dev/sda1 does not exist and no size was specified.
A search of the adafruit forums indicated that their script had shown this behaviour before and one user’s comment had suggested that the HDD could not keep up with the script. His solution was to include a sleep 1 statement before the sudo mkfs -t ext4 -L rootfs /dev/sda1 line in the script.
I have no idea whether this theory is true but this suggestion works for me and I report it here for information as it produces very little delay. 
However, in my study of the script a number of other questions were raised:
The script was written with a SD card with only a FAT boot and ext4 root partition in mind. Unless I am completely misunderstanding how to interpret an fstab table, the third /home/pi/data partition where the SQL database and the time series data is stored and more importantly written still resides on the SD card as shown by sudo cat /etc/fstab:
tmpfs /tmp tmpfs nodev,nosuid,size=200M,mode=1777 0 0
tmpfs /var/log tmpfs nodev,nosuid,size=50M,mode=1777 0 0
tmpfs /var/lib/dhcp tmpfs nodev,nosuid,size=1M,mode=1777 0 0
tmpfs /var/lib/dhcpcd5 tmpfs nodev,nosuid,size=1M,mode=1777 0 0
tmpfs /var/lib/openhab tmpfs nodev,nosuid,size=40M,mode=1777 0 0
proc /proc proc defaults 0 0
/dev/mmcblk0p1 /boot vfat defaults,noatime,nodiratime 0 2
/dev/mmcblk0p3 /home/pi/data ext2 defaults,rw,noatime,nodiratime,errors=remount-ro 0 2
/dev/disk/by-uuid/333004b2-5c84-4a8a-9cef-b00a1f89ab02 / ext4 defaults,noatime 0
Does this not defeat the objective of moving the data logging onto the HDD drive if only the ro root partition is moved over? If this is not the case could somebody explain why not, please! There is virtually no activity on the HDD LED light.
I intend to experiment with:
- using gParted to shrink the size of the root partition on the HDD
- create an ext2 partition in the unallocated space on the HDD
- copy over the /home/pi/data partition to this new ext2 partition
- alter the fstab table to reflect these changes
In theory, this could be done in the usb_hdd script but I am far from competent to make such changes. Once working from the HDD reverting back by restoring the original /boot/cmdline.txt file should work with the proviso that any data collected on the HDD would be missing.
There is one further request I would make of the development team:
Is there a description or script in existence to revert the new Jessie Lite image back to a full write version that will enable the PHPFIWA engine to be selected again once running on the HDD?
Keep up the good work team!
Thanks
I cannot speak for debian, but on a redhat system you need to wait for the kernel to have triggered a re-read of the partition table, and possibly for udev (if it is in control) to generate the device file. I would previously have forced this with a call to
sfdisk -R /dev/sda
Same for Debian.
Two other tools that can force a PT re-read are partprobe
http://www.cyberciti.biz/tips/re-read-the-partition-table-without-rebooting-linux-system.html
and hdparm
Hi did you have any luck
Trying to get the script to move over 2 partitions to a new msata addon card i bought
anyone else get this to work getting lost