Community
OpenEnergyMonitor

Community

Increase emonSD pre-built SD card to 8GB min

Tags: #<Tag:0x00007f6e043e8360>

Hahah yes, I did expect that comment! I’m sure you will also be expecting my reply :wink:

IMHO it’s unnecessary to run a full desktop OS on a headless box.

As for future proofness, Jessie lite is officially released and supported by the Raspbian team and can access any Jessie package from the repositories.

I’m not planning on doing a full image re-build this time, just an interim release pulling in all the latest security updates, fixes and possible adding a couple of new features.

If a user wishes Jessie Lite can be ‘upgraded’ to full Jessie by installing the following:

adwaita-icon-theme alacarte alsa-base aspell aspell-en blt bluej claws-mail
claws-mail-i18n coinor-libcbc3 coinor-libcgl1 coinor-libclp1
coinor-libcoinmp1:armhf coinor-libcoinutils3 coinor-libosi1 cryptsetup-bin
cups-bsd cups-client cups-common dbus-x11 dconf-gsettings-backend:armhf
dconf-service debian-reference-common debian-reference-en desktop-base
desktop-file-utils dh-python dictionaries-common dillo eject emacsen-common
epiphany-browser epiphany-browser-data esound-common fontconfig
fontconfig-config fonts-dejavu fonts-dejavu-core fonts-dejavu-extra
fonts-freefont-ttf fonts-opensymbol fonts-roboto fonts-sil-gentium-basic
freepats fuse galculator gconf-service gconf2 gconf2-common gdebi-core
gettext-base giblib1:armhf gir1.2-atk-1.0 gir1.2-freedesktop:armhf
gir1.2-gdkpixbuf-2.0 gir1.2-glib-2.0:armhf gir1.2-gmenu-3.0
gir1.2-gtk-3.0:armhf gir1.2-pango-1.0:armhf git git-core git-man gksu
glib-networking:armhf glib-networking-common glib-networking-services
gnome-desktop3-data gnome-

from https://www.raspberrypi.org/forums/viewtopic.php?t=145665

1 Like

PlatformIO is installed but not Home Assistant yet. Just re-partitioned and this is how things look like on an 8GB SD. Plenty of space :slight_smile:

[email protected]:~ $ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/root       4.9G  2.9G  1.8G  63% /
devtmpfs        483M     0  483M   0% /dev
tmpfs           487M     0  487M   0% /dev/shm
tmpfs           487M  6.6M  480M   2% /run
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           487M     0  487M   0% /sys/fs/cgroup
tmpfs            40M  6.6M   34M  17% /var/lib/openhab
tmpfs           1.0M  4.0K 1020K   1% /var/lib/dhcpcd5
tmpfs           1.0M     0  1.0M   0% /var/lib/dhcp
tmpfs            50M  800K   50M   2% /var/log
tmpfs            30M   40K   30M   1% /tmp
/dev/mmcblk0p1   60M   21M   40M  35% /boot
/dev/mmcblk0p3  1.9G  171M  1.7G  10% /home/pi/data

@glyn.hudson - make sure the pre-release built is clean;

“apt-get clean all” - for example this cleaned up ~700MB of apt cache on my system.
I’m currently looking at the rest of the file system to see what else can be cleaned up :slight_smile:

/var/swap exists in the image - despite swap being off, so thats another ~100MB to clean up.

Nice find,

Does the just clean up remove unused tempory package files. Is there no detrimental effects? Why is clean no run automatically as part of the linux package instal?

I’ve added it onto my next image ‘to do’ list:

This may help answer your question:

1 Like

Packages are NOT removed, only cached information (and therefore cached copies of installed applications).
Since the installed base isn’t likely to change - I would encourage you to “apt-get clean all” to remove the crap.

IF you want to remove installed but un-used applications, that should be "apt-get autoremove --purge"
However your system is already clean;

[email protected](rw):~$ sudo apt-get autoremove --purge
Reading package lists... Done
Building dependency tree        
Reading state information... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

Another nice addition - you might notice my prompt that tells me when the filesystem is “ro” or “rw” - add the following code to the end of /etc/bash.bashrc

set_bash_prompt(){
    fs_mode=$(mount | sed -n -e "s/^\/dev\/mmcblk0p2 on \/ .*(\(r[w|o]\).*/\1/p")

    if [ $(id -u) -eq 0 ];
      then
        PS1='\[\033[01;31m\]\[email protected]\h${fs_mode:+($fs_mode)}\[\033[00m\]:\[\033[01;34m\]\W\[\033[00m\]\$ '
      else
        PS1='\[\033[01;32m\]\[email protected]\h${fs_mode:+($fs_mode)}\[\033[00m\]:\[\033[01;34m\]\W\[\033[00m\]\$ '
    fi
}

# setup fancy prompt
PROMPT_COMMAND=set_bash_prompt

This also turns the whole prompt RED if you are root :slight_smile:
Hope those help.

2 Likes

Post clean-up scores…

[email protected](ro):~$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/root       3.4G  1.5G  1.8G  46% /
1 Like

Wow, that’s really useful! I’ve lost count of the number of times I edit a file with nano but then unable to save changes since I forgot to toggle RW. Nice work, I’ll make sure that get’s into the next release :thumbsup:

I guess it might also be possible to add it to existing systems via emonpi update…

1 Like

Of course it is.

As for the filesystem sizing, you could (now that we have a proven way to clean up) actually reduce your root volume to 3G leaving another 400MB for logging data.

Not sure what you use for closing the SD card image, but the way I do it is to use a modified version of this;
https://raw.githubusercontent.com/billw2/rpi-clone/master/rpi-clone

My modifications include creating specifically sized partitions (rather than filling the target SD card) and writing zeros (from /dev/zero) in all of the free space, so that when I then pull the image back, truncate it and compress it - I get a SERIOUS compression advantage (because compressed zeros take up zero space…)

Hope those tips help.

Nice, thanks for the tips. That script looks useful. I have been using dd after removing the SD card from the Pi and Gparted if partitions need editing.

So I use that to clone the working system onto a new SD card, mine is modified to not copy things like .bash_history etc, so I have some added excludes (you can find these easily enough)

        --exclude '/var/log' \
        --exclude '/home/pi/.ssh' \
        --exclude '/home/pi/.bash_history' \
        --exclude '/root/.bash_history' \

Then at the end - before the target is unmounted;

mkdir -p $CLONE/var/log
#
# Clean the disk (full the partition with zeros to help with later compression)
#
echo ""
echo "*** Writing zero's to the remaining disk space, this will take a few mins..."
echo ""

dd if=/dev/zero of=$CLONE/large.file bs=1M count=2048
sleep 5

rm -rf $CLONE/large.file

Once the image write has completed, I then use DD to pull the whole image back, and then use truncate to rip the image down to only the used space on the SD Card;
Read this: https://evilshit.wordpress.com/2014/02/07/how-to-trim-disk-images-to-partition-size/

Now you have your collapsed SD Card image, you can zip that as normal and it should get significantly reduced in size.

I should have added this part too, this is where i set the size of the target root filesystem;

# Borrowed from do_expand_rootfs in raspi-config
expand_rootfs()
        {
        # Get the starting offset of the root partition
        # (with Jessie's parted, now need to strip trailing 's' from PART_START)
        PART_START=$(parted /dev/mmcblk0 -ms unit s p  \
                        | grep "^2" | cut -f 2 -d: | cut -f 1 -d s)
        [ "$PART_START" ] || return 1
        # Return value will likely be error for fdisk as it fails to reload the
        # partition table because the root fs is mounted
        fdisk /dev/$DST_DISK > /dev/null <<EOF
p
d
2
n
p
2
$PART_START
+1780M
p
w
q
EOF
        }

You will need to modify this to set the root partition size AND to create the /home/pi/data partition too

That’s triggered something I’ve wanted to ask for ages, but never got around to.

Is there a good reason why there are two separate scripts, one for “ro” and one for “rw”, instead of one that takes “-ro” and “-rw” as parameters?

Then, the command becomes syntactically the same as everything else that takes options.

Adding this to the /etc/bash.bashrc will remove the need for the files altogether :slight_smile:

# Aliases to control re-mounting
alias rpi-ro='sudo mount -o remount,ro / ; sudo mount -o remount,ro /boot'
alias rpi-rw='sudo mount -o remount,rw / ; sudo mount -o remount,rw /boot'

I should point out that if you have other scripts that rely on running ‘rpi-ro’ and or ‘rpi-rw’ - using aliases is probably not a good idea…

No there isn’t.

Some time ago I was working on a “rpi” script that offered many options from creating a data partition, adding a hdd, switching between a RO OS and standard OS, enabling MQTT, enabling the LAMP etc etc, This was offered to T&G way before the emonPi or the emonPi image was here. I have not pursued this due to the current trend not to ssh in favor of the emoncms config pages.

Here’s my forwarding only emonbase via ssh,

Using username "pi".
Authenticating with public key "pb66" from agent
Linux energyPi 3.18.11+ #781 PREEMPT Tue Apr 21 18:02:18 BST 2015 armv6l
  ___                 ___                        __  __          _ _
 / _ \ _ __  ___ _ _ | __|_ _  ___ _ _ __ _ _  _|  \/  |___ _ _ (_) |_ ___ _ _
| (_) | '_ \/ -_) ' \| _|| ' \/ -_) '_/ _` | || | |\/| / _ \ ' \| |  _/ _ \ '_|
 \___/| .__/\___|_||_|___|_||_\___|_| \__, |\_, |_|  |_\___/_||_|_|\__\___/_|
      |_|                             |___/ |__/                          .org
(type 'emon' for details)
[email protected] ~ $ rpi

The 'rpi' command options

General utilities
   -c | --config    Opens an extended 'raspi-config' user menu
   -h | --help      Display this help for 'rpi' command options
   -u | --update    Updates the Raspbian OS & debian packages
   -l | --lock      Lock (read-only enabled filesystem)
   -o | --open      Open (read-only enabled filesystem)
   -s | --status    Report status information

File system utilities
  -er | --enable-ro     Configures filesystem as read-only
  -dr | --disable-ro    Reverse changes to read-only filesystem
  -ro | --read-only     Lock (read-only enabled filesystem)
  -rw | --read-write    Open (read-only enabled filesystem)
  -ad | --add-datapart  Add a data partition
  -mr | --move-rootfs   Configures OS & rootfs on USB drive
  -em | --enable-mqtt   Enable Mosquitto MQTT v3.1.1 server

[email protected] ~ $ emonhub

The 'emonhub' command options

emonHub utilities
   -c | --config    Configure emonHub by editing emonhub.conf
   -t | --crontab   Opens emonHub's crontab time scheduler
   -h | --help      Display help for 'emonhub' command options
   -r | --restart   Restart the emonhub service
   -s | --start     Start the emonhub service
   -u | --update    Update emonHub via the git repository
   -v | --version   Display the emonHub version installed
   -x | --exit      Exit emonHub by stopping the emonhub service
   -d | --disable   Disable emonHub from starting automatically
   -e | --enable    Enable emonHub to always start automatically

emonHub logging
  -dl | --display-log   Display last 10 & on-going log messages
  -ol | --open-logfile  Open logfile in nano text editor
  -rl | --reset-logfile Reset the current logfile (deletes log!)
  -vl | --view-logfile  View the entire logfile

[email protected] ~ $ emoncms

Note emonCMS has not been fully installed & configured yet
The following commands will be available once emonCMS is installed

The 'emoncms' command options

emonCMS utilities
   -c | --config    Opens emoncms.conf for configuration
   -h | --help      Display this 'emoncms' command options help
   -u | --update    Updates emonCMS via the git repository

emonCMS setup
  -da | --disable-all   Disables the server, database & emonCMS
  -el | --enable-lite   Enables server, database & lite emonCMS
  -ef | --enable-full   Enables server, database & full emonCMS

To install emonCMS use 'emoncms --install' for installer options

[email protected] ~ $

Bill suggested the changes to bash.rc for (RW) and (RO) way back, at the time I actually got the prompt to change colour, red for RO and green for RW but I’ve since lost that feature some where along the line.

EDIT - Oh! I forget I made the “emon” option that listed them all, just spotted above.

[email protected] ~ $ emon

 The 'OpenEnergyMonitor' Open-source Hardware & Software project

    navigation via the 'rpi', 'emonhub' & 'emoncms' commands


The 'rpi' command options

General utilities
   -c | --config    Opens an extended 'raspi-config' user menu
   -h | --help      Display this help for 'rpi' command options
   -u | --update    Updates the Raspbian OS & debian packages
   -l | --lock      Lock (read-only enabled filesystem)
   -o | --open      Open (read-only enabled filesystem)
   -s | --status    Report status information

File system utilities
  -er | --enable-ro     Configures filesystem as read-only
  -dr | --disable-ro    Reverse changes to read-only filesystem
  -ro | --read-only     Lock (read-only enabled filesystem)
  -rw | --read-write    Open (read-only enabled filesystem)
  -ad | --add-datapart  Add a data partition
  -mr | --move-rootfs   Configures OS & rootfs on USB drive
  -em | --enable-mqtt   Enable Mosquitto MQTT v3.1.1 server


The 'emonhub' command options

emonHub utilities
   -c | --config    Configure emonHub by editing emonhub.conf
   -t | --crontab   Opens emonHub's crontab time scheduler
   -h | --help      Display help for 'emonhub' command options
   -r | --restart   Restart the emonhub service
   -s | --start     Start the emonhub service
   -u | --update    Update emonHub via the git repository
   -v | --version   Display the emonHub version installed
   -x | --exit      Exit emonHub by stopping the emonhub service
   -d | --disable   Disable emonHub from starting automatically
   -e | --enable    Enable emonHub to always start automatically

emonHub logging
  -dl | --display-log   Display last 10 & on-going log messages
  -ol | --open-logfile  Open logfile in nano text editor
  -rl | --reset-logfile Reset the current logfile (deletes log!)
  -vl | --view-logfile  View the entire logfile


Note emonCMS has not been fully installed & configured yet
The following commands will be available once emonCMS is installed

The 'emoncms' command options

emonCMS utilities
   -c | --config    Opens emoncms.conf for configuration
   -h | --help      Display this 'emoncms' command options help
   -u | --update    Updates emonCMS via the git repository

emonCMS setup
  -da | --disable-all   Disables the server, database & emonCMS
  -el | --enable-lite   Enables server, database & lite emonCMS
  -ef | --enable-full   Enables server, database & full emonCMS

To install emonCMS use 'emoncms --install' for installer options

[email protected] ~ $

Wow, never seen this before. Looks very tidy.

Yes, if SSH’ing should be a last resort.

Let’s stick with rpi-rw and rpi-ro, too many scripts depend on these commands to consider changing unless there is a very compelling reason to.

Ironically enough, I made the same suggestion about a year ago and got shot down. Go figure.

Not all of us forgot that Bill…

Also the use of apt-get clean was discussed at length way back when Glyn was struggling to get the image on a 4gb card before going down the slim OS route.

Neither of these points take anything away from Andy, they are both good points to be raised again, it just shows how much unused and at the time perhaps unappreciated information is hidden away in the depths of these forums.

1 Like

Aye, you’ve got that right!

Sorry, I do my best :disappointed:

Things will occasionally get missed, this doesn’t mean their unappreciated.