Community
OpenEnergyMonitor

Community

Increase emonSD pre-built SD card to 8GB min

Tags: #<Tag:0x00007f6e08eb2f30>

Packages are NOT removed, only cached information (and therefore cached copies of installed applications).
Since the installed base isn’t likely to change - I would encourage you to “apt-get clean all” to remove the crap.

IF you want to remove installed but un-used applications, that should be "apt-get autoremove --purge"
However your system is already clean;

[email protected](rw):~$ sudo apt-get autoremove --purge
Reading package lists... Done
Building dependency tree        
Reading state information... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

Another nice addition - you might notice my prompt that tells me when the filesystem is “ro” or “rw” - add the following code to the end of /etc/bash.bashrc

set_bash_prompt(){
    fs_mode=$(mount | sed -n -e "s/^\/dev\/mmcblk0p2 on \/ .*(\(r[w|o]\).*/\1/p")

    if [ $(id -u) -eq 0 ];
      then
        PS1='\[\033[01;31m\]\[email protected]\h${fs_mode:+($fs_mode)}\[\033[00m\]:\[\033[01;34m\]\W\[\033[00m\]\$ '
      else
        PS1='\[\033[01;32m\]\[email protected]\h${fs_mode:+($fs_mode)}\[\033[00m\]:\[\033[01;34m\]\W\[\033[00m\]\$ '
    fi
}

# setup fancy prompt
PROMPT_COMMAND=set_bash_prompt

This also turns the whole prompt RED if you are root :slight_smile:
Hope those help.

2 Likes

Post clean-up scores…

[email protected](ro):~$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/root       3.4G  1.5G  1.8G  46% /
1 Like

Wow, that’s really useful! I’ve lost count of the number of times I edit a file with nano but then unable to save changes since I forgot to toggle RW. Nice work, I’ll make sure that get’s into the next release :thumbsup:

I guess it might also be possible to add it to existing systems via emonpi update…

1 Like

Of course it is.

As for the filesystem sizing, you could (now that we have a proven way to clean up) actually reduce your root volume to 3G leaving another 400MB for logging data.

Not sure what you use for closing the SD card image, but the way I do it is to use a modified version of this;
https://raw.githubusercontent.com/billw2/rpi-clone/master/rpi-clone

My modifications include creating specifically sized partitions (rather than filling the target SD card) and writing zeros (from /dev/zero) in all of the free space, so that when I then pull the image back, truncate it and compress it - I get a SERIOUS compression advantage (because compressed zeros take up zero space…)

Hope those tips help.

Nice, thanks for the tips. That script looks useful. I have been using dd after removing the SD card from the Pi and Gparted if partitions need editing.

So I use that to clone the working system onto a new SD card, mine is modified to not copy things like .bash_history etc, so I have some added excludes (you can find these easily enough)

        --exclude '/var/log' \
        --exclude '/home/pi/.ssh' \
        --exclude '/home/pi/.bash_history' \
        --exclude '/root/.bash_history' \

Then at the end - before the target is unmounted;

mkdir -p $CLONE/var/log
#
# Clean the disk (full the partition with zeros to help with later compression)
#
echo ""
echo "*** Writing zero's to the remaining disk space, this will take a few mins..."
echo ""

dd if=/dev/zero of=$CLONE/large.file bs=1M count=2048
sleep 5

rm -rf $CLONE/large.file

Once the image write has completed, I then use DD to pull the whole image back, and then use truncate to rip the image down to only the used space on the SD Card;
Read this: https://evilshit.wordpress.com/2014/02/07/how-to-trim-disk-images-to-partition-size/

Now you have your collapsed SD Card image, you can zip that as normal and it should get significantly reduced in size.

I should have added this part too, this is where i set the size of the target root filesystem;

# Borrowed from do_expand_rootfs in raspi-config
expand_rootfs()
        {
        # Get the starting offset of the root partition
        # (with Jessie's parted, now need to strip trailing 's' from PART_START)
        PART_START=$(parted /dev/mmcblk0 -ms unit s p  \
                        | grep "^2" | cut -f 2 -d: | cut -f 1 -d s)
        [ "$PART_START" ] || return 1
        # Return value will likely be error for fdisk as it fails to reload the
        # partition table because the root fs is mounted
        fdisk /dev/$DST_DISK > /dev/null <<EOF
p
d
2
n
p
2
$PART_START
+1780M
p
w
q
EOF
        }

You will need to modify this to set the root partition size AND to create the /home/pi/data partition too

That’s triggered something I’ve wanted to ask for ages, but never got around to.

Is there a good reason why there are two separate scripts, one for “ro” and one for “rw”, instead of one that takes “-ro” and “-rw” as parameters?

Then, the command becomes syntactically the same as everything else that takes options.

Adding this to the /etc/bash.bashrc will remove the need for the files altogether :slight_smile:

# Aliases to control re-mounting
alias rpi-ro='sudo mount -o remount,ro / ; sudo mount -o remount,ro /boot'
alias rpi-rw='sudo mount -o remount,rw / ; sudo mount -o remount,rw /boot'

I should point out that if you have other scripts that rely on running ‘rpi-ro’ and or ‘rpi-rw’ - using aliases is probably not a good idea…

No there isn’t.

Some time ago I was working on a “rpi” script that offered many options from creating a data partition, adding a hdd, switching between a RO OS and standard OS, enabling MQTT, enabling the LAMP etc etc, This was offered to T&G way before the emonPi or the emonPi image was here. I have not pursued this due to the current trend not to ssh in favor of the emoncms config pages.

Here’s my forwarding only emonbase via ssh,

Using username "pi".
Authenticating with public key "pb66" from agent
Linux energyPi 3.18.11+ #781 PREEMPT Tue Apr 21 18:02:18 BST 2015 armv6l
  ___                 ___                        __  __          _ _
 / _ \ _ __  ___ _ _ | __|_ _  ___ _ _ __ _ _  _|  \/  |___ _ _ (_) |_ ___ _ _
| (_) | '_ \/ -_) ' \| _|| ' \/ -_) '_/ _` | || | |\/| / _ \ ' \| |  _/ _ \ '_|
 \___/| .__/\___|_||_|___|_||_\___|_| \__, |\_, |_|  |_\___/_||_|_|\__\___/_|
      |_|                             |___/ |__/                          .org
(type 'emon' for details)
[email protected] ~ $ rpi

The 'rpi' command options

General utilities
   -c | --config    Opens an extended 'raspi-config' user menu
   -h | --help      Display this help for 'rpi' command options
   -u | --update    Updates the Raspbian OS & debian packages
   -l | --lock      Lock (read-only enabled filesystem)
   -o | --open      Open (read-only enabled filesystem)
   -s | --status    Report status information

File system utilities
  -er | --enable-ro     Configures filesystem as read-only
  -dr | --disable-ro    Reverse changes to read-only filesystem
  -ro | --read-only     Lock (read-only enabled filesystem)
  -rw | --read-write    Open (read-only enabled filesystem)
  -ad | --add-datapart  Add a data partition
  -mr | --move-rootfs   Configures OS & rootfs on USB drive
  -em | --enable-mqtt   Enable Mosquitto MQTT v3.1.1 server

[email protected] ~ $ emonhub

The 'emonhub' command options

emonHub utilities
   -c | --config    Configure emonHub by editing emonhub.conf
   -t | --crontab   Opens emonHub's crontab time scheduler
   -h | --help      Display help for 'emonhub' command options
   -r | --restart   Restart the emonhub service
   -s | --start     Start the emonhub service
   -u | --update    Update emonHub via the git repository
   -v | --version   Display the emonHub version installed
   -x | --exit      Exit emonHub by stopping the emonhub service
   -d | --disable   Disable emonHub from starting automatically
   -e | --enable    Enable emonHub to always start automatically

emonHub logging
  -dl | --display-log   Display last 10 & on-going log messages
  -ol | --open-logfile  Open logfile in nano text editor
  -rl | --reset-logfile Reset the current logfile (deletes log!)
  -vl | --view-logfile  View the entire logfile

[email protected] ~ $ emoncms

Note emonCMS has not been fully installed & configured yet
The following commands will be available once emonCMS is installed

The 'emoncms' command options

emonCMS utilities
   -c | --config    Opens emoncms.conf for configuration
   -h | --help      Display this 'emoncms' command options help
   -u | --update    Updates emonCMS via the git repository

emonCMS setup
  -da | --disable-all   Disables the server, database & emonCMS
  -el | --enable-lite   Enables server, database & lite emonCMS
  -ef | --enable-full   Enables server, database & full emonCMS

To install emonCMS use 'emoncms --install' for installer options

[email protected] ~ $

Bill suggested the changes to bash.rc for (RW) and (RO) way back, at the time I actually got the prompt to change colour, red for RO and green for RW but I’ve since lost that feature some where along the line.

EDIT - Oh! I forget I made the “emon” option that listed them all, just spotted above.

[email protected] ~ $ emon

 The 'OpenEnergyMonitor' Open-source Hardware & Software project

    navigation via the 'rpi', 'emonhub' & 'emoncms' commands


The 'rpi' command options

General utilities
   -c | --config    Opens an extended 'raspi-config' user menu
   -h | --help      Display this help for 'rpi' command options
   -u | --update    Updates the Raspbian OS & debian packages
   -l | --lock      Lock (read-only enabled filesystem)
   -o | --open      Open (read-only enabled filesystem)
   -s | --status    Report status information

File system utilities
  -er | --enable-ro     Configures filesystem as read-only
  -dr | --disable-ro    Reverse changes to read-only filesystem
  -ro | --read-only     Lock (read-only enabled filesystem)
  -rw | --read-write    Open (read-only enabled filesystem)
  -ad | --add-datapart  Add a data partition
  -mr | --move-rootfs   Configures OS & rootfs on USB drive
  -em | --enable-mqtt   Enable Mosquitto MQTT v3.1.1 server


The 'emonhub' command options

emonHub utilities
   -c | --config    Configure emonHub by editing emonhub.conf
   -t | --crontab   Opens emonHub's crontab time scheduler
   -h | --help      Display help for 'emonhub' command options
   -r | --restart   Restart the emonhub service
   -s | --start     Start the emonhub service
   -u | --update    Update emonHub via the git repository
   -v | --version   Display the emonHub version installed
   -x | --exit      Exit emonHub by stopping the emonhub service
   -d | --disable   Disable emonHub from starting automatically
   -e | --enable    Enable emonHub to always start automatically

emonHub logging
  -dl | --display-log   Display last 10 & on-going log messages
  -ol | --open-logfile  Open logfile in nano text editor
  -rl | --reset-logfile Reset the current logfile (deletes log!)
  -vl | --view-logfile  View the entire logfile


Note emonCMS has not been fully installed & configured yet
The following commands will be available once emonCMS is installed

The 'emoncms' command options

emonCMS utilities
   -c | --config    Opens emoncms.conf for configuration
   -h | --help      Display this 'emoncms' command options help
   -u | --update    Updates emonCMS via the git repository

emonCMS setup
  -da | --disable-all   Disables the server, database & emonCMS
  -el | --enable-lite   Enables server, database & lite emonCMS
  -ef | --enable-full   Enables server, database & full emonCMS

To install emonCMS use 'emoncms --install' for installer options

[email protected] ~ $

Wow, never seen this before. Looks very tidy.

Yes, if SSH’ing should be a last resort.

Let’s stick with rpi-rw and rpi-ro, too many scripts depend on these commands to consider changing unless there is a very compelling reason to.

Ironically enough, I made the same suggestion about a year ago and got shot down. Go figure.

Not all of us forgot that Bill…

Also the use of apt-get clean was discussed at length way back when Glyn was struggling to get the image on a 4gb card before going down the slim OS route.

Neither of these points take anything away from Andy, they are both good points to be raised again, it just shows how much unused and at the time perhaps unappreciated information is hidden away in the depths of these forums.

1 Like

Aye, you’ve got that right!

Sorry, I do my best :disappointed:

Things will occasionally get missed, this doesn’t mean their unappreciated.

That wasn’t aimed at you specifically, just a general observation of the way things work around here sometimes.

If a good idea doesn’t fit the need of the day it generally gets blown off and lost rather than appreciating it as a good idea on the whole and adding it to a plan or todo list as you have done in this thread, it’s much easier to remove something from a todo list after further consideration than it is to try and remember past things that were not of particular interest at the time.

1 Like

I agree it’s not something worth doing on a whim. especially if it could cause a clash with a fuller script (like the one I was working on). One day I may pick this up again as not all emonbases have to have a webserver running and for some of us ssh if still the preferred option.

Worth noting though, that “rpi” script also had backward compatibility provided by adding

alias rpi-ro='rpi -ro'
alias rpi-rw='rpi -rw'

to the bashrc much like Andy suggested above.

And it also tested if the OS was in RO or RW mode when rpi -rw or -ro was used and only remounted if needed, it also provided a way that scripts could use rpi to “unlock if needed” and then “re-lock if previously locked” so that any scripts using rpi to unlock/lock would not alter the current state set by the user, much more ‘ssh’ user friendly.

1 Like
  then
    PS1='\[\033[01;31m\]\[email protected]\h${fs_mode:+($fs_mode)}\[\033[00m\]:\[\033[01;34m\]\W\[\033[00m\]\$ '
  else
    PS1='\[\033[01;32m\]\[email protected]\h${fs_mode:+($fs_mode)}\[\033[00m\]:\[\033[01;34m\]\W\[\033[00m\]\$ '

Changing the \W to \w gives you the full path to the current working directory and is probably more useful.

Just to put some closure on this thread, I have decided to stick with 4GB as the default emonSD card size. After flashing to a card larger than 4GB to SD card the data partition can be expanded by running:

$ sudo emonSDexpand

Then following prompts.

A new SD card image is currently in beta, see thread:

Thanks, I have made the change :thumbsup: :

1 Like