I’ve mentioned this a couple of times elsewhere on the forum but the “swappiness” needs reigning in on the emonSD images.
The swappiness is a setting between 0 and 100 that adjusts how aggressively the os swaps page files out of ram on to disk (or sdcard in our case) where 100 is really aggressive and the os tries to free up as much ram as it possibly can and 0 is either off (kernels 3.5>) or “minimum swappiness without disabling it entirely” (kernels >3.5).
Whilst 60 is the Linux default, earlier Raspbian images had the swappiness explicitly set to “1”. More recent Raspbian images are missing the explicit setting and therefore default to 60.
New 2019-04-08 Raspbian lite
pi@raspberrypi:~ $ uname -a
Linux raspberrypi 4.14.98-v7+ #1200 SMP Tue Feb 12 20:27:48 GMT 2019 armv7l GNU/Linux
pi@raspberrypi:~ $ sysctl vm.swappiness
vm.swappiness = 60
verses a much older image
pi@raspberrypi~ $ uname -a
Linux raspberrypi 4.1.19+ #858 Tue Mar 15 15:52:03 GMT 2016 armv6l GNU/Linux
pi@raspberrypi~ $ sysctl vm.swappiness
vm.swappiness = 1
There are many discussions out there on this topic and many different opinions, mainly due to different applications and/or different hardware, some users running high traffic websites from a fast hdd might want high swappiness (for example), but for the emonSD image I think off or at least “1” is better for the life of the sdcard. For ages we used RO images so a swapfile wasn’t accessible regardless of the setting so we know we can run on zero swapfile, with the Oct2018 image the os is not RO so it is using the default “60” which is 60% of the way towards “swap everything you possibly can” and way too high!
Whether swap is used or not will depend on personal application and activity, but a setting of 60 is intended to try and swap out if possible without being overly aggressive about it, so the use of swap doesn’t indicate a fault by itself.
IMO, we are not big users of RAM on the average emonSD so if swap is being depended upon, there is an issue to be resolved (eg the recent emonhub.logs etc) and if there is an issue, it will quickly run out of swap space as we have seen. For the emonSD, the use of swap space does 2 things, prolongs the inevitable crash if there is an issue where memory is being filled and nibbles away at the sdcard life expectancy.
Whilst I agree we have seen the swap space used (filled) when the log partition fills up, but i struggle to understand how the memory can overflow by over 100MB (size of swap file) because the max 50M limit is reached by the logs. Surely if the logs max out at 50MB that leaves 950MB for the system (1050MB including the available swap) so there are possibly other issues at large here and the log partition filling up just reduces the space available to the OS plus adding to it’s load by not allowing any logs to be written, thus causing the jam.
We could just reduce the swappiness to ~1-10 to retain it as an emergency escape lane (which would probably remove Ians 6-10% usage), but IMO any systems reliant on the swap to keep running are simply burning away sdcard life and not addressing any issues that make the system dependent on swap.
If anyone wants to test a lower swappiness, the easy way to do so is using
sudo sysctl vm.swappiness=0
where “0” is the level of swappiness between 0 and 100 (current default 60), this will revert at reboot and/or can be run more than once to adjust further.
The more permanent way is to write a line to a new drop-in file to /etc/sysctl.d/
eg
printf "vm.swappiness = 0" | sudo tee /etc/sysctl.d/50-emonsd.conf
where “50-emonsd.conf” is the file name, must end in .conf
and the “50” is a priority level (I chose 50 as a midpoint) .
to load the changes to that file without a reboot use
sudo sysctl --system
Assuming everything is tickety-boo and swap isn’t needed this could actually make the webpages more responsive because loading from ram cache is so much faster than loading from disk cache (ie swapped out pages), but if we are so short of ram we need swap it might make things slower as any caching will be affected, but do we want faster at the cost of sdcard reliability and life expectancy? It would be better to make sure OEM/emoncms can always work within the limits of the RAM rather than rely on the swapfile.
Just my tuppence worth!