I have just suffered the /var/log filling up issue. The symptom is everything seems to stop.
This is a emonSD-17Oct19 based system that I have just updated.
The issue is that logrotate has changed and this is creating an issue with the logrotate and the changes to emonhub user. This will happen if you update the OS (currently the update doesn’t do that).
To confirm use the command
[email protected]:/var/log $ df -h /var/log
Filesystem Size Used Avail Use% Mounted on
log2ram 50M 50M 0 100% /var/log
To find the largest file in the log folder
sudo find /var/log -type f -printf "%s\t%p\n" | sort -n | tail -1 | cut -f 2
If you try and manually trigger logrotate you get
[email protected]:/var/log $ sudo /usr/sbin/logrotate /etc/logrotate.conf
error: skipping "/var/log/emonhub/emonhub.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation.
If everything has stopped, SSH into the Pi and
sudo rm /var/log/emonhub/emonhub.log.*
sudo chown pi:root emonhub
sudo systemctl restart emonhub
systemctl status emonhub.service
sudo /usr/sbin/logrotate /etc/logrotate.conf
Should clear it.
reboot to be sure.