emonSD next steps: emonhub logging

emonSD
13

So far my overnight test has worked exactly how I wanted. The process I have set up does this;

  1. emonhub.log is written to /var/log/
  2. The emonhub.log file is monitored by monit and when it is over a specified size, monit calls logrotate (not a forced rotate, as the logrotate directives will cause it to do the rotation anyway).
  3. Logrotate is configured to rotate the emonhub.log file at the same specified size as above.
  4. A postrotate directive firstly moves the rotated file to the data folder on disk. The moved file is then concatenated with any existing emonhub.log file and deleted.
  5. The emonhub.log file grows over a 24hr period.
  6. On a daily basis, the emonhub.log file in the data folder is rotated and compressed by the daily run of logrotate.

Over a period of time, a number of days of logs for emonhub can be stored for debugging purposes.

Notes:

  1. This setup is independednt of how the emonhub.log file is generated (in this case by modifying the behaviour of rsyslog).
  2. This setup is independent of where /var/log/ is mounted and any other management of that folder.
  3. monit is a standard apt package (really useful!).
  4. This setup does not require any changes to the standard setup of logrotate or monit other than the addition of a configuration file for each.
  5. The length of time logs are retained for can be set in terms of days.
  6. Setting the size directive reasonably low will minimise data lost on a system crash / reboot, but there are other mechanisms that could mitigate that.
  7. It could easily be replicated for other logs.

This has been a 36hr test so far. Needs to be left a bit longer. It was built by removing the emoncms changes to logrotate on a standard EmonSD card (plus modifications to the generation of the emonhub.log file).

Configuration files (currently).

File /etc/monit/conf-enabled/emonhub-monitrc

check file emonhub.log with path /var/log/emonhub.log
       if size > 2 MB then exec "/usr/sbin/logrotate -v /etc/logrotate.conf"

File /etc/logrotate.d/emonhub

/var/log/emonhub.log {
        rotate 6
        daily
        size 2M
        maxsize 2M
        missingok
#        create 640 pi pi
        copytruncate
        notifempty
        postrotate
             mv /var/log/emonhub.log.1 /home/pi/data/emonhub-log/
             cat /home/pi/data/emonhub-log/emonhub.log.1 >> /home/pi/data/emonhub-log/emonhub.log
             rm /home/pi/data/emonhub-log/emonhub.log.1
        endscript
}

/home/pi/data/emonhub-log/emonhub.log {
        rotate 10
        daily
        compress
        missingok
        notifempty
}

[Edit 2] Checking out this item re the use of CopyRotate rsyslog with logrotate: reload rsyslog vs copytruncate - Server Fault

1 Like