Emonhub logging - a discussion

borpin · 11 February 2023 15:15

I have just created an issue on GitHub regarding emonhub logging.

If anyone is interested, please read and comment there.

Disconnect between logrotate and emonhub rotation of log files

opened 02:58PM - 11 Feb 23 UTC

I'm sure this has come up before. There is a disconnect/conflict between the sel…f rotation of logs by `emonhub` and the `logrotate` schemes. There is also inconsistency in log file creation & permissions depending on where it is done. https://github.com/openenergymonitor/emonhub/blob/1215a9f930057a7049d4f2985ce35a195e02c595/src/emonhub.py#L321-L323 https://github.com/openenergymonitor/EmonScripts/blob/master/defaults/etc/logrotate.d/emonhub Add to this the service files and logging and the install scripts for emonhub and EmonScripts and `emonhub` logging is all a bit messed up. https://github.com/openenergymonitor/emonhub/blob/4539b9e928e3f7b5bc3dd1922440cf7e725ac6b9/install.sh#L103-L110 https://github.com/openenergymonitor/emonhub/blob/4539b9e928e3f7b5bc3dd1922440cf7e725ac6b9/service/emonhub.service#L8-L16 There was some discussion here https://github.com/openenergymonitor/emonpi/issues/103 Where the internal python rotation happens, the subsequent rotated log files will not be rotated by logrotate. You will end up with a discontinuous log file (which would not help debugging). As set currently, if the logfile happens to be between 3M and 5M in size when logrotate runs, the file will be rotated and only in this circumstance. I think there should be a systematic review of the logging for `emonhub`. ### My suggestion If there is a no log file specified in the `emonhub.service` file, `emonhub` logging reverts to stderr, which ends up in `rsyslog.log`. This logging can then be directed to a log file using a file `/etc/rsyslog.d/10-emonhub.conf`. ``` template(name="EmonhubMsg" type="string" string="%msg:::drop-last-lf%\n") if ( $programname == 'emonhub' or $programname == 'emonhub.py' ) then { action(type="omfile" file="/var/log/emonhub/emonhub.log" template="EmonhubMsg" FileCreateMode="0644" dirCreateMode="0744") stop } ``` This would then be correctly rotated by logrotate. However, with the current logrotate settings mean that `logrotate` only runs daily, but the rotation settings mean the log is rotated weekly (unless too big). Moving to hourly rotation can be solved by a `logrotate` drop-in to change the timer to hourly with minimal impact on resources and then rotate once the size is greater than 5M. `/lib/systemd/system/logrotate.timer.d/00_emonhub.conf` ``` [Timer] OnCalendar= OnCalendar=hourly ``` Finally, the logrotate config needs amending slightly. Assuming logrotate is run hourly, the file will rotate when greater than 5M (but no earlier), but at least 3M, keep 14 files, compress on next rotation. ``` /var/log/emonhub/emonhub.log { size 5M minsize 3M maxsize 5M rotate 14 compress delaycompress olddir /var/log.old/emonhub createolddir 775 root root renamecopy } ``` Service file would therefore be simplified as well. ``` [Unit] Description=emonHub data multiplexer # The config file lives in /etc/emonhub/emonhub.conf # The log file lives in /var/log/emonhub/emonhub.log Requires=var-log.mount After=var-log.mount network.target [Service] Type=exec ExecStart=/usr/local/bin/emonhub/emonhub.py --config-file=/etc/emonhub/emonhub.conf User=pi Restart=always RestartSec=5 SyslogIdentifier=emonhub [Install] WantedBy=multi-user.target ``` Currently running this on a NOV22 EmonPi. ** Install and update scripts need considering ** Probably needs removing. https://github.com/openenergymonitor/emonhub/blob/4539b9e928e3f7b5bc3dd1922440cf7e725ac6b9/install.sh#L103-L115