emonSD next steps: log2ram

[pb66 (Paul)]

@borpin

Inherently, our fiddles with the stock image is the reason there is a need to fix this i.e. we mount /var/log to tempfs, so limiting the size, without doing sufficient other actions to prevent it filling up

Amen, I have brought this up so many times in the past, but first I was told it isn’t a problem, then when it was, the hourly logrotation was rolled out as a cure.

@borpin

Perhaps add in an email warning mechanism for when the log folder fills up

This has already been suggested by me previously, I also suggested adding /var/log to the admin page alongside the ram and disc usage.

@borpin

we fiddle with the logging quite a lot to reduce writes this would just be a different fiddle.

We need to fiddle less, full stop! I am not suggesting log2ram is a total solution (especially as it stands now) but it can remove most if not all of those fiddles. This is why I added the olddir logrotate to the standard L2R (log2ram) to try and fill it out to better suit our needs, a forced rotate via monit sounds like a good move (not totally researched yet).

@borpin

For instance, logrotate only works if there is enough destination space to rotate to.

True, but when logrotate has been running for a while it has files to delete to make room for the new ones so it becomes less of an issue after the retention period is reached after start up. But this is a managable issue anyway. The monit trigger to force a rotate when the ram partition is full can simply check for space first, make space (by deleting older logs) if needed or dump the rotations to /dev/null (or simply delete/empty the logfiles in situ) to stop the system failing. This is where we should be focusing are efforts, in a global policy that works transparently for all logs and promotes logfile retention without impacting the card life. NOT fiddling here there and everywhere, removing logs, reducing loglevels, deleting valuable log data etc etc.

@borpin

Not necessarily. If a combination of log2ram, logrotate, journald & rsyslog are setup correctly, then any package (and more will go the journald route I believe) installed will just work with that setup.

Exactly my point!
I’m not against using journalctl were appropriate, I’m just not wanting us to waste time converting everything to use it when it won’t suit everything and it will introduce more issues. I am always for choice! L2R and/or journalctl, MQTT and/or HTTP, JSON and/or CSV etc etc I always try to reign in the sudden revolt to another option, it only because those trying to force the total rollout of what ever is new and shiny to everyone whether they want it or not, that causes me to argue against that position that makes me appear against it, I’m not, but it can seem that way when I am forced to keep going over the same points again and again to retain CHOICE or slow the rollout in order to get it right.

@borpin

(and I don’t think that is what you are saying).

You are right, it is not what I’m saying.

@borpin

I am not wedded to any approach, I just want to discuss and establish a clear strategy to work to in creating an install script that works.

If only that was the position BEFORE making all the changes! There is no point being so sure that something is right to roll out and then being open to discussion after it doesn’t work out so well and the horse has bolted.