Thanks @pb66 I’ve noted this for discussion with @glyn.hudson
Yes interested! Could you document a step by step guide taking the latest image and modifying it to use log2ram? @glyn.hudson and I can then follow along here and try it
And contrary to the impression I may have given, I too am interested. So please do. I will add more in a bit.
I’ve discussed this with @glyn.hudson and we had a look at the log2ram repository. The ability to avoid the /var/log setup currently in rc.local as you describe @pb66 is really interesting. Is it possible to turn off the syncing to disk? We are keen to avoid writing the logs to disk, I think this has been discussed before. Is there a way the behaviour could be user selectable? will log2ram work with older distributions, /etc/init.d scripts etc?
Glyn has also just tested the logrotate script in the emoncms repository that we had running on the previous image emoncms/scripts/logger at master · emoncms/emoncms · GitHub (this was mistakenly left out in this current image). The script works well, rotates and deletes the logs but does of course require the rc.local entries.
We’ve also looked again at Better handling of files in tmpfs /var/log - #13 by pb66, the requirement for /var/log/directory/logfile.log structure seems to be a sticking issue. We also need to rotate /var/log/daemon.log and /var/log/syslog.
For the current emonpi image with the existing rc.local, solution 2 will fix the immediate problem of the logs filling up quickly. Glyn’s keen to merge that in to the update process in the interim so that we can then consider a better solution as part of the longer term plan
As somebody who wasn’t involved in any previous discussions and whose initial opinion would be the opposite, I’m keen to read these previous discussions, or see a precis of the arguments in favour of discarding logs (which logs, all logs?), if you can supply either.
I don’t really see why. We are not suggesting writing on a continual basis to the SDcard (at worst) but rotating the logs from tmpfs to a semi permanent storage location to aid debugging at a later time.
Logrotate can easily do that and keep the logs for a specified period of time.
No, that is the core of how it works, I guess you could use a really long sync interval eg daily, monthly etc but then if users install a package from apt-get and there is a powercut or the plug is pulled accidentally, it may fail to reboot properly as the newly created logfiles will not be there. By syncing hourly, as long as power isn’t interrupted within 1 hour max, the new logs and folders will be persisted, as they will at a controlled reboot also. The persistence to disc IS the advantage of using log2ram.
I’m with the others that this is a bad idea IMO. This is the coding and development equivalent of poking your eyes out and sticking you fingers in your ears.
It’s only a few MB every hour, it will not take up space, it will not wear the card and it will not take much resource. But it will provide enormous amount of help towards debugging and development.
Cast you mind back to the issues with rfm2pi/emonpi’s locking out (rfm’s needing re-initialising) and then all the issues with emonhub interfacer threading and emoncms mqtt that could have been fixed so much sooner if we had more logs available sooner.
So many users just instinctively reboot.
This thread (Reboot Required Every 20 Days Or So To Restore Logging - #31 by TrystanLea) is a year old and we cannot help because we have no info, each time a user reboots and then asks what could it have been, we have to guess rather than inspect the logs and give some proper answers.
Ok we will consider persisting the logs, we would like to check the write load and see what can be done to reduce the logging level where it is not needed. E.g only logging at warning or error and above. daemon.log is also mirrored in syslog.
@pb66 do you use rsync with log2ram? I see that option in the config. Do you also disable the mail option? seems like that would be sensible.
Looks like its fairly straightfoward to disable the mirrored logging to syslog for logging also destined to daemon.log.
sudo nano /etc/rsyslog.conf
change:
*.*;auth,authpriv.none -/var/log/syslog
to:
*.*;auth,authpriv.none,daemon.none -/var/log/syslogNo it is apparently slower.
I don’t recall changing it but it won’t work if there’s no email server configured, IMO it would be a good option to have since it will email you if/when the log partition is short of space. Not a bad idea IMO.
Thanks, I wonder if more processor use in order to reduce disk wear is a valid trade-off to make in this case…?
mb = milli bits, as I understand it. 
At the risk of repeating myself yet again, the rules for the multipliers are:
If the unit is named after a person, that has a capital letter, else
if it’s multiplying by more than 1, and the capital letter is not already taken, it’s a capital letter, else,
it’s lower case.
so K = Kelvin, which forces k = kilo by the second rule
but S = Siemens and s = seconds
M = Mega, G = Giga, T = Tera, etc
m = milli, μ = micro, p = pico, etc
And it seems to be a convention that b = bit, B = byte.
1 Like
Thank you for bring all this to my attention, I’ve deleted my comments since they were so factually incorrect.
Not known to be a problem 
…
Never knew that so it was worth repeating.
I do remember about GB v GIB though.
millibar surely? 
No, that’s hPa 
It really cheeses me off when people who one would expect to know better write about UHF frequencies in Mhz. Heinrich should be turning in his grave.
(Yes, I wasn’t thinking - comes of posting whilst watching TV.)
1 Like
I’ve just got some more information about log2ram to share. I’m not sure where the best place to post it is: either here or maybe emonSD next steps - #20 by TrystanLea but I decided to post here. Please move it if appropriate.
OK, so a couple of things:
(1) I asked on the systemd list about how to configure journald to still write to persistent store even after /var/log is made volatile. See [systemd-devel] logging in RAM and journald configuration issue if you’re interested but TL;DR is to make another persistent directory just for journald and bind mount /var/log/journal to it (How about /var/log.journal ?) That gives us the best of both worlds and I suggest we implement it.
(2) In researching the above I looked at log2ram.service and noticed its Before= line and in particular ‘apache2.service’ on that line. Does that not suggest that every service that needs to log in /var/log needs adding to that line as well? In particular redis but also other things that get mentioned frequently. That makes the whole idea a bit more fragile, so it would be nice if it wasn’t true but then if not why is apache listed there?
That defeats the object of putting the log to volatile i.e minimising the number of writes to the SDcard to prolong its life.
So this may well be a vain attempt to ensure the log files are created before the other unit starts. Actually it is very difficult to do and probably futile. That Paul has found it to work is probably more luck than design if that is the purpose of the before= statement…
systemd-tmpfiles is a more robust mechanism to create tmpfiles and there was a script posted here somewhere to generate the right config file based on existing logs.
Please read my posts early in the thread where I describe the purpose and sanity of this.
You’re probably thinking of Better handling of files in tmpfs /var/log - #13 by pb66
Another hint from the systemd-devel mailing list is that the Before dependency of journald.service should instead be for systemd-journal-flush.service