Could you clarify this? Im not sure that I follow, from my work and testing on this I definetly needed the log folder… e.g /var/log/emon assuming we move the logs to that location.
Remember the log folder issue is not just an emon* problem. Mosquitto and Openhab to name 2 expect the folder that they create at install time to be there but it is lost on reboot because the log folder is on tmpfs.
You do need that folder, but if the service-runner service unit is creating it’s own log folder (which is the same path) you do not need to create the log folder in rc.local or even in the installer/update scripts, and a step further is the feedwriter and emoncms_mqtt services log to emoncms.log (service-runner doesn’t AFAIR) so even though emoncms itself isn’t a service and cannot create it’s own logfolder, either of those services would have.
Just to clarify further, this isn’t a concern for a regular emoncms only custom install as the installer (script or human) can add the folder to /var/log and it will persist, but in the case of emonpi/SD and any other “log in ram” installs, this is more robust, but more to the point here, there is no reason for any emonhub or emoncms logs to be created in the out-going rc.local.
Ah ok, I see what your saying, we make use of each service’s ability to create /var/log/emon, to ensure that in the event that it does get deleted either service-runner or perhaps emoncms_mqtt creates it. Its not reliant on the install script and is therefore more robust. We would still have it in the install script but at least it would get recreated if deleted for some reason.
I agree we are moving away from rc.local, Im only using it on the current image temporarily, given that the logs are in tmpfs without log2ram. Although are you saying it may be safe enough to depend on the service files themselves to handle this on the current image?
After being one of the very few banging on about it for so many years, it is not something I’m likely to forget!!!
Whilst running the likelyhood of sounding like a broken record, log2 ram will address this accros the board. But just because you have a mop and bucket handy, that’s no reason to be careless about spilling stuff on your floor. We know emonhub and emoncms are likely to be installed to a sdcard (tmpfs log) so there is no point in writing weak services, a couple of lines adds a huge amount of robustness. We might not be able to easily change redis, mosquitto and mariadb (hence L2R!) but that’s no reason to be sloppy elsewhere.
1 Like
I would go as far as to say much safer than rc.local.
I do understand rc.local is going to be replaced. I just don’t see why we are avoiding removing the bits that are addressed in code that will be kept moving forward. eg if emonhub is going to create it’s own log folder eventually, why not now? why add the log folder creation to rc.local, it isn’t needed, even temporarily, just make the service robust enough for use with tmpfs logs and jobdone, makeing it vulnerable because rc.local is currently needed for other things just doesn’t make sense to me.
1 Like
Great, makes sense.
@djh could you give an example of how to use the EnvironmentFile to modify the ExecStart= entry in the emonhub.service? This SE suggests using bash to run the command systemd, EnvironmentFile, re-using variables - how? - Unix & Linux Stack Exchange it that how you would do it?
Just on log files, I am seeing this error in daemon.log
May 27 14:17:03 emonpi cron[331]: 2019-05-27 14:17:03 1hVFUs-0000kZ-3g Cannot open main log file "/var/log/exim4/mainlog": Permission denied: euid=113 egid=117
Do you have an MTA installed? That message looks like it was generated by cron trying to email that something has failed, ordinarily I would expect it to say something like “no MTA installed” but that messages sort of implies there is an MTA installed (exim, included with some Debian distros but not raspbian) but it can’t create it’s log file/folder, perhaps the exim package (if you installed it) created a file and folder but the log in tmpfs lost it on a reboot? Just a guess!
I think it is doable fairly easily by creating the directories/files in the service units (with ExecStartPre). The commands can be inserted into the service units using drop-in files in /etc/systemd/system/<whatever> or even perhaps /run/systemd/system/<whatever>.
I’m not sure why you would need to use bash? Do you have an example of the service file somewhere so we can discuss actualities. I suspect that if you have EMON_HUB_BASE=/some/path in your environment file and ExecStart=$EMON_HUB_BASE/bin/emonhub or whatever in your service unit it will work.
edit: Or ${EMON_HUB_BASE} if required to disambiguate
Yes indeed it is, however in future images we still need L2R to ensure users do not have issues when installing other stuff.
- We should make OUR services as robust as possible, there is no good reason not too.
- We should still install L2R for a more system wide solution to not only logfile creation, but also for persisting log files.
This pretty much means that the other essential services are still covered without the complexity of a overriding “edit” for each service, that’s not to say we absolutely shouldn’t do it, just that it isn’t as easy as writing our own service units and not “essential to make it run” as is the case with the current rc.local mods, L2R works transparently for all (eg possibly exim4 in the preceding posts above).
We could indeed add edit files to each of the servivces restarted in rc.local as a temp fix to the existing image I suppose, but my points above were not aimed as getting shot of rc.local in the existing installs, it was aimed at putting the changes into emonhub and service-runner etc, not holding back because rc.local was still in play.
Ah, I think I misunderstood your point then; in fact I still don’t understand it. I entirely agree with you about the desirability of L2R and making services robust etc. I don’t like the idea of edits either unless there’s no way around them. It was just you predicated something on not being able to modify other services, when it is in fact possible.
It is a standard SDImage. Not added anything else.
[edit]
The only thing I have done is reinstall Logrotate to get back to default conf file. Perhaps that is the issue.
Thanks @djh, if I have for example:
[Service]
EnvironmentFile=/home/pi/emonhub.env
User=emonhub
PIDFile=/var/run/emonhub.pid
ExecStart=$EMONHUB_SRC/emonhub.py --config-file=/home/pi/data/emonhub.conf --logfile=/var/log/emonhub/emonhub.log
....
with emonhub.env:
EMONHUB_SRC=/home/pi/emonhub/src
I get “Executable path is not absolute, ignoring: $EMONHUB_SRC/emonhub.py” I get the same with ${EMONHUB_SRC}
what do you get with
ExecStart=echo ${EMONHUB_SRC}/emonhub.py --config-file=/home/pi/data/emonhub.conf --logfile=/var/log/emonhub/emonhub.log
Aargh! Sometimes I just hate systemd 
I think it will work if you do:
/usr/bin/python $EMONHUB_SRC/emonhub.py --config-file=/home/pi/data/emonhub.conf --logfile=/var/log/emonhub/emonhub.log
or even /usr/bin/env python $EMONHUB_SRC... etc
[edit]: obviously the final version will want $variables for /home/pi/data and /var/log etc.
Unfortunatly still not working, both give:
● emonhub.service - emonHub service description
Loaded: loaded (/home/pi/emonhub/service/emonhub.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Mon 2019-05-27 17:26:31 UTC; 559ms ago
Process: 3358 ExecStart=/usr/bin/env python $EMONHUB_SRC/emonhub.py --config-file=/home/pi/data/emo
Main PID: 3358 (code=exited, status=2)
May 27 17:26:31 emonpi systemd[1]: emonhub.service: Main process exited, code=exited, status=2/INVALI
May 27 17:26:31 emonpi systemd[1]: emonhub.service: Unit entered failed state.
May 27 17:26:31 emonpi systemd[1]: emonhub.service: Failed with result 'exit-code'.
May 27 17:26:31 emonpi systemd[1]: emonhub.service: Service hold-off time over, scheduling restart.
May 27 17:26:31 emonpi systemd[1]: Stopped emonHub service description.
May 27 17:26:31 emonpi systemd[1]: emonhub.service: Start request repeated too quickly.
May 27 17:26:31 emonpi systemd[1]: Failed to start emonHub service description.
May 27 17:26:31 emonpi systemd[1]: emonhub.service: Unit entered failed state.
May 27 17:26:31 emonpi systemd[1]: emonhub.service: Failed with result 'exit-code'.
It at least looks like its going to start when you run restart but then doesnt seem to be getting much further.