It’s supposed to be the fix-all solution to everything, and to some extent it is, but we’re at the point where there are some stupid dependancies relating to desktop type services that now rely on systemd, crazy IMO.
As for logging - there are usually two way to specify the log, in the unit file or in the config for the application / process.
Looking at the old init.d init script - you were setting the log file there previously;
NAME=mqtt_input
DESC="Daemon for the emoncms MQTT script"
PIDFILE="/var/run/${NAME}.pid"
LOGFILE="/var/log/${NAME}.log"
DAEMON="/usr/bin/php"
DAEMON_OPTS="/var/www/emoncms/scripts/phpmqtt_input.php"
Since this is the case, we should be doing the same in the systemd unit file, and this is where you run into problems.
Using the init.d - you were actually redirecting the output from the application to STDOUT to a log file, you can replicate that in systemd - kinda…
I have modified my version of the unit file as follows;
# Systemd unit file for mqtt input script
# INSTALL:
# sudo cp /var/www/emoncms/scripts/mqtt_input.service /etc/systemd/system/mqtt_input.service
# RUN AT STARTUP
# sudo systemctl daemon-reload
# sudo systemctl enable mqtt_service
# START / STOP With:
# sudo systemctl start mqtt_input
# sudo systemctl stop mqtt_input
# VIEW STATUS / LOG
# sudo systemctl status mqtt_input -n50
# where -nX is the number of log lines to view
###
#
# All Emoncms code is released under the GNU Affero General Public License.
# See COPYRIGHT.txt and LICENSE.txt.
#
# ---------------------------------------------------------------------
# Emoncms - open source energy visualisation
# Part of the OpenEnergyMonitor project:
# http://openenergymonitor.org
###
[Unit]
Description=Emoncms MQTT Input Script
After=mosquitto.service mysql.service redis.service
[Service]
Type=forking
ExecStart=/bin/sh -c '/usr/bin/php /var/www/emoncms/scripts/phpmqtt_input.php 2>&1 > /var/log/mqtt_input.log &'
# Restart script if stopped
#Restart=always
Restart=on-failure
# Wait 60s before restart
RestartSec=60
[Install]
WantedBy=multi-user.target
One other note - running the update failed on my Pi - the init.d script was removed, but the systemd unit file was not added to /etc/systemd/system - check that you have the file system writable at that point in the script…
Also because you are modifying systemd - make sure you call “systemctl daemon-reload” after adding the file, and then make it active with “systemctl enable mqtt_input”.
I have not checked your update script yet to see where these errors come from yet…
I should point out that all my changes do - is revert to using the /var/log/mqtt_input.log rather than using /var/log/daemon.log as systemd will do by default.
You / We still need to work out why when I updated the init.d script was removed but the systemd script was not added.
I tried sudo journalctl -f -u mqtt_input -o cat | grep emontx ( and some similar searches) before changing anything and found it worked ok prior to the tag being added, also there is no time and date printed can that be set in command line?
Since updating emonpi I am unable to restart the service and have an error message
pi@emonpi(rw):~$ sudo systemctl restart mqtt_input.service
Failed to restart mqtt_input.service: Unit mqtt_input.service failed to load: Bad message. See system logs and 'systemctl status mqtt_input.service' for details.
pi@emonpi(rw):~$ systemctl status mqtt_input.service
● mqtt_input.service - Emoncms MQTT Input Script
Loaded: error (Reason: Bad message)
Active: active (running) since Thu 2017-01-05 13:34:59 UTC; 22h ago
Docs: https://github.com/emoncms/emoncms/blob/master/docs/RaspberryPi/MQTT.md
Main PID: 1832 (php)
CGroup: /system.slice/mqtt_input.service
└─1832 /usr/bin/php /var/www/emoncms/scripts/phpmqtt_input.php
I have had a look and see no obvious (to me) reason, the update was done after the typo corrections to the comment lines.
Since the above search worked prior to the update and yet the “Docs” url has been added to the status output I’m unclear if the new unit is in service or not, but the daemon.log has not changed, I expected to see the tag there or is the daemon.log an additional log to a seperate systemd internal log? if it is, then great we can redirect what is going to the daemon.log to the mqtt_input.log and the systemd/systemctl/journalctl internal log will remain intact perhaps?
I notice @glyn.hudson that you have also added a commented out option to log to mqtt_input.log, but won’t that get overwritten at every emonpi update?
What changes are these? Sorry if I missed something. As I’m typing this, your order has just been shipped
Woops, my fault. I noticed a typo in the unit file which a corrected a few seconds later on the master branch. I created a PR to merge into stable but forgot to merge it…until now. Please try and update again
@glyn.hudson thanks the correction fixed the unit issue, unfortunately your alternative log option doesn’t seem to work, even after reloading and restarting the unit the logs still go to the daemon.log, although I can now see the “mqtt_inpu” tag in place of the “php” tag.
pi@emonpi(rw):~$ tail /var/log/daemon.log
Jan 6 13:06:35 emonpi mqtt_input[22099]: emon/emonpi/3 0
Jan 6 13:06:35 emonpi mqtt_input[22099]: emon/emonpi/4 0
Jan 6 13:06:35 emonpi mqtt_input[22099]: emon/emonpi/5 0
Jan 6 13:06:35 emonpi mqtt_input[22099]: emon/emonpi/6 0
Jan 6 13:06:35 emonpi mqtt_input[22099]: emon/emonpi/7 0
Jan 6 13:06:35 emonpi mqtt_input[22099]: emon/emonpi/8 0
Jan 6 13:06:35 emonpi mqtt_input[22099]: emon/emonpi/9 0
Jan 6 13:06:35 emonpi mqtt_input[22099]: emon/emonpi/10 0
Jan 6 13:06:35 emonpi mqtt_input[22099]: emon/emonpi/11 0
Jan 6 13:06:35 emonpi mqtt_input[22099]: emon/emonpi/rssi 0
and that does send the logs to mqtt_input,log or more importantly, does not send them to ‘daemon.log’, (thanks Andy) however there is no time and date, I have not yet checked to see if this is due to the actual script not attaching it or if it is getting removed en-route.
Apologies, I missed your post above. Got it now, thanks.
Is possible I would like to move to using the systemd journal log, this seems to be the preferred way to go ‘all-in’ with systemd. This is how nodeRED runs. I have been looking at the nodeRED unit file as an example.
However, it would be good to give the option for users to log to mqtt_input.log if they prefer as a commented out option in the unit file which can easily be enabled.
I have updated the unit file to use @Andy_Taylor logging code as an alternative commented out option. I don’t quite understand what is meant by forking. I will leaf type=forking commented out.
If you have a choice then use Type=simple for your systemd services. It is simpler and systemd will take care or proper process deamonizing, restarts and shutdowns.
Type=forking is for historical services that cannot be run in foreground. In this case process itself will take care of its deamonizing, but systemd should know the child pid somehow so you need to provide PIDFile= option as well.
I choose type=idle to ensure the script starts after all other services have started. Here is a full explanation of the different service types:
I’ve just tested again on a fresh emonPi and the update script worked. Could it be that you have un-committed changes to either ~/emonpi or /var/www/emoncms folders on your pi? Try git pulling these folder manually.
Glyn - anything is possible - my Pi gets quite messed with while I am working on things - I’ll re-image it in the next few days and try it again - if your update works - its almost certainly an issue with mine.
I will also take a look at the log mqtt logs before / after the update and see if I can bottom that out.
Having played with this a bit this afternoon, I have discovered the “Type=forking” is required for Andy’s code to work, however I have also figured out that the mqtt_script is both “logging” and “echoing” messages and the messages we are trying to redirect are std out not “log messages” as I expected. eg
I have found that the actual “log messages” are directed to emoncms.log if the loglevel is set correctly in settings.php and that the mqtt_input.log is superfluous.
This is why there is no timestamp when redirected.
I have run though phpmqtt_input.php and commented out most of (but not all) of the echo lines and returned the unit to it’s standard form and now the logging works as expected.
Just deleted my init.d script to start mqtt_input, and got the following results on a ‘self-build’ emoncms (up to date with git).
sudo cp /var/www/emoncms/scripts/mqtt_input.service /etc/systemd/system/mqtt_input.service sudo systemctl daemon-reload
…everythink ok so far, but when I try and enable the service; sudo systemctl enable mqtt_service
I get; Failed to execute operation: No such file or directory
If I look in the /etc/systemd/system directory, I can see the mqtt_input.service file is present.
Checking the log, I see; Loaded: loaded (/etc/systemd/system/mqtt_input.service; disabled) Active: inactive (dead)
I can manually start the service OK; sudo systemctl start mqtt_input
…but as the service is not enabled, it won’t restart after a reboot.
I have corrected the typo in the readme.md, but I think that section needs a re-write and the old instructions resurrecting as per my comment on the commit.
I’ve done a quick re-write to add back the section which I wrote in Oct 2015, which uses init.d instead of systemd as an alternative solution for older OS’s.
…For consideration.
