MQTT Log files

emonSD
,
1

I have just set up a Raspberry Pi with EmonCMS (mostly) following the instructions on building the SD card and I ran into some troubles with Mosquitto not starting (even though the service status was running…). It took a bit of hear pulling but turned out to be that I needed to create /var/log/mosquitto and set the owner as with the other services (in rc.local)

So my question is why doe the emonSD image not need this change, or to phrase it another way, what step did i miss ;-)?

FYI the version of mosquitto I installed is;

1474024728: mosquitto version 1.4.10 (build date Thu, 25 Aug 2016 10:12:09 +0100) starting

and the version on the Emon SD image is

1474024926: mosquitto version 1.4.8 (build date Sun, 14 Feb 2016 15:06:55 +0000) starting

and I had to make the following changes to ‘fix’ the issue;

diff --git a/rc.local_jessieminimal b/rc.local_jessieminimal
index 187f78f..611ce43 100755
--- a/rc.local_jessieminimal
+++ b/rc.local_jessieminimal
@@ -8,7 +8,7 @@
 # Will only run if /var/log is mounted in tmpfs
 if ( mount | grep "on /var/log "| grep -q "^tmpfs " )
 then
-  for i in "redis" "apache2" "mysql" "openhab" "logrotate"; do mkdir /var/log/"$i"; done
+  for i in "redis" "apache2" "mysql" "openhab" "logrotate" "mosquitto"; do mkdir /var/log/"$i"; done
   for i in "emoncms.log" "mysql.log" "mqtt_input.log" "redis/redis-server.log" "service-runner.log" "mysql/error.log" "apache2/error.log" ; do touch /var/log/"$i"; done
   for i in "emoncms.log" "mysql.log" "mqtt_input.log" "redis/redis-server.log" "service-runner.log" "mysql/error.log" "apache2/error.log" ; do ""chmod 666"" /var/log/"$i"; done
   chown -R root:adm /var/log/apache2
@@ -16,14 +16,16 @@ then
   chown -R mysql:adm /var/log/mysql
   chown -R openhab:openhab /var/log/openhab
   chown -R pi:pi /var/log/logrotate
+  chown -R mosquitto:mosquitto /var/log/mosquitto

   # Restart Apache & Redis, now the directories are defined
   service apache2 restart
   service redis-server restart
+  service mosquitto restart
 fi

 # Run emonPi Update of first factory boot (~/data/emonpiupdate.log does not exist)
-/home/pi/emonpi/./firstbootupdate
+#/home/pi/emonpi/./firstbootupdate

 sleep 20
 service mqtt_input restart
2

I am fighting a similar battle and could not figure out how to turn on any MQTT-ish type logs! I was about to erase it all and start again! So my answer is “yes” it needs this change. Thank you!

3

Was the firstboot commented out for your own needs (Raspberry Pi with EmonCMS (mostly)) or for a different reason??

EDIT: and does anything need to be done with logrotate to keep the mosquitto.log a reasonable size?

4

I think this was overwriting my settings so I commented it out, will remove it from a PR if one is needed.

Probably, I will have a look.

5

Were you able to solve your Mosquitto not starting issue? I ask because I am experiencing MQTT errors and I cannot figure out what went wrong. I am also trying to determine why there is no info written to the mqtt_input.log (/var/log/mqtt_input.log).

6

Yeah, that change fixed it. To work it out I just ran mosquito from the command line rather than using systemd to start it.

7

Thanks to @jeremypoulter and @Jon for helping to debug this. The fix has now been committed to emonpi, after emonpi update and a reboot the mosquitto log file folder will be created and mosquitto V1.4.10 will now run happily :slight_smile:

1 Like
8

Thanks, was just about to do this :wink: Did you look at if anything needs doing with logrotate?

1 Like
9

Current logrotate config should catch the mosquitto logs:

1 Like
10

Cool, that was the only thing I had not got around to checking. I can also confirm that this fixes update of an EmonPi SD card image.

1 Like
11

@glyn.hudson

Can you check on the mqtt_input.log (/var/log/mqtt_input.log)? When I was trying to troubleshoot MQTT issues (and my odd slowness issue) I noticed mqtt_input.log was always empty. I can see it being created but nothing seems to add INFO, WARN or debug data.

12

Output from /var/log/mqtt_input.log

pi@emonpi:~ $ cat /var/log/mqtt_input.log
/usr/bin/php already running.
13

Is there no other info added if there are issues? Like INFO, WARN or ERROR type messages?

14

/var/log/mqtt_input.log is just the log output from the service /etc/init.d/mqtt_input.

The log from the actual mqtt script emoncms/scripts/phpmqtt_input.php is routed to /var/log/emoncms.log.

Here is the output for me when $log_level = 1; // INFO in settings.php

Make sure you’re running the latest master branch. I just merged a change to allow sub base topics as well as slightly improved logging.

15

Tnx! I changed the $log_level = 1; for INFO just so I understand what happens (besides filling up a log quickly).

I went searching (googling, binging,etc) but could not find what a sub base topic is or why it might help… So what is sub base topic? why would I need sub base topic? :thinking:

16

Currently, we use emon as the sub-topic meaning the mqtt input scripts subscribers to emon/# . Any messages posted to higher level topics with base-topic emon are also subscribed to e.g emon/emontx/power

A base sub-topic allows is a sub-topic as a base topic e.g. emon/house the mqtt script will now subscribe to emon/house/# meaning all messages posted to emon/house and above will be posted to emoncms e.g. emon/house/power1 but all lower level messages e.g. emon/power will be discarded.

I’m sure it could be explained more elegantly! Its a niche use case; the organisation that Ben works for has got lots of mqtt topics and he needed to be able to define a base sub-topic to select only the data he wanted to be posted to emoncms.