MQTT no longer working after update from emonSD-03May16 to most current version

cormad · 6 November 2017 13:42

MQTT no longer working after I updated from emonSD-03May16- RELEASE to most current version. I followed these instructions;

You will get all the important security updates and Emoncms updates by running:
Admin > emonpiupdate in local Emoncms and upgrading Debian packages

rpi-rw
sudo apt-get update && sudo apt-get upgrade
sudo apt-get dist-upgrade 
sudo reboot

When given the option choose to keep the version of php.ini currently installed

Not sure what to try next ??

glyn.hudson · 6 November 2017 16:39

Have you also ran emonPi update?

Login to local Emoncms e.g. http://emonpi.local/emoncms
Then select Setup > Admin > emonPi update
Wait for it to finish then reboot the system.

Its it’s still not working please post your emonhub log file. This can be downloaded from the Setup > emonHub section of local Emoncms.

cormad · 6 November 2017 21:37

Hi Glyn,

Yes I ran emonpi update and then;

rpi-rw
sudo apt-get update && sudo apt-get upgrade
sudo apt-get dist-upgrade
sudo reboot

Note: I never saw this option on the update “When given the option choose to keep the version of php.ini currently installed”

this is from emonhub log file

2017-11-06 21:18:11.918|WARN|phpmqtt_input.php|Starting MQTT Input script
2017-11-06 21:30:42.130|WARN|phpmqtt_input.php|Not connected, retrying connection
2017-11-06 21:30:45.047|WARN|phpmqtt_input.php|Starting MQTT Input script
2017-11-06 21:30:46.193|WARN|phpmqtt_input.php|Not connected, retrying connection
2017-11-06 21:30:52.001|WARN|phpmqtt_input.php|Not connected, retrying connection
2017-11-06 21:30:58.001|WARN|phpmqtt_input.php|Not connected, retrying connection
2017-11-06 21:31:04.001|WARN|phpmqtt_input.php|Not connected, retrying connection
2017-11-06 21:31:10.000|WARN|phpmqtt_input.php|Not connected, retrying connection

glyn.hudson · 7 November 2017 16:30

That is actually the emoncms log file. Cold you post to emonhub log file?

It seems there is an issue with MQTT. How difficult would it be to give me SSH access? This could be done using https://www.dataplicity.com/ if you don’t want to port forward?

Alternatively the next step would be to figure out why MQTT service us not running:

sudo service mosquitto status

cormad · 7 November 2017 19:00

Hi Glyn,

I have no issue giving you SSH access, just need to figure out how. I’m in the eastern standard time zone. I will post the emonhub log file tonight and also check the status.

cormad · 7 November 2017 21:32

Hi Glyn,
Info from sudo service mosquitto status

pi@emonpi(ro):~$ sudo service mosquitto status -1
● mosquitto.service - Mosquitto MQTT Broker
  Loaded: loaded (/lib/systemd/system/mosquitto.service; enabled)
  Active: failed (Result: start-limit) since Mon 2017-11-06 21:43:53 UTC; 23h ago
    Docs: man:mosquitto(8)
          https://mosquitto.org/
 Process: 678 ExecStart=/usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf (code=exited, status=1/FAILURE)
Main PID: 678 (code=exited, status=1/FAILURE)

Nov 06 21:43:53 emonpi systemd[1]: Unit mosquitto.service entered failed state.
Nov 06 21:43:53 emonpi systemd[1]: mosquitto.service holdoff time over, scheduling restart.
Nov 06 21:43:53 emonpi systemd[1]: Stopping Mosquitto MQTT Broker...
Nov 06 21:43:53 emonpi systemd[1]: Starting Mosquitto MQTT Broker...
Nov 06 21:43:53 emonpi systemd[1]: mosquitto.service start request repeated too quickly, refusing to start.
Nov 06 21:43:53 emonpi systemd[1]: Failed to start Mosquitto MQTT Broker.
Nov 06 21:43:53 emonpi systemd[1]: Unit mosquitto.service entered failed state.
Nov 06 21:44:00 emonpi systemd[1]: Starting Mosquitto MQTT Broker...
Nov 06 21:44:00 emonpi systemd[1]: mosquitto.service start request repeated too quickly, refusing to start.
Nov 06 21:44:00 emonpi systemd[1]: Failed to start Mosquitto MQTT Broker.
pi@emonpi(ro):~$

cormad · 7 November 2017 21:37

Hey Glyn,

emonhub log file

2017-11-07 21:34:17,728 INFO     MQTT       Could not connect...
2017-11-07 21:34:18,053 DEBUG    MainThread SIGINT received.
2017-11-07 21:34:18,053 INFO     MainThread Exiting hub...
2017-11-07 21:34:18,730 INFO     MainThread Exit completed
2017-11-07 21:34:19,437 INFO     MainThread EmonHub emonHub emon-pi variant v2.0.0
2017-11-07 21:34:19,437 INFO     MainThread Opening hub...
2017-11-07 21:34:19,438 INFO     MainThread Logging level set to DEBUG
2017-11-07 21:34:19,438 INFO     MainThread Creating EmonHubJeeInterfacer 'RFM2Pi' 
2017-11-07 21:34:19,440 DEBUG    MainThread Opening serial port: /dev/ttyAMA0 @ 38400 bits/s
2017-11-07 21:34:21,442 WARNING  MainThread Device communication error - check settings
2017-11-07 21:34:21,444 INFO     MainThread Setting RFM2Pi frequency: 433 (4b)
2017-11-07 21:34:22,446 INFO     MainThread Setting RFM2Pi group: 210 (210g)
2017-11-07 21:34:23,448 INFO     MainThread Setting RFM2Pi quiet: 0 (0q)
2017-11-07 21:34:24,449 INFO     MainThread Setting RFM2Pi baseid: 5 (5i)
2017-11-07 21:34:25,451 INFO     MainThread Setting RFM2Pi calibration: 110V (2p)
2017-11-07 21:34:26,454 DEBUG    MainThread Setting RFM2Pi subchannels: ['ToRFM12']
2017-11-07 21:34:26,455 DEBUG    MainThread Setting RFM2Pi pubchannels: ['ToEmonCMS']
2017-11-07 21:34:26,456 INFO     MainThread Creating EmonHubMqttInterfacer 'MQTT' 
2017-11-07 21:34:26,457 INFO     MainThread MQTT Init mqtt_host=127.0.0.1 mqtt_port=1883 mqtt_user=emonpi
2017-11-07 21:34:26,460 INFO     MainThread Creating EmonHubEmoncmsHTTPInterfacer 'emoncmsorg' 
2017-11-07 21:34:26,561 INFO     MQTT       Connecting to MQTT Server
2017-11-07 21:34:26,563 INFO     MQTT       Could not connect...
2017-11-07 21:34:27,665 INFO     MQTT       Connecting to MQTT Server
2017-11-07 21:34:27,667 INFO     MQTT       Could not connect...
2017-11-07 21:34:28,769 INFO     MQTT       Connecting to MQTT Server
2017-11-07 21:34:28,770 INFO     MQTT       Could not connect...
2017-11-07 21:34:29,872 INFO     MQTT       Connecting to MQTT Server

jsroques · 7 November 2017 22:31

Hello,

Just created another post about the same problem here : Mosquitto won't start on boot after raspbian and emonsd update. It seems Mosquitto is restarted “too early” after creating the log file directory.

First, try to start mosquitto manually :
sudo service mosquitto start

If it works, could you try to make a small change to /home/pi/emonpi/rc.local_jessieminimal (I recommend you do a backup of the file before…) ?

After line 22 (comment for service restart), add sleep 11 (line 23).

 22   # Restart services,they should run happy now log dir's are created
 23   sleep 11
 24   service apache2 restart
 25   service redis-server restart
 26   service mosquitto restart
 27   service supervisor restart

Then, reboot !

Could you confirm it works (or not) ?

I am not sure it is the best / cleanest solution though…

JS

cormad · 7 November 2017 22:48

Hi JS,

You rock . I can confirm that works !

Much Thanks

glyn.hudson · 8 November 2017 15:43

Thanks @jsroques

I will temporary add this change to the rc.local script. This change will be automatically pulled in by all emonPi’s:

let’s continue the discussion on the other thread how starting the services can be better engineered going forward: