Phantom Duplicated MQTT Topics after reboot

It’s hard to debug something that happens so rarely. I’ve only seen this happen once in 100’s of reboots on many emonPi’s!

Just to summarise where we are at:

ISSUE

A new set of Emoncms Inputs for the same node are created by the Emoncms MQTT input script. These inputs have exactly the same names as the old ones. The MQTT input script obtains the names of the inputs from the MQTT topic names.

e.g The MQTT topic emon/emontx/power1 results in an Emoncms Input called power1 from emontx node. Any data published to MQTT base topic (default emon/) will appear in Emoncms inputs.

Here is a screen grab illustrating the issue, when this occurs the user needs to move their input processes to the newly created inputs to keep their feeds updating:

POSSIBLE CAUSES

  1. Publisher - emonHub publishing a slightly different mqtt topic name
  2. Broker - Mosquitto MQTT broker corrupting the topic name somehow
  3. Subscriber - Emoncms PHP input script which uses Mosquitto-PHP library corrupting/changing the topic name when subscribing

If I had to guess I would guess the that subscriber (3.) would be the culprit. The Mosquitto Debian repository is called libmosquitto-dev and the PHP Mosquitto library is called Mosquitto-alpha . Dev and alpha don’t inspire confidence!

See how MQTT input script and Mosquitto PHP is setup on the latest emonPi image here.


Debug

I suppose a way to try and get to the bottom of this would be to setup a log file to log all MQTT messages then when the issues occurs try and see if it holds any clues. If the MQTT messages all look correct then the problem must be with (3.) the subscribers. I will setup a test…

Update:

If anyone is interested I have used nodeRED that’s built into the latest emonPi image to log all MQTT messages on the emon/ topic to a log /mnt/1TB/log/emonpimqtt.log file together with the topic name, payload and time + date. I have mounted an external hard-drive to on the emonPi to preserve space on the emonPi and avoid hammering the SD card. This log is going to get BIG!

Here is the nodeRED flow:

[{"id":"286b0130.d794fe","type":"mqtt-broker","z":"","broker":"localhost","port":"1883","clientid":"","usetls":false,"verifyservercert":true,"compatmode":true,"keepalive":"15","cleansession":true,"willTopic":"","willQos":"0","willRetain":null,"willPayload":"","birthTopic":"","birthQos":"0","birthRetain":null,"birthPayload":""},{"id":"6e2c3d80.227114","type":"file","z":"41656479.be9a9c","name":"","filename":"/mnt/1TB/log/emonpimqtt.log","appendNewline":true,"createDir":false,"overwriteFile":"false","x":890,"y":290,"wires":[]},{"id":"c58edf00.c2873","type":"mqtt in","z":"41656479.be9a9c","name":"","topic":"emon/#","broker":"286b0130.d794fe","x":446,"y":304,"wires":[["18439c97.f2f4d3"]]},{"id":"18439c97.f2f4d3","type":"function","z":"41656479.be9a9c","name":"","func":"var time = new Date().toString();\nvar newMsg = { payload: msg.topic + \",\" + msg.payload +\",\" + time};\nreturn newMsg;","outputs":1,"noerr":0,"x":610,"y":294,"wires":[["f5833444.a2dc98","6e2c3d80.227114"]]},{"id":"f5833444.a2dc98","type":"debug","z":"41656479.be9a9c","name":"","active":false,"console":"false","complete":"false","x":731,"y":201,"wires":[]}]