Interesting, that’s for the invite. Have emonSD build are you running?
Yes, good point. I’m also running openHAB and have noticed this. This proves that the new mqtt topic name is exactly the same. Maybe the name just changed for a second or two which triggered emoncms to created new inputs.
@glyn.hudson I’m running a home brew image based around your EMONPI image build instructions from the middle of February. To this I added the new MQTT setup from the forum instructions, and also the feed creation and mqtt_input fixes. Its running on a Pi3 with RFM69 without the display script as I need access to all my GPIO’s. It also doesn’t have Node Red or Lightwave.
I just had this duplicate name for the first time - I was updating node-red to publish more topics under the same hierarchy and the original input went n/a and three new ones popped up, one with the same name. Could this be something to do with the way the MQTT subscriber iterates through multiple inputs?
@glyn.hudson I’m not entirely sure, although I seem to remember someone mentioned retained messages as a possible problem when I reread the thread on the old forum before I started this one
I run a non OEM copy of Openhab, and the value on my system was set, yet on the line above in openhab.cfg it says:
# Optional. True or false. Defines if the broker should retain the messages sent to
# it. Defaults to false.
Suggesting that “true” isn’t a default.
In the V1.8 Runtime I just found in my downloads folder it isn’t set at all, and as I don’t remember setting it, its either a default from the V1.7 days or something I’ve done whilst troubleshooting a long time ago, and definitely before the MQTT changes that lead to the problem appearing.
I can understand how the retained messages setting would be useful, but on a read only file system its likely that it doesn’t work too well unless wherever it retains them has been moved to a writable location. In my case Openhab lives in /home/pi/data/openhab so it can write what it wants, but not knowing about this setting until now everything is setup to not use it.
I’ll leave it at false and report what happens.
I raised the issue of retained messages here as the likely cause back in March.
Further reading about them can be found in this guide. @kevin - even though you’ve set it to false, you may need to delete the retained messages which are slushing about in your system. The guide link above explains how, and I flushed mine by setting up a MQTT node in node-red for each of the ‘ghost’ MQTT ID’s, with a zero byte payload and retain flag set.
I changed the ID for each of the ghosts in turn, sending just one message to each, and they have not returned since.
As a test, why not just delete half of them initially, and then see what, if any ghosts reappear.
@Paul@glyn.hudson, when I looked into this back then I noted that mosquito.conf referenced a retained messages file on the read only part of the file system. Is it worthwhile amending this and the documentation so it is redirected into somewhere like /tmp that is stores in RAM assuming the permissions are set correctly? That way it can’t retain anything between reboots. (Never quite finished working on this back then due to life getting in the way)
Persistence is turned off in Mosquito by default on the current emonSD image. Your correct that persistence_location /var/lib/mosquitto/, is mentioned however since persistence is turned off this location will not be used.
Here is the default mosquitto conf on the image.:
pi@emonpi:~ $ cat /etc/mosquitto/mosquitto.conf
# Place your local configuration in /etc/mosquitto/conf.d/
# A full description of the configuration file is at
log_dest file /var/log/mosquitto/mosquitto.log
I’ve been looking at this issue in my local installation. I think the issue has to do with phpmqtt_input.php and input_model.php in the EmonCMS project. After a reboot, they load a list of node names and input names from Redis (assuming it is enabled), and use that list of channels as a basis for deciding when to create new inputs in the future.
The problem is (at least on my installation) that Redis doesn’t appear to save that info across reboots, and so it feeds an empty list of nodes and channels into EmonCMS.
I switched the code to load the list from MySQL, and suddenly everything is back to normal (almost, I did have to delete all of the extra channels from MySQL).
I’m not sure if this was the right way to fix this, but hopefully it helps someone figure it out. I edited input_model.php around line 126 to always use the MySQL option.
Sorry to wake an old thread but did anything happen with tracking this down in the end?
I recently “lost” my emonpi for a while after a brown out caused it to shutdown and the network switch is was connected to hard failed. Now replaced, plugged the SD card into another RPi and let the long fsck complete before putting it back in place and now I have all duplicated inputs.
Before I manually fix this I am doing a full backup and update, but would love to know if this will likely happen again?
A while back MQTT input script was changed to use systemd, it should wait for msql and redis to be running:
However, redis and apache get started in rc.local after the log folders are created in /var/log, this is need since the /var/logis mounted in ram as tmpfs. I can’t see any harm in added a delay into the MQTT input systemd service to ensure all services are up and running properly.
Are you familiar with how to delay a systemd service from starting? It looks like we would need to involve a .timer file
Or better still write a new service and prioritize it to run early in the boot process just to create all these log files before any of the affected services need them, that way the system can work as it was intended rather than being undermined by all the entries in rc.local that could then be totally removed. No delays, no restarting of services, everything would effectively work just as if the logfiles were never lost.
Mmm yeah good suggestions, either of those options could work. However it could get tricky trying to upgrade existing emonPi’s. Before we get carried away it would be good to establish if this is indeed the cause of the issue. I’m struggling to recreate this issue.