Hi chaps,
I’ve got another instance of the graphs stopped working days ago issue.
This time it’s not browser cache related (first thing I tried) or lack of disk space (plenty free).
The feedwriter service is running fine and all the data is being stored locally on the Pi.
The only thing I’ve done to this Pi recently is I ran an “apt update/upgrade” on it to pull the OS up to date.
Everything seems to be running OK, the main dash is up to date, the inputs are arriving and look like they are getting logged into the feeds, but the charts all stop four days ago.
As a last resort, I’ve rebooted the Pi, but it doesn’t seem to have made any difference.
Any ideas?
I can’t say 100%, but they were both some time that evening.
The full EmonCMS log is -
2019-07-29 07:46:05.066|WARN|phpmqtt_input.php|Not connected, retrying connection
2019-07-29 07:46:05.117|WARN|phpmqtt_input.php|Connecting to MQTT server: Connection Accepted.: code: 0
2019-07-29 07:46:06.877|WARN|phpmqtt_input.php|Not connected, retrying connection
2019-07-29 07:46:06.937|WARN|phpmqtt_input.php|Connecting to MQTT server: Connection Accepted.: code: 0
2019-07-29 07:47:05.502|ERROR|input_controller.php|{“success”: false, “message”: “Format error: csv value is not numeric”} for User: 1
2019-07-29 07:47:05.511|ERROR|input_controller.php|{“success”: false, “message”: “Format error: csv value is not numeric”} for User: 1
Which is all this morning before I found the problem.
Yes, everything looked like it was running OK before I rebooted.
Server Info below (yes, it’s an old version, I’m still playing with the new version on a different Pi)
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36
Screen
Resolution
1600 x 900
Window
Size
1543 x 801
No, it was just an update & upgrade. I’ve had issues with dist-upgrades, hence building a new version.
No, no other updates done. It’s odd, I’ve done this several times to this Pi/EmonCMS & not had issues.
No, none of the inputs were duplicated. I’ve had that before too.
These lines might point to something, can you check emonhub.log?
can you also confirm that the mosquitto broker is running ok, I notice there is no “mqtt version” in your server info.
Ahh! something else to check! Mosquitto used to be installed direct from the mosquitto repo’s (not the debian repos) to get the latest version. This has changed in more recent emonSD images. The most recent Mosquitto versions (1.5.7’ish) have breaking changes. When you used apt-get to upgrade it will have pulled in a “broken” version.
Please check the Mosquitto version, if it’s not 1.4.10’ish then I think you need to rollback by removing the current mosquitto broker and then editing the source.list to remove the mosquitto repo url and then re-install from the Debian repos.
[edit - just seen your latest post]
Ok, maybe your Mosquitto wasn’t updated? Still worth checking I think.
[edit2] - also, unrelated but, your sleep setting for the feedwriter is 10 mins (600s), emonpi’s are normally set to 60s. 0mins of data is a lot to be buffering, this can delay data being displayed in graphs etc and/or if there’s a “hiccup” it’s better to lose up to 1 min of data rather than 10mins. This can be set in emoncms/settings.php
That’s a good idea, unfortunately they’re been log rotated and the mornings logs are gone.
There’s no repeat of the error in the three logs that currently exist.
I’ll check again tomorrow at 7:47 to see it it reoccurs.
Yes, I noticed that about Mosquitto too. It was running before the reboot, and it’s still running now and has been since the reboot.
pi@emonpi:/var/log/emonhub $ sudo service mosquitto status
● mosquitto.service - LSB: mosquitto MQTT v3.1 message broker
Loaded: loaded (/etc/init.d/mosquitto)
Active: active (running) since Mon 2019-07-29 08:45:47 BST; 13h ago
Process: 1569 ExecStop=/etc/init.d/mosquitto stop (code=exited, status=0/SUCCESS)
Process: 1585 ExecStart=/etc/init.d/mosquitto start (code=exited, status=0/SUCCESS)
CGroup: /system.slice/mosquitto.service
└─1592 /usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf
pi@emonpi:~ $ dpkg -s mosquitto
Package: mosquitto
Status: install ok installed
Priority: optional
Section: net
Installed-Size: 419
Maintainer: Roger A. Light [email protected]
Architecture: armhf
Multi-Arch: foreign
Version: 1.6.3-0mosquitto1~jessie1
Depends: adduser (>= 3.10), libuuid1, lsb-base (>= 4.1+Debian3), libc6 (>= 2.4), libssl1.0.0 (>= 1.0.0), libwebsockets3 (>= 1.2), libwrap0 (>= 7.6-4~)
Suggests: apparmor
Conffiles:
/etc/init.d/mosquitto 7c1c057c3c625deaac042ba97032ec69
/etc/logrotate.d/mosquitto d02c7d232915df80a983e120d96b3b08
/etc/mosquitto/ca_certificates/README c1c6ae67f2def06c6a483be09b49d4de
/etc/mosquitto/certs/README 4d8a70d4cefab07d4dabc5be1f786c1f
/etc/mosquitto/conf.d/README b4ac621550824082a735732bfb42b51d
/etc/mosquitto/mosquitto.conf 379a6d6e30d19bfdedf8099a9f3f8770
/etc/init/mosquitto.conf 97d9bc16841e4ba66a68ef1ee280fd9f obsolete
Description: MQTT version 3.1/3.1.1 compatible message broker
This is a message broker that supports version 3.1 and 3.1.1 of the MQTT
protocol.
.
MQTT provides a method of carrying out messaging using a publish/subscribe
model. It is lightweight, both in terms of bandwidth usage and ease of
implementation. This makes it particularly useful at the edge of the network
where a sensor or other simple device may be implemented using an arduino for
example.
Homepage: http://mosquitto.org/
So I guess aim for 1.4.15 ?
Do you know what are the problems were with later releases? As I’m pretty sure I upgraded at the beginning of this years sometime, so I’m guess I’d have been running a newer version of Mosquitto for a while?
I don’t know the specific reasons behind the issues, I’m just aware there have been issues. I just looked up a couple of threads and realised my memory wasn’t 100% regards the version numbers. 1.5.7 is actually the latest stable version from the Debian Buster repos and is ok (with Buster at least), Stretch is 1.4.10 and Jessie is still trailing behind at 1.3.4 so whilst you are still running Jessie I guess you will need to run a mosquitto source rather than debian. The problem versions are from 1.6’ish
I can’t answer why your’s is ok with 1.6.2, nor tell you why others have had less luck. I can only say that the known good versions are 1.4.10 and 1.5.7. 1.4.15 rings a bell but I seem to recall it wasn’t for a good reason and a quick search turned up this Mosquitto issues and Stretch (DietPi) - #8 by borpin.
Without doing some research I can’t even tell you why you have such a short shortlist, (versions available) unless there was good reason to do otherwise, I would try to go for 1.4.10 on Jessie as it is tried and tested. I wish I could be of more help but I’m not sure what to suggest.
The list comes from the “apt-cache showpkg mosquitto” command that shows what versions of a package are available in the APT repository. This presents a slight problem in that 1.4.10 and 1.5.7 are not there.
I see them on the list on the Mosquitto website that you pointed to, but those downloads don’t seem to be tailored to the RPi and must be compiled from source. The website jst points us back to the repo.
I think I need to test the compiling on a different Pi, as while I’m messing about MQTT won’t be running and data will be getting lost. Unless anyone else can point me in the right direction?
