I have two instances of emonTx/RPi running emonSD Oct 2018 ver 9.9.8 – Node 12 & Node 13.
In each case, the emonTx is serial direct connected (3 short leads on the GPIO pins of the RPi).
Each has the rootfs & /home/pi/data in partitions on a USB HDD.
Each uses the EmonHubEmoncmsHTTPInterfacer to send data to watchman.
watchman is an RPi on the local network also running emonSD Oct 2018 ver 9.9.8. It has a USB HDD connected but no emonTx.
Nodes 12 & 13 are plagued with crashes. They both crashed yesterday and Node 13 crashed again today. (I fsck the USB HDD’s using Raspbian Lite to recover – which resets the free block count).
I also have Node 11 & Node 14 running – identical in every way (ver 9.9.8 and USB HDD) except they do not use the EmonHubEmoncmsHTTPInterfacer as they are not sending data to watchman.
Nodes 11 & 14 and watchman have been crash free for weeks/months.
EmonHubMqttInterfacer …
This is included in the emonhub.conf file for all 4 nodes and watchman.
Is it needed in my simple installations? – not using Node-Red, etc
Also what MQTT services or solely related services could be stopped?
EmonHubMqttInterfacer is the only interfacer in watchman’s emonhub.conf file. Does it even need that Interfacer?
watchman does not have the file /var/log/emonhub/emonhub.log (the Download Log button does not work) but strangely the web interface page for EmonHub does show the following …
-- Logs begin at Thu 2016-11-03 17:16:45 GMT, end at Sun 2019-03-17 20:07:28 GMT. --
Mar 17 00:51:17 watchman systemd[1]: Started emonHub service description.
Mar 17 00:51:24 watchman emonhub.py[360]: 2019-03-17 00:51:24,483 INFO MainThread EmonHub emonHub emon-pi variant v2.1.2
Mar 17 00:51:24 watchman emonhub.py[360]: 2019-03-17 00:51:24,484 INFO MainThread Opening hub...
Mar 17 00:51:24 watchman emonhub.py[360]: 2019-03-17 00:51:24,485 INFO MainThread Logging level set to DEBUG
Mar 17 00:51:24 watchman emonhub.py[360]: 2019-03-17 00:51:24,485 INFO MainThread Creating EmonHubMqttInterfacer 'MQTT'
Mar 17 00:51:24 watchman emonhub.py[360]: 2019-03-17 00:51:24,488 DEBUG MainThread Setting MQTT subchannels: ['ToEmonCMS']
Mar 17 00:51:24 watchman emonhub.py[360]: 2019-03-17 00:51:24,489 DEBUG MainThread Setting MQTT pubchannels: ['ToRFM12']
Mar 17 00:51:24 watchman emonhub.py[360]: 2019-03-17 00:51:24,489 INFO MainThread Setting MQTT nodevar_format_enable: 1
Mar 17 00:51:24 watchman emonhub.py[360]: 2019-03-17 00:51:24,490 INFO MainThread Setting MQTT node_format_enable: 1
Mar 17 00:51:24 watchman emonhub.py[360]: 2019-03-17 00:51:24,490 INFO MainThread Setting MQTT nodevar_format_basetopic: emon/
Mar 17 00:51:40 watchman systemd[1]: Stopping emonHub service description...
Mar 17 00:51:40 watchman systemd[1]: Stopped emonHub service description.
Mar 17 00:51:40 watchman systemd[1]: Started emonHub service description.
Mar 17 00:51:42 watchman emonhub.py[1104]: 2019-03-17 00:51:42,919 INFO MainThread EmonHub emonHub emon-pi variant v2.1.2
Mar 17 00:51:42 watchman emonhub.py[1104]: 2019-03-17 00:51:42,920 INFO MainThread Opening hub...
Mar 17 00:51:42 watchman emonhub.py[1104]: 2019-03-17 00:51:42,921 INFO MainThread Logging level set to DEBUG
Mar 17 00:51:42 watchman emonhub.py[1104]: 2019-03-17 00:51:42,921 INFO MainThread Creating EmonHubMqttInterfacer 'MQTT'
Mar 17 00:51:42 watchman emonhub.py[1104]: 2019-03-17 00:51:42,924 DEBUG MainThread Setting MQTT subchannels: ['ToEmonCMS']
Mar 17 00:51:42 watchman emonhub.py[1104]: 2019-03-17 00:51:42,925 DEBUG MainThread Setting MQTT pubchannels: ['ToRFM12']
Mar 17 00:51:42 watchman emonhub.py[1104]: 2019-03-17 00:51:42,925 INFO MainThread Setting MQTT nodevar_format_enable: 1
Mar 17 00:51:42 watchman emonhub.py[1104]: 2019-03-17 00:51:42,925 INFO MainThread Setting MQTT node_format_enable: 1
Mar 17 00:51:42 watchman emonhub.py[1104]: 2019-03-17 00:51:42,926 INFO MainThread Setting MQTT nodevar_format_basetopic: emon/
And that is all - it does not move forward with time passing which is expected.
EmonHubEmoncmsHTTPInterfacer …
This contains the line … sendinterval = 10
I think I got that from the Guide. However the emonhub.log (on the web pages) for nodes 12 & 13 show that data is only sent to watchman every 30 secs and watchman acknowledges receipt.
Whist, on the other hand, I’ve observed that watchman only updates its phptimeseries every 60 secs. Is this expected behaviour?
As it may help understanding, here is the syslog for a typical cycle of Node 13 sending to watchman …
13 syslog cycle.zip (2.0 KB)
And here are the log lines just containing fatal, error or warn from the watchman daemon.log starting from the last time I powered watchman up …
Mar 17 00:51:31 watchman supervisord[548]: 2019-03-17 00:51:31,117 WARN For [program:tuxtunnel], redirect_stderr=true but stderr_logfile has also been set to a filename, the filename has been ignored
Mar 17 00:51:40 watchman mosquitto[1082]: Stopping network daemon:: mosquittostart-stop-daemon: warning: failed to kill 425: No such process
Mar 17 00:51:41 watchman emonPiLCD[1109]: Stopping system emonPiLCD daemon:start-stop-daemon: warning: failed to kill 421: No such process
Mar 17 00:51:45 watchman supervisord[1204]: 2019-03-17 00:51:45,931 WARN For [program:tuxtunnel], redirect_stderr=true but stderr_logfile has also been set to a filename, the filename has been ignored
Mar 17 00:51:19 watchman raspi-config[353]: Checking if shift key is held down:Error opening '/dev/input/event*': No such file or directory
Mar 17 00:51:28 watchman redis-server[564]: *** FATAL CONFIG FILE ERROR ***
Mar 17 00:51:35 watchman emoncms_mqtt[880]: Can't connect to database, please verify credentials/configuration in settings.php<br />Error message: <b>Connection refused</b>
Mar 17 00:51:28 watchman redis-server[564]: *** FATAL CONFIG FILE ERROR ***
Comments or suggestions?
PS - Until a few days ago I had /var/log mounted on the USB HDD /home/pi/data partition. There were several crashes that self re-booted and from the persistent logs I found that there were many NUL’s across the log line at crash time. On 3 occasions immediately before the crash, syslog records MQTT publishing. On the other occasion, it was Serial Tx being sent. Might this be relevant?