1

Hi all,

I have been running the later version for the emonpi low-write 9.8.25, some of the inputs stopped working today

2017-12-04 09:08:24.215|WARN|phpmqtt_input.php|Starting MQTT Input script
2017-12-04 09:08:25.480|WARN|phpmqtt_input.php|Subscribing to: emon/#
2017-12-04 09:08:25.481|WARN|phpmqtt_input.php|Not connected, retrying connection
2017-12-04 09:08:25.482|WARN|phpmqtt_input.php|Connecting to MQTT server: Connection Accepted.: code: 0
2017-12-04 09:08:25.482|WARN|phpmqtt_input.php|Subscribed to topic: emon/#
2017-12-04 09:08:28.510|WARN|phpmqtt_input.php|Starting MQTT Input script
2017-12-04 09:08:29.644|WARN|phpmqtt_input.php|Subscribing to: emon/#
2017-12-04 09:08:29.644|WARN|phpmqtt_input.php|Not connected, retrying connection
2017-12-04 09:08:29.645|WARN|phpmqtt_input.php|Connecting to MQTT server: Connection Accepted.: code: 0
2017-12-04 09:08:29.645|WARN|phpmqtt_input.php|Subscribed to topic: emon/#
2017-12-10 08:51:06.452|ERROR|phpmqtt_input.php|exception 'Mosquitto\Exception' with message 'The client is not currently connected.' in /var/www/emoncms/scripts/phpmqtt_input.php:112
Stack trace:
#0 /var/www/emoncms/scripts/phpmqtt_input.php(112): Mosquitto\Client->loop()
#1 {main}
2017-12-10 08:51:07.257|WARN|phpmqtt_input.php|Not connected, retrying connection
2017-12-12 07:09:49.215|ERROR|phpmqtt_input.php|exception 'Mosquitto\Exception' in /var/www/emoncms/scripts/phpmqtt_input.php:112
Stack trace:
#0 /var/www/emoncms/scripts/phpmqtt_input.php(112): Mosquitto\Client->loop()
#1 {main}
2017-12-12 07:09:51.433|WARN|phpmqtt_input.php|Not connected, retrying connection
2017-12-12 07:19:14.784|ERROR|phpmqtt_input.php|exception 'RedisException' with message 'read error on connection' in /var/www/emoncms/Modules/process/process_processlist.php:657
Stack trace:
#0 /var/www/emoncms/Modules/process/process_processlist.php(657): Redis->exists('process:whaccum...')
#1 [internal function]: Process_ProcessList->wh_accumulator('17', 1513062619, 2487, NULL, Object(Process))
#2 /var/www/emoncms/Modules/process/process_model.php(60): call_user_func_array(Array, Array)
#3 /var/www/emoncms/Modules/process/process_model.php(103): Process->__call('wh_accumulator', Array)
#4 /var/www/emoncms/Modules/process/process_model.php(103): Process->wh_accumulator('17', 1513062619, 2487, NULL)
#5 /var/www/emoncms/scripts/phpmqtt_input.php(270): Process->input(1513062619, '2487', '1:16,2:1,34:17')
#6 [internal function]: message(Object(Mosquitto\Message))
#7 /var/www/emoncms/scripts/phpmqtt_input.php(112): Mosquitto\Client->loop()
#8 {main}

[attached is the log file emoncms(1).txt (2.4 KB)]

I logged in via SSH

sudo systemctl restart mqtt_input
did not make any difference

but
sudo service emonhub restart
kick it back in to life. I did try restarting from the web interface

Paul

[Title was ambiguous - expanded to clarify. Moderator (RW)]

2

Hi Paul, Can you tells us about the inputs that stopped. Would I be right in thinking they were all RFM networked? (eg emonTH emonTx etc)

Can you supply the emonhub.log if it reoccurs?

Thanks for supplying the emoncms.log, whilst I do not think any of those errors are the cause of this issue, I’m sure the info will be useful.

3

Hi Paul

Where do i find the emoncms.log (it would be great if they was a button on the web interface that would zip up all of the log files and allow you to download them)

The interfaces that stopped Emonpi, emonth5, BMWi3, 2 x node-red (modbus) 1x node-red (weather).
Interfaces still working were openevse and IotaWatt

4

You have already shown us the emoncms.log, which is accessible/downloadable via the emoncms=>admin page, the emonhub.log is accessible and downloadable via the emoncms=>emonhub page (on an emonSD).

I assume you are running the latest emonSD on and emonPi? You say emoncms is the latest, have you updated via the “update emonpi” button in emoncms=>admin?

At this point I have no idea how restarting emonHub could impact the “2 x node-red (modbus) 1x node-red (weather).” inputs directly, I assume they are posting direct to emoncms and not through emonHub, are they posting via MQTT or HTTP?

5

I will keep my eye on it and grab the log files next time they is a problem

The node-red is posting via MQTT the same as the inbuilt weather flow

The update was installed via the “update emonpi” on admin, also click the database update&check

6

It has crashed again.

Screenshot and log files attached.

1
11913×501 45.1 KB
emoncms(2).txt (7.5 KB)
emonhub.txt (773.8 KB)

sudo systemctl status mqtt_input -n50 -l
● mqtt_input.service - Emoncms MQTT Input Script
Loaded: loaded (/etc/systemd/system/mqtt_input.service; enabled)
Active: active (running) since Tue 2017-12-19 21:31:16 GMT; 4 days ago
Docs: https://github.com/emoncms/emoncms/blob/master/docs/RaspberryPi/MQTT.md
Main PID: 3275 (php)
CGroup: /system.slice/mqtt_input.service
└─3275 /usr/bin/php /var/www/emoncms/scripts/phpmqtt_input.php

Dec 24 01:44:26 emonpi mqtt_input[3275]: PHP Fatal error: Call to a member function fetch_object() on boolean in /var/www/emoncms/Modules/input/input_model.php on line 605
Dec 24 01:44:26 emonpi mqtt_input[3275]: Fatal error: Call to a member function fetch_object() on boolean in /var/www/emoncms/Modules/input/input_model.php on line 605
Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.

I will leave it in a broken state for a few hours if you would like a remote session to have a look, just send me a PM

Paul

7

Try and update to 9.8.27:

This release includes a significant fix to mqtt input script. The issue was the MYSQL connection was timing out after the mqtt input script had been working for a while. This has now been fixed.

8

I have upgraded and will report back in a few days.

9

just been to look and it looks like the MQTT service stopped 2 days ago

10

I am still having a problem with the a service stopping or crashing, i have upgraded to the latest version low-write 9.8.27

If i look under administration it reports “Daemon is not running, start it at ~/scripts/feedwriter”

image
image1847×415 12.1 KB

emoncms log looks like

2018-01-20 19:17:01.688|ERROR|phpmqtt_input.php|exception ‘ErrorException’ with message ‘Invalid argument supplied for foreach()’ in /var/www/emoncms/Modules/input/input_model.php:197
Stack trace:
#0 /var/www/emoncms/Modules/input/input_model.php(197): exceptions_error_handler(2, ‘Invalid argumen…’, ‘/var/www/emoncm…’, 197, Array)
#1 /var/www/emoncms/Modules/input/input_model.php(182): Input->redis_get_inputs(1)
#2 /var/www/emoncms/scripts/phpmqtt_input.php(220): Input->get_inputs(1)
#3 [internal function]: message(Object(Mosquitto\Message))
#4 /var/www/emoncms/scripts/phpmqtt_input.php(116): Mosquitto\Client->loop()
#5 {main}
2018-01-20 19:17:01.690|ERROR|phpmqtt_input.php|exception ‘ErrorException’ with message ‘Invalid argument supplied for foreach()’ in /var/www/emoncms/Modules/input/input_model.php:200
Stack trace:
#0 /var/www/emoncms/Modules/input/input_model.php(200): exceptions_error_handler(2, ‘Invalid argumen…’, ‘/var/www/emoncm…’, 200, Array)
#1 /var/www/emoncms/Modules/input/input_model.php(182): Input->redis_get_inputs(1)
#2 /var/www/emoncms/scripts/phpmqtt_input.php(220): Input->get_inputs(1)
#3 [internal function]: message(Object(Mosquitto\Message))
#4 /var/www/emoncms/scripts/phpmqtt_input.php(116): Mosquitto\Client->loop()
#5 {main}
2018-01-20 19:17:01.702|ERROR|phpmqtt_input.php|exception ‘ErrorException’ with message ‘Invalid argument supplied for foreach()’ in /var/www/emoncms/Modules/input/input_model.php:197
Stack trace:
#0 /var/www/emoncms/Modules/input/input_model.php(197): exceptions_error_handler(2, ‘Invalid argumen…’, ‘/var/www/emoncm…’, 197, Array)
#1 /var/www/emoncms/Modules/input/input_model.php(182): Input->redis_get_inputs(1)
#2 /var/www/emoncms/scripts/phpmqtt_input.php(220): Input->get_inputs(1)
#3 [internal function]: message(Object(Mosquitto\Message))
#4 /var/www/emoncms/scripts/phpmqtt_input.php(116): Mosquitto\Client->loop()
#5 {main}

Anything to try?

11

It looks like the feedwriter scrip has stopped this can be restarted with

sudo service feedwriter restart

This question is why is stopped in the first place. This error is interesting, I’ve not seen this before:

What version emonSD are you running?

What are you running on the emonPi? What inputs are you posting? Anything non standard?

If you update to the latest version V9.8.28 (via emonPi update) you will be able to copy all the info admin info page. Please paste the full output here:

See the copy to clipboard button

25
252515×1164 454 KB

12

Hi Glyn,
I am running 2.8.27
I did try running “sudo service feedwriter restart” and the buffer counted down but the data was still missing.

Input wise i have 2 node-red running modbus every 5 seconds, BMW input, OpenEVSE, EmonPi, Iotawatt, weather.

I have clicked the emonpi update button but it has not download the latest version

Paul

13

That’s strange, emonPi update should update to 9.8.28. Please post your emonpiupdate log

14

Log File Attached emonpiupdate.txt (6.9 KB)

15

Thanks. From the logfile is seems that default.emonpi.settings.php and settings.php have been user modified. The change to default.emonpi.settings.php is causing emonPi update to not be able to pull in any new changes.

If’s possible to keep your custom settings in settings.php and just restore default.emonpi.settings.php to it’s default state. This file is is only used to store the default settings for reference, its not actually used by the application. You can reset default.emonpi.settings.php back to it’s default state by running

rpi-rw
cd  /var/www/emoncms
git checkout default.emonpi.settings.php
git pull

The final git pull command should now pull in the latest changes and update you to 9.8.28. This won’t necessarily fix the issue you have reported but should enable us to debug on a common platform. Once updated to 9.8.28 please use the new copy to clipboard function to copy your Emoncms system info to clipboard and paste into the forum. It should look something like this:

16

Hi Glyn,
Thats fixed it :smiley:

Server Information















Emoncms
Version
low-write 9.8.28 : 2018.01.27





Modules
Administration : App v1.1.0 : Backup v1.1.2 : EmonHub Config v1.0.0 : Dashboard v1.1.1 : EventProcesses : Feed : Graph v1.2.0 : Input : postprocess : CoreProcess : Schedule : setup : Time : User : Visualisation : WiFi v1.0.0





Buffer
loading…





Writer
Daemon is running with sleep 60s




Server
OS
Linux 4.9.35-v7+





Host
emonpi emonpi (127.0.1.1)





Date
2018-01-29 08:54:05 UTC





Uptime
08:54:05 up 3 days, 6 min,  1 user,  load average: 0.57, 0.62, 0.44




HTTP
Server
Apache/2.4.10 (Raspbian) HTTP/1.1 CGI/1.1 80




MySQL
Version
5.5.57-0+deb8u1





Host
localhost (127.0.0.1)





Date
2018-01-29 08:54:04 (UTC 00:00‌​)





Stats
Uptime: 259601  Threads: 2  Questions: 320456  Slow queries: 0  Opens: 68  Flush tables: 1  Open tables: 61  Queries per second avg: 1.234




Redis
Version
2.8.17





Host
localhost:6379 (127.0.0.1)





Size
518 keys  (616.68K)





Uptime
3 days




MQTT
Version
1.4.14





Host
localhost:1883 (127.0.0.1)




Pi
CPU Temp
53.69°C





Release
emonSD-07Nov16





File-system
Set root file-system temporarily to read-write, (default read-only)




Memory
RAM
Used: 87.39% Total: 970.93 MB Used: 848.46 MB Free: 122.46 MB




Disk
Mount
Stats





/
Used: 65.37% Total: 3.33 GB Used: 2.18 GB Free: 1015.86 MB





/boot
Used: 36.32% Total: 59.95 MB Used: 21.77 MB Free: 38.17 MB





/home/pi/data
Used: 5.75% Total: 24.98 GB Used: 1.44 GB Free: 22.26 GB




PHP
Version
5.6.30-0+deb8u1 (Zend Version 2.6.0)





Modules
apache2handler : bcmath : bz2 : calendar : Core v5.6.30-0+deb8u1 : ctype : curl : date v5.6.30-0+deb8u1 : dba : dio v0.0.4RC4 : dom v20031129 : ereg : exif v1.4 : fileinfo v1.0.5 : filter v0.11.0 : ftp : gettext : hash v1.0 : iconv : json v1.3.6 : libxml : mbstring : mcrypt : mhash : mosquitto v0.3.0 : mysql v1.0 : mysqli v0.1 : openssl : pcre : PDO v1.0.4dev : pdo_mysql v1.0.2 : Phar v2.0.2 : posix : readline v5.6.30-0+deb8u1 : redis v2.2.7 : Reflection : session : shmop : SimpleXML v0.1 : soap : sockets : SPL v0.2 : standard v5.6.30-0+deb8u1 : sysvmsg : sysvsem : sysvshm : tokenizer v0.1 : wddx : xml : xmlreader v0.1 : xmlwriter v0.1 : Zend OPcache v7.0.6-devFE : zip v1.12.5 : zlib v2.0 :

Client Information















HTTP
Browser
Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0




Screen
Resolution
1920 x 1080




Window
Size
1903 x 948

1 Like
17

It looks like I’m having the same issue. Trystan recommended I update to 9.8.28 as Glyn suggested. So, let’s run this and see what happens.

Paul, Are you having a failure between 2 and 6 days the same as I am?

Also, Can you send me a few excerpts of your raw input?

Thanks
Bernie Carter

18

Hi @Bernie_Carter

Yes that sounds correct. It looks like it’s something to do with the MQTT /nodered service, as anything that is posting to that stops. But other services like openevse and iotawatt till post ok.

I am away at the moment so I am unable to post any log files

Paul

19

@Pukka

That is possible, I will review my logs again and see, but I didn’t notice anything out of line the first run through. Just a reference to a broken pipe.

I’ve had a ton of issues with nodered right from the start. In fact this is my 3 try at keeping it going. The first two ran into all kinds of programming issues, and had to be wiped to clear all.

It’s also possible that your issue is unrelated to mine, they just have similar outcomes.

Bernie Carter

20

It has just crashed again, does anyone want to login and have a look at it? I will leave it broken over night.

Paul