Problem with full update emoncms

Emoncms
,
1

Hello,
I’m having problems updating my system which may be caused by something in MQTT. Not being a programmer, I’m not sure where to look. The system seems to be working but won’t update.
I don’t see how to attach a file so will include here several outputs: Server Information, emoncms.log, and apache2-error.log. I’ll be happy to send any additional information needed.
Thanks in advance.
Cheers,
Bob

Server Information
-----------------------

Services
	emonhub :	 Active Running                      
	emoncms_mqtt :	 Active Running                      
	feedwriter :	 Active Running - sleep 300s 226 feed points pending write
	service-runner :	 Active Running                      
	redis-server :	 Active Running                      
	mosquitto :	 Active Running                      
	demandshaper :	 Active Running                      
	emoncms_sync :	 Active Running                      
	emonPiLCD :	 Failed loaded failed failed                      
Emoncms
	Version :	 low-write 11.6.7
	Git :	 
		URL :	 origin
		Branch :	 - Describe :	 - Components :	 Emoncms Core v11.6.7 | App v3.1.0 | EmonHub Config v2.1.5 | Dashboard v2.4.0 | Device v2.2.7 | Graph v2.2.4 | Network Setup v1.0.5 | WiFi v2.1.1 | Backup v2.3.3 | DemandShaper v2.2.3 | Postprocess v2.4.7 | Sync v3.1.7 | Usefulscripts v2.3.11 | EmonScripts v1.8.7 | RFM2Pi v1.4.2 | Avrdude-rpi v1.0.3 | Emonhub v2.6.15 | EmonPi v3.0.3

Server
	CPU :	 1 Threads(s) | 4 Core(s) | 1 Sockets(s) | Cortex-A72 | 270.00MIPS | 
	OS :	 Linux 5.10.103-v7l+
	Host :	 emonpi | emonpi | (192.168.1.174)
	Date :	 2024-11-13 11:14:09 EST
	Uptime :	 11:14:09 up 6 min,  1 user,  load average: 0.00, 0.12, 0.08

Memory
	RAM :	 Used: 2.84%
		Total :	 7.71 GB
		Used :	 224.36 MB
		Free :	 7.49 GB
	Swap :	 Used: 0.00%
		Total :	 100 MB
		Used :	 0 B
		Free :	 100 MB

Disk
	 :	 - / :	 Used: 66.53%
		Total :	 4.07 GB
		Used :	 2.71 GB
		Free :	 1.17 GB
		Read Load :	 -2 B/s
		Write Load :	 -0 B/s
		Load Time :	 178 days 20 hours 44 mins
	/mnt/usb0 :	 Used: 5.58%
		Total :	 56.35 GB
		Used :	 3.15 GB
		Free :	 50.34 GB
		Read Load :	 n/a
		Write Load :	 n/a
		Load Time :	 n/a
	/boot :	 Used: 19.14%
		Total :	 252.05 MB
		Used :	 48.24 MB
		Free :	 203.81 MB
		Read Load :	 -0.4 B/s
		Write Load :	 -0 B/s
		Load Time :	 178 days 20 hours 44 mins
	/var/opt/emoncms :	 Used: 2.20%
		Total :	 54.13 GB
		Used :	 1.19 GB
		Free :	 50.19 GB
		Read Load :	 -179.3 B/s
		Write Load :	 -0.01 B/s
		Load Time :	 178 days 20 hours 44 mins
	/var/log :	 Used: 9.77%
		Total :	 50 MB
		Used :	 4.89 MB
		Free :	 45.11 MB
		Read Load :	 n/a
		Write Load :	 n/a
		Load Time :	 n/a

HTTP
	Server :	 Apache/2.4.59 (Raspbian) HTTP/1.1 CGI/1.1 80

MySQL
	Version :	 5.5.5-10.3.39-MariaDB-0+deb10u2
	Host :	 127.0.0.1 (127.0.0.1)
	Date :	 2024-11-13 11:14:09 (UTC -05:00‌​)
	Stats :	 Uptime: 409  Threads: 11  Questions: 716  Slow queries: 0  Opens: 51  Flush tables: 1  Open tables: 44  Queries per second avg: 1.750

Redis
	Version :	 
		Redis Server :	 5.0.14
		PHP Redis :	 5.3.4
	Host :	 localhost:6379
	Size :	 870 keys (913.72K)
	Uptime :	 0 days

MQTT Server
	Version :	 Mosquitto 1.5.7
	Host :	 localhost:1883 (127.0.0.1)

PHP
	Version :	 7.3.31-1~deb10u7 (Zend Version 3.3.31)
	Run user :	 User: www-data Group: www-data video systemd-journal Script Owner: pi
	Modules :	 apache2handlercalendar Core ctype curl date dom v20031129exif fileinfo filter ftp gd gettext hash iconv json v1.7.0libxml mbstring mosquitto v0.4.0mysqli mysqlnd vmysqlnd 5.0.12-dev - 20150407 - $Id: 7cc7cc96e675f6d72e5cf0f267f48e167c2abb23 $openssl pcre PDO pdo_mysql Phar posix readline redis v5.3.4Reflection session shmop SimpleXML sockets sodium SPL standard sysvmsg sysvsem sysvshm tokenizer wddx xml xmlreader xmlwriter xsl Zend OPcache zlib 
Pi
	Model :	 Raspberry Pi  Model N/A Rev  -  ()
	Serial num. :	 10000000E91C8CAC
	CPU Temperature :	 54.53°C
	GPU Temperature :	 54.5°C
	emonpiRelease :	 emonSD-21Jul21
	File-system :	 read-write

Client Information
-----------------------

HTTP
	Browser :	 Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:132.0) Gecko/20100101 Firefox/132.0
	Language :	 en-US,en;q=0.5

Window
	Size :	 1657 x 936

Screen
	Resolution :	 1920 x 1080


==========================================
emoncms.log

2024-11-13 16:09:23.432|ERROR|input_controller.php|{"success": false, "message": "Request contains no data via csv, json or data tag"} for User: 1
2024-11-13 16:09:28.438|ERROR|input_controller.php|{"success": false, "message": "Request contains no data via csv, json or data tag"} for User: 1
2024-11-13 16:09:33.448|ERROR|input_controller.php|{"success": false, "message": "Request contains no data via csv, json or data tag"} for User: 1
============================================
apache2-error.log

 git config --global --add safe.directory /var/www/emoncms
fatal: detected dubious ownership in repository at '/var/www/emoncms'
To add an exception for this directory, call:

        git config --global --add safe.directory /var/www/emoncms
[Wed Nov 13 11:09:06.822760 2024] [php7:notice] [pid 760] [client 192.168.1.13:61005] PHP Notice:  Undefined index: hw in /var/www/emoncms/Modules/admin/Views/admin_main_view.php on line 235, referer: http://192.168.1.174//admin/update
cat: /sys/devices/virtual/dmi/id/board_vendor: No such file or directory
cat: /sys/devices/virtual/dmi/id/product_name: No such file or directory
cat: /sys/devices/virtual/dmi/id/board_name: No such file or directory
cat: /sys/devices/virtual/dmi/id/bios_version: No such file or directory
cat: /sys/devices/virtual/dmi/id/bios_date: No such file or directory
sh: 1: iostat: not found
sh: 1: iostat: not found
sh: 1: iostat: not found
sh: 1: iostat: not found
sh: 1: iostat: not found
sh: 1: iostat: not found
sh: 1: iostat: not found
sh: 1: iostat: not found
sh: 1: iostat: not found
sh: 1: iostat: not found
sh: 1: iostat: not found
sh: 1: iostat: not found
sh: 1: iostat: not found
sh: 1: iostat: not found
fatal: detected dubious ownership in repository at '/var/www/emoncms'
To add an exception for this directory, call:

        git config --global --add safe.directory /var/www/emoncms
fatal: detected dubious ownership in repository at '/var/www/emoncms'
To add an exception for this directory, call:

        git config --global --add safe.directory /var/www/emoncms
[Wed Nov 13 11:09:34.466721 2024] [php7:notice] [pid 758] [client 192.168.1.13:61031] PHP Notice:  Undefined index: hw in /var/www/emoncms/Modules/admin/Views/admin_main_view.php on line 235, referer: http://192.168.1.174//admin/update
2

Drag & drop into the ‘edit’ box where you’re typing, or use the upload icon just right of middle above where you type.

3

Thanks, Robert.

For some unknown reason I seem to be able to update now and was able to make a backup. I saw Brian’s message about updating to 11.6.8 and ran the git checkout master.

I’m still having a problem which seems to be MQTT related. The emoncms.log is still, every few seconds, showing:

2024-11-13 19:54:27.004|ERROR|input_controller.php|{"success": false, "message": "Request contains no data via csv, json or data tag"} for User: 1
2024-11-13 19:54:32.010|ERROR|input_controller.php|{"success": false, "message": "Request contains no data via csv, json or data tag"} for User: 1
2024-11-13 19:54:37.008|ERROR|input_controller.php|{"success": false, "message": "Request contains no data via csv, json or data tag"} for User: 1

I have several sensors sending MQTT data to the emoncms server but don’t know how to trace the source of the error to a particular sensor. As best I can tell, the sensors are working correctly but I must have one that is causing this error. I’m using Node-Red flows to get the data into the server.
Cheers,
Bob

4

I don’t know anything about this, however if the errors occur with any degree of regularity, there’s a simple technique: disable the sensors one by one and when the errors cease, either you’ve probably found it, or you’ve been unlucky.

5

Use an MQTT tool to look at the data on the MQTT Broker to check the data is in the right format.

Changing the debug setting to debug in the settings.ini file will give you more information.

6

Brian,
In settings.ini I have my log level on #3, ERROR. I didn’t see a debug option offered. I added DEBUG to the emonhub.log and have attached a file of the output. Could you have a quick look to see if anything looks odd to you? Thanks.
Bob
emonhub.pdf (178.7 KB)

For some reason it didn’t like my .txt file so I had to copy it to .pdf

7

This means you only get error messages. I can’t remember what the other levels are, but put a 1 in.

This is in the emoncms settings not the emonhub settings and you have to SSH in.

nano /var/www/emoncms/settings.ini

Please click on ‘reply’ to my message else I do not get a notification if you just click the blue reply button.

image
image320×137 1.6 KB

8

Thanks. Here is some of the emoncms.log after setting log level to #1.
emoncms_log.txt (163.1 KB)

Cheers,
Bob

9

Hello @bgrattan this is the emonhub log:

image
image643×311 28.3 KB

10

I would revert the log level for emoncms to avoid your logs eating up disk space.

11

Here is a section of syslog which shows problems with MQTT. I notice that emoncms_mqtt is stopping and trying to auto-start. Thanks. Bob

syslog.txt (4.3 KB)

Nov 14 17:10:11 emonpi systemd[1]: emoncms_mqtt.service: Service RestartSec=1min expired, scheduling restart.
Nov 14 17:10:11 emonpi systemd[1]: emoncms_mqtt.service: Scheduled restart job, restart counter is at 195.
Nov 14 17:10:11 emonpi systemd[1]: Stopped Emoncms emoncms_mqtt script.
Nov 14 17:10:11 emonpi systemd[1]: Started Emoncms emoncms_mqtt script.
Nov 14 17:11:10 emonpi emoncms_mqtt[16214]: PHP Fatal error:  Uncaught Mosquitto\Exception: Invalid function arguments provided. in /var/www/emoncms/scripts/services/emoncms_mqtt/emoncms_mqtt.php:188
Nov 14 17:11:10 emonpi emoncms_mqtt[16214]: Stack trace:
Nov 14 17:11:10 emonpi emoncms_mqtt[16214]: #0 /var/www/emoncms/scripts/services/emoncms_mqtt/emoncms_mqtt.php(188): Mosquitto\Client->publish('', '70.7')
Nov 14 17:11:10 emonpi emoncms_mqtt[16214]: #1 {main}
Nov 14 17:11:10 emonpi emoncms_mqtt[16214]:   thrown in /var/www/emoncms/scripts/services/emoncms_mqtt/emoncms_mqtt.php on line 188
Nov 14 17:11:10 emonpi emoncms_mqtt[16214]: Fatal error: Uncaught Mosquitto\Exception: Invalid function arguments provided. in /var/www/emoncms/scripts/services/emoncms_mqtt/emoncms_mqtt.php:188
Nov 14 17:11:10 emonpi emoncms_mqtt[16214]: Stack trace:
Nov 14 17:11:10 emonpi emoncms_mqtt[16214]: #0 /var/www/emoncms/scripts/services/emoncms_mqtt/emoncms_mqtt.php(188): Mosquitto\Client->publish('', '70.7')
Nov 14 17:11:10 emonpi emoncms_mqtt[16214]: #1 {main}
Nov 14 17:11:10 emonpi emoncms_mqtt[16214]:   thrown in /var/www/emoncms/scripts/services/emoncms_mqtt/emoncms_mqtt.php on line 188
Nov 14 17:11:10 emonpi systemd[1]: emoncms_mqtt.service: Main process exited, code=exited, status=255/EXCEPTION
Nov 14 17:11:10 emonpi systemd[1]: emoncms_mqtt.service: Failed with result 'exit-code'.
Nov 14 17:12:10 emonpi systemd[1]: emoncms_mqtt.service: Service RestartSec=1min expired, scheduling restart.
Nov 14 17:12:10 emonpi systemd[1]: emoncms_mqtt.service: Scheduled restart job, restart counter is at 196.
Nov 14 17:12:10 emonpi systemd[1]: Stopped Emoncms emoncms_mqtt script.
Nov 14 17:12:10 emonpi systemd[1]: Started Emoncms emoncms_mqtt script.
Nov 14 17:13:09 emonpi emoncms_mqtt[17113]: PHP Fatal error:  Uncaught Mosquitto\Exception: Invalid function arguments provided. in /var/www/emoncms/scripts/services/emoncms_mqtt/emoncms_mqtt.php:188
Nov 14 17:13:09 emonpi emoncms_mqtt[17113]: Stack trace:
Nov 14 17:13:09 emonpi emoncms_mqtt[17113]: #0 /var/www/emoncms/scripts/services/emoncms_mqtt/emoncms_mqtt.php(188): Mosquitto\Client->publish('', '70.7')
Nov 14 17:13:09 emonpi emoncms_mqtt[17113]: #1 {main}
Nov 14 17:13:09 emonpi emoncms_mqtt[17113]:   thrown in /var/www/emoncms/scripts/services/emoncms_mqtt/emoncms_mqtt.php on line 188
Nov 14 17:13:09 emonpi emoncms_mqtt[17113]: Fatal error: Uncaught Mosquitto\Exception: Invalid function arguments provided. in /var/www/emoncms/scripts/services/emoncms_mqtt/emoncms_mqtt.php:188
Nov 14 17:13:09 emonpi emoncms_mqtt[17113]: Stack trace:
Nov 14 17:13:09 emonpi emoncms_mqtt[17113]: #0 /var/www/emoncms/scripts/services/emoncms_mqtt/emoncms_mqtt.php(188): Mosquitto\Client->publish('', '70.7')
Nov 14 17:13:09 emonpi emoncms_mqtt[17113]: #1 {main}
Nov 14 17:13:09 emonpi emoncms_mqtt[17113]:   thrown in /var/www/emoncms/scripts/services/emoncms_mqtt/emoncms_mqtt.php on line 188
Nov 14 17:13:09 emonpi systemd[1]: emoncms_mqtt.service: Main process exited, code=exited, status=255/EXCEPTION
Nov 14 17:13:09 emonpi systemd[1]: emoncms_mqtt.service: Failed with result 'exit-code'.
Nov 14 17:14:09 emonpi systemd[1]: emoncms_mqtt.service: Service RestartSec=1min expired, scheduling restart.
Nov 14 17:14:09 emonpi systemd[1]: emoncms_mqtt.service: Scheduled restart job, restart counter is at 197.
Nov 14 17:14:09 emonpi systemd[1]: Stopped Emoncms emoncms_mqtt script.
Nov 14 17:14:09 emonpi systemd[1]: Started Emoncms emoncms_mqtt script.
12

Logrotate should handle that. I have my logs set to 1 permanently! If it doesn’t, there is something wrong with the logrotate setup that should be fixed!

13

It looks like some data is being published to the MQTT Broker that is incorrect.

Do you have anything other than emonhub publishing data to the MQTT Broker?

I did previously suggest;

14

Hi Brian,
I think the culprit may be my Dragino LHT52 temp/hum sensor but I not sure. I’m attaching a couple of files which may tell you something. The only odd thing my untrained eye sees is the NR debug data shows “undefined” for the temp and hum data; however it sends it to the emoncms and it appears in the inputs.

NodeRed_flow_info.txt (408 Bytes)
LHT52_debug.txt (2.0 KB)

The syslog is still showing an error what shuts down emoncms_mqtt which then restarts itself after a short period.

Sorry to be wasting your time on this. Things seem to be working as far as sending data to the emonpi–it’s just the constant error messages in the log that bother me. Robert’s suggestion of shutting down sensors to locate the problem is probably the best at this point but I have been avoiding it because I have things in so many different locations.

Much appreciated for your (and others) help. Maybe we should just let it go for now.
Cheers,
Bob

15

Ok, one last shot at this. I’ve all but pulled the house mains fuse to stop the error messages. The MQTT explorer wasn’t showing anything that I could find.
I’m just about 100% sure that the culprit was my old emonth2 which I tossed aside because I was unable to get it working. It was still plugged in to a test power source!
I killed it for sure and the error messages in syslog stopped. However, in the emoncms.log I’m still getting the following every 5 seconds:

2024-11-15 20:35:42.044|ERROR|input_controller.php|{"success": false, "message": "Request contains no data via csv, json or data tag"} for User: 1
2024-11-15 20:35:47.052|ERROR|input_controller.php|{"success": false, "message": "Request contains no data via csv, json or data tag"} for User: 1
2024-11-15 20:35:52.060|ERROR|input_controller.php|{"success": false, "message": "Request contains no data via csv, json or data tag"} for User: 1
2024-11-15 20:35:57.051|ERROR|input_controller.php|{"success": false, "message": "Request contains no data via csv, json or data tag"} for User: 1

I’ve removed the emonth2 (it always listed as emonth5) from the inputs and feeds but the errors are still coming. Rebooting didn’t seem to help anything. Is there something else I need to get rid of to stop the errors? Thanks and sory to keep going on this.
Cheers,
Bob

16

Looking at the code for the Feb 24 image, and in the absence of a line no. in the error message, it appears that the error is coming from either a ‘post’ input or a ‘bulk’ input, which is sending messages every 5 s to emonCMS.

Does this help?

Could it be something as silly as an unwanted space character or a newline, or a quote?

17

But, I’ll say it again, Get an MQTT tool to examine the data on the MQTT Broker. You might need to check the data is valid in the function (google is your friend) or in a switch node before sending it to the MQTT broker.

18

Brian,
Thanks for your patience. I have been using MQTT Explorer to search for the origin of this problem. Also using debug in the Node-Red flows. I have found something which looks suspicious and maybe you can comment.

The explorer shows the various MQTT sensors sending to the emonpi. There is an instance of emonhub showing:

emonhub/rx/8
values = 3661.7088,4035.7888,287,57,120.54159360000001,17.8,19,21,20.200000000000003,300,300,10550,-27

which sends data every 5 seconds and corresponds to the flow of errors in emoncms.log every five seconds. I am attaching a screen shot from MQTT Explorer.

This seems odd as the data already is sent under emontx3. This may show some sort of misconfiguration (emonhub.conf) on my part.
Any ideas here?
Cheers,
Bob
Doc1.pdf (134.9 KB)

19

May have found something here in the emonhub.log (section attached). The correct data is being published as shown in the various lines:

2024-11-17 14:51:54,955 DEBUG    MQTT       Publishing: emon/emontx3/power3 291
2024-11-17 14:51:54,957 DEBUG    MQTT       Publishing: emon/emontx3/power4 57
2024-11-17 14:51:54,959 DEBUG    MQTT       Publishing: emon/emontx3/vrms 122.21578240000001

After the data is published every 5 seconds, I get a line:

2024-11-17 14:51:45,114 DEBUG    MQTT       Publishing: emonhub/rx/8/values 218.0352,384.768,290,58,122.54621440000001,18.1,19.6,19.3,20.700000000000003,300,300,10560,-28

emoncms_log.txt (163.1 KB)

Not sure where “emonhub/rx/8/values” is coming from but it looks as if it is causing the error:

|ERROR|input_controller.php|{"success": false, "message": "Request contains no data via csv, json or data tag"} for User: 1

Since the data is already published under “emon/emontx3/” then it shouldn’t be repeated in a string of values.
Maybe I’m wrong but it seems as if this is causing the problem.
Cheers,
Bob

20

It isn’t an error - emonhub is doing what yu are telling it to do!

Disable node_format_enable (set to 0). If this isn’t there then add it to the interface in emonhub.

It doesn’t make any difference though as it isn’t read by emoncms,

        # emonhub/rx/10/values format
        # Use with emoncms Nodes module
        node_format_enable = 0
        node_format_basetopic = emonhub/