Emoncms_mqtt.php very high ram use

Nothing of any substance apart from that redis check. How to debug and get to the bottom of this? emoncms_mqtt seems to be running just fine here so it’s difficult for me to dig into it.

Im running Mosquitto 2.0.11 & PHP Module: mosquitto v0.4.0 which seems to be the same as you @Darren. @ullmf @lucwillems what versions are you running?

What sort of MQTT traffic do you all have?

Is redis enabled and running ok? looks like it is for you @Darren.

What do the emoncms inputs look like? it might be worth clearing inputs and devices that are unused…

emonpi.local/device/clean.json
emonpi.local/input/clean.json

@TrystanLea can you try a fresh build? Could easily be an underlying package that has changed.

I’m also having this issue and wanted to see if there has been any movement on this.

Going on your questions. My mosquitto is the same as yours. My inputs are only two OpenEVSE MQTT and two IotaWatts. Yet, I am also experiencing the memory issue with emoncms_mqtt.php.

I have not been able to find a solution to the problem so far.
I have tested different Ubuntu versions and also different PHP versions without success.
Furthermore, a lot of data was lost, there were always gaps in the graphics.
After many unsuccessful attempts, I reinstalled the system on an old PI3 and it runs without problems.
This is exactly what I didn’t want, but I found no other solution.

I think my solution now is to setup InfluxDB and Grafana. It looks like I can do the same things using MQTT with that as I could with EmonCMS. Namely ensuring my two OpenEVSE can follow solar output for charging.

I would happily work with folks to find a solution here so others could go this route with EmonCMS on a server setup, but that requires some back and forth.

1 Like

The issue is it isn’t repeatable. I run emoncms on Proxmox quite happily with a huge number of MQTT feeds.

I think you have been using 64bit. Try 32bit 22.04 and see if that works.

image

From what I can see, Ubuntu 22.04 doesn’t come in 32 bit. I thought they stopped offering it in 32 bit a couple of years ago.

I guess I should ask. But did you install your LXC using the scripts? I didn’t notice it ask me for 64 or 32 bit during the MQTT install. Looks like that is still 64 bit in the end. If this is an issue with 64 bit I have a feeling it might begin to creep into the Pi world soon enough as I believe the Pi 4 is 64 bit now.

I can recreate/experience the same problem the emoncms_mqtt.php high ram usage problem by installing EmonCMS through the scripts at

and

Using the following Raspberry Pi OS Lite version on a Raspberry Pi 3 Model B Rev 1.2

Raspberry Pi OS Lite

  • Release date: May 3rd 2023
  • System: 32-bit
  • Kernel version: 6.1
  • Debian version: 11 (bullseye)

I can also recreate it by instead using Raspberry Pi OS Lite 2021-03-25 which is on Debian version: 10 (buster) and Linux 5. Note that for this version of Raspberry Pi OS Lite the packages were first upgraded, but still Debian 10 (buster) and Linux 5.

Problem description
After 20 minutes the Raspberry Pi runs out of memory and freezes.

I installed my previous version of EmonCMS using EmonSD which works perfectly with MQTT.
Old install
On the same Raspberry Pi with the standard EmonSD (versions below)

emonpiRelease: emonSD-08May21,
EmonCMS: low-write 11.3.0, 
OS: Linux 5.10.63-v7+Components: Emoncms Core v11.3.0 | App v2.6.10 | EmonHub Config v2.1.5 | Dashboard v2.3.3 | Device v2.2.2 | Graph v2.2.3 | Network Setup v1.0.2 | WiFi v2.1.1 | Backup v2.3.2 | DemandShaper v2.2.2 | Postprocess v2.2.7 | Sync v2.1.4 | Usefulscripts v2.3.10 | EmonScripts v1.6.7 | RFM2Pi v1.4.1 | Emonhub v2.3.3 | EmonPi v2.9.5.

I am happy to share more information

1 Like

@TrystanLea - one for you :slight_smile:

Logs, please (emoncms set to DEBUG and php/apache logs perhaps). Also how many topics are there on the emon base topic?

Thank you for testing that with a Pi. I didn’t have one handy (still in storage due to moving and renovating the new place) to try that out with. Hoping this gets resolved.

I cannot see/find any particular interesting things in the logs.

In the settings.ini of emoncms, I set the log level to INFO.

The lowest log level that I can configure in settings.ini is INFO.

[log]
; Log Level: 1=INFO, 2=WARN, 3=ERROR
level = 1

I am not sure how to configure a lower loglevel here. I tried level = 0, but that does not work.

In my case, there are 20 topics updating every seconds and there are 3 topics updating every minute. The 20 topics every second are fed with data from a Python script running on the same Raspberry Pi.

From tail /var/log/emoncms/emoncms.log

2023-08-08 19:07:37.-125|INFO|emoncms_mqtt.php|emon/SolarEdge/Vermogen P1 -0.0
2023-08-08 19:07:37.-111|INFO|emoncms_mqtt.php|emon/SolarEdge/Vermogen P2 -0.0
2023-08-08 19:07:37.-98|INFO|emoncms_mqtt.php|emon/SolarEdge/Vermogen P3 -0.0
2023-08-08 19:07:37.-84|INFO|emoncms_mqtt.php|emon/SolarEdge/current 0.0
2023-08-08 19:07:37.-71|INFO|emoncms_mqtt.php|emon/SolarEdge/l1_current 0.0
2023-08-08 19:07:37.-57|INFO|emoncms_mqtt.php|emon/SolarEdge/l2_current 0.0
2023-08-08 19:07:37.-44|INFO|emoncms_mqtt.php|emon/SolarEdge/l3_current 0.0
2023-08-08 19:07:37.-21|INFO|emoncms_mqtt.php|emon/SolarEdge/l1n_voltage 232.3
2023-08-08 19:07:37.-08|INFO|emoncms_mqtt.php|emon/SolarEdge/l2n_voltage 232.20000000000002
2023-08-08 19:07:38.005|INFO|emoncms_mqtt.php|emon/SolarEdge/l3n_voltage 232.3
2023-08-08 19:07:38.019|INFO|emoncms_mqtt.php|emon/SolarEdge/frequency 49.97
2023-08-08 19:07:38.032|INFO|emoncms_mqtt.php|emon/SolarEdge/power_apparent 0.0
2023-08-08 19:07:38.045|INFO|emoncms_mqtt.php|emon/SolarEdge/power_reactive 0.0
2023-08-08 19:07:38.059|INFO|emoncms_mqtt.php|emon/SolarEdge/power_factor 0.0
2023-08-08 19:07:38.072|INFO|emoncms_mqtt.php|emon/SolarEdge/current_dc 0.0
2023-08-08 19:07:38.085|INFO|emoncms_mqtt.php|emon/SolarEdge/voltage_dc 23.200000000000003
2023-08-08 19:07:38.098|INFO|emoncms_mqtt.php|emon/SolarEdge/power_dc 0.0
2023-08-08 19:07:38.112|INFO|emoncms_mqtt.php|emon/SolarEdge/temperature 36.32
2023-08-08 19:07:38.-222|INFO|emoncms_mqtt.php|emon/SolarEdge/Vermogen 0.0
2023-08-08 19:07:38.-208|INFO|emoncms_mqtt.php|emon/SolarEdge/Energie 50363.852
2023-08-08 19:07:38.-194|INFO|emoncms_mqtt.php|emon/SolarEdge/Vermogen P1 -0.0
2023-08-08 19:07:38.-180|INFO|emoncms_mqtt.php|emon/SolarEdge/Vermogen P2 -0.0
2023-08-08 19:07:38.-166|INFO|emoncms_mqtt.php|emon/SolarEdge/Vermogen P3 -0.0
2023-08-08 19:07:38.-153|INFO|emoncms_mqtt.php|emon/SolarEdge/current 0.0
2023-08-08 19:07:38.-139|INFO|emoncms_mqtt.php|emon/SolarEdge/l1_current 0.0
2023-08-08 19:07:38.-126|INFO|emoncms_mqtt.php|emon/SolarEdge/l2_current 0.0
2023-08-08 19:07:38.-113|INFO|emoncms_mqtt.php|emon/SolarEdge/l3_current 0.0
2023-08-08 19:07:38.-100|INFO|emoncms_mqtt.php|emon/SolarEdge/l1n_voltage 232.60000000000002
2023-08-08 19:07:38.-86|INFO|emoncms_mqtt.php|emon/SolarEdge/l2n_voltage 231.9
2023-08-08 19:07:38.-73|INFO|emoncms_mqtt.php|emon/SolarEdge/l3n_voltage 232.60000000000002
2023-08-08 19:07:38.-60|INFO|emoncms_mqtt.php|emon/SolarEdge/frequency 49.97
2023-08-08 19:07:38.-47|INFO|emoncms_mqtt.php|emon/SolarEdge/power_apparent 0.0
2023-08-08 19:07:38.-33|INFO|emoncms_mqtt.php|emon/SolarEdge/power_reactive 0.0
2023-08-08 19:07:38.-20|INFO|emoncms_mqtt.php|emon/SolarEdge/power_factor 0.0
2023-08-08 19:07:38.-07|INFO|emoncms_mqtt.php|emon/SolarEdge/current_dc 0.0
2023-08-08 19:07:39.005|INFO|emoncms_mqtt.php|emon/SolarEdge/voltage_dc 22.400000000000002
2023-08-08 19:07:39.019|INFO|emoncms_mqtt.php|emon/SolarEdge/power_dc 0.0
2023-08-08 19:07:39.032|INFO|emoncms_mqtt.php|emon/SolarEdge/temperature 36.29
2023-08-08 19:07:39.-220|INFO|emoncms_mqtt.php|emon/SolarEdge/Vermogen 0.0
2023-08-08 19:07:39.-207|INFO|emoncms_mqtt.php|emon/SolarEdge/Energie 50363.852
2023-08-08 19:07:39.-195|INFO|emoncms_mqtt.php|emon/SolarEdge/Vermogen P1 -0.0
2023-08-08 19:07:39.-183|INFO|emoncms_mqtt.php|emon/SolarEdge/Vermogen P2 -0.0
2023-08-08 19:07:39.-171|INFO|emoncms_mqtt.php|emon/SolarEdge/Vermogen P3 -0.0
2023-08-08 19:07:39.-159|INFO|emoncms_mqtt.php|emon/SolarEdge/current 0.0
2023-08-08 19:07:39.-147|INFO|emoncms_mqtt.php|emon/SolarEdge/l1_current 0.0
2023-08-08 19:07:39.-135|INFO|emoncms_mqtt.php|emon/SolarEdge/l2_current 0.0
2023-08-08 19:07:39.-122|INFO|emoncms_mqtt.php|emon/SolarEdge/l3_current 0.0
2023-08-08 19:07:39.-110|INFO|emoncms_mqtt.php|emon/SolarEdge/l1n_voltage 232.5
2023-08-08 19:07:39.-98|INFO|emoncms_mqtt.php|emon/SolarEdge/l2n_voltage 231.8
2023-08-08 19:07:39.-86|INFO|emoncms_mqtt.php|emon/SolarEdge/l3n_voltage 232.20000000000002
2023-08-08 19:07:39.-74|INFO|emoncms_mqtt.php|emon/SolarEdge/frequency 49.97
2023-08-08 19:07:39.-62|INFO|emoncms_mqtt.php|emon/SolarEdge/power_apparent 0.0
2023-08-08 19:07:39.-50|INFO|emoncms_mqtt.php|emon/SolarEdge/power_reactive 0.0
2023-08-08 19:07:39.-38|INFO|emoncms_mqtt.php|emon/SolarEdge/power_factor 0.0
2023-08-08 19:07:39.-26|INFO|emoncms_mqtt.php|emon/SolarEdge/current_dc 0.0
2023-08-08 19:07:39.-14|INFO|emoncms_mqtt.php|emon/SolarEdge/voltage_dc 21.400000000000002
2023-08-08 19:07:39.-02|INFO|emoncms_mqtt.php|emon/SolarEdge/power_dc 0.0
2023-08-08 19:07:40.009|INFO|emoncms_mqtt.php|emon/SolarEdge/temperature 36.29

More details on the installation, note EmonHUB is disabled

Server Information
-----------------------

Services
	emoncms_mqtt :	 Active Running                      
	feedwriter :	 Active Running - sleep 300s 0 feed points pending write
	service-runner :	 Active Running                      
	redis-server :	 Active Running                      
	mosquitto :	 Active Running                      
	demandshaper :	 Active Running                      
	emonPiLCD :	 Failed loaded failed failed                      
	emonhub :	 Inactive loaded inactive dead                      
Emoncms
	Version :	 low-write 11.3.22
	Git :	 
		URL :	 https://github.com/emoncms/emoncms.git
		Branch :	 * stable
		Describe :	 11.3.22
	Components :	 Emoncms Core v11.3.22 | App v2.7.3 | EmonHub Config v2.1.5 | Dashboard v2.3.3 | Device v2.2.2 | Graph v2.2.3 | Network Setup v1.0.2 | WiFi v2.1.1 | Backup v2.3.2 | DemandShaper v2.2.2 | Postprocess v2.2.7 | Sync v2.1.4 | Usefulscripts v2.3.10 | EmonScripts v1.6.20 | RFM2Pi v1.4.1 | Avrdude-rpi v1.0.1 | Emonhub v2.5.8 | EmonPi v2.9.5

Server
	CPU :	 1 Threads(s) | 4 Core(s) | 1 Sockets(s) | Cortex-A53 | 76.80MIPS | 
	OS :	 Linux 5.10.103-v7+
	Host :	 emonpi | emonpi | (192.168.178.35)
	Date :	 2023-08-08 21:15:44 CEST
	Uptime :	 21:15:44 up 13 min,  1 user,  load average: 0.06, 0.11, 0.09

Memory
	RAM :	 Used: 40.22%
		Total :	 923.21 MB
		Used :	 371.28 MB
		Free :	 551.93 MB
	Swap :	 Used: 0.00%
		Total :	 100 MB
		Used :	 0 B
		Free :	 100 MB

Disk
	 :	 - / :	 Used: 12.94%
		Total :	 19.41 GB
		Used :	 2.51 GB
		Free :	 16.08 GB
		Read Load :	 n/a
		Write Load :	 n/a
		Load Time :	 n/a
	/var/opt/emoncms :	 Used: 0.04%
		Total :	 9.56 GB
		Used :	 3.77 MB
		Free :	 9.07 GB
		Read Load :	 n/a
		Write Load :	 n/a
		Load Time :	 n/a
	/boot :	 Used: 19.14%
		Total :	 252.05 MB
		Used :	 48.24 MB
		Free :	 203.81 MB
		Read Load :	 n/a
		Write Load :	 n/a
		Load Time :	 n/a
	/var/log :	 Used: 9.03%
		Total :	 50 MB
		Used :	 4.52 MB
		Free :	 45.48 MB
		Read Load :	 n/a
		Write Load :	 n/a
		Load Time :	 n/a

HTTP
	Server :	 Apache/2.4.38 (Raspbian) HTTP/1.1 CGI/1.1 80

MySQL
	Version :	 10.3.39-MariaDB-0+deb10u1
	Host :	 127.0.0.1 (127.0.0.1)
	Date :	 2023-08-08 21:15:43 (UTC 02:00‌​)
	Stats :	 Uptime: 799  Threads: 12  Questions: 532  Slow queries: 0  Opens: 47  Flush tables: 1  Open tables: 40  Queries per second avg: 0.665

Redis
	Version :	 
		Redis Server :	 5.0.14
		PHP Redis :	 6.0.0-dev
	Host :	 localhost:6379
	Size :	 62 keys (864.45K)
	Uptime :	 0 days

MQTT Server
	Version :	 Mosquitto 1.5.7
	Host :	 localhost:1883 (127.0.0.1)

PHP
	Version :	 8.1.21 (Zend Version 4.1.21)
	Run user :	 User: www-data Group: www-data video Script Owner: pi
	Modules :	 apache2handler calendar Core ctype curl date exif FFI fileinfo filter ftp gd gettext hash iconv json libxml mbstring mosquitto v0.4.0mysqli mysqlnd vmysqlnd 8.1.21openssl pcre PDO pdo_mysql Phar posix readline redis v6.0.0-devReflection session shmop sockets sodium SPL standard sysvmsg sysvsem sysvshm tokenizer Zend OPcache zlib 
Pi
	Model :	 Raspberry Pi 3 Model B Rev 1.2 - 1GB (Sony UK)
	Serial num. :	 3C1E76BB
	CPU Temperature :	 44.01°C
	GPU Temperature :	 44.5°C
	emonpiRelease :	 emonSD-10Nov22
	File-system :	 read-write

Furthermore the full apache2 log (/var/log/apache2/error.log):

[Wed Jul 26 18:12:14.821059 2023] [mpm_event:notice] [pid 1908:tid 1996177984] AH00489: Apache/2.4.38 (Raspbian) configured -- resuming normal operations
[Wed Jul 26 18:12:14.821533 2023] [core:notice] [pid 1908:tid 1996177984] AH00094: Command line: '/usr/sbin/apache2'
[Wed Jul 26 18:12:22.921236 2023] [mpm_event:notice] [pid 1908:tid 1996177984] AH00491: caught SIGTERM, shutting down

which only contains old messages.

The apach2-error.log contains:

AH01626: authorization result of <RequireAny>: granted, referer: http://192.168.178.35/input/view
[Tue Aug 08 21:07:25.651145 2023] [deflate:debug] [pid 812] mod_deflate.c(856): [client 192.168.178.201:52522] AH01384: Zlib: Compressed 2723 to 381 : URL /index.php, referer: http://192.168.178.35/input/view
[Tue Aug 08 21:07:30.588389 2023] [authz_core:debug] [pid 812] mod_authz_core.c(820): [client 192.168.178.201:52522] AH01626: authorization result of Require all granted: granted, referer: http://192.168.178.35/input/view
[Tue Aug 08 21:07:30.588474 2023] [authz_core:debug] [pid 812] mod_authz_core.c(820): [client 192.168.178.201:52522] AH01626: authorization result of <RequireAny>: granted, referer: http://192.168.178.35/input/view
[Tue Aug 08 21:07:30.588799 2023] [authz_core:debug] [pid 812] mod_authz_core.c(820): [client 192.168.178.201:52522] AH01626: authorization result of Require all granted: granted, referer: http://192.168.178.35/input/view
[Tue Aug 08 21:07:30.588829 2023] [authz_core:debug] [pid 812] mod_authz_core.c(820): [client 192.168.178.201:52522] AH01626: authorization result of <RequireAny>: granted, referer: http://192.168.178.35/input/view
[Tue Aug 08 21:07:30.600931 2023] [deflate:debug] [pid 812] mod_deflate.c(856): [client 192.168.178.201:52522] AH01384: Zlib: Compressed 227 to 98 : URL /index.php, referer: http://192.168.178.35/input/view
[Tue Aug 08 21:07:30.622558 2023] [authz_core:debug] [pid 812] mod_authz_core.c(820): [client 192.168.178.201:52522] AH01626: authorization result of Require all granted: granted, referer: http://192.168.178.35/input/view
[Tue Aug 08 21:07:30.622626 2023] [authz_core:debug] [pid 812] mod_authz_core.c(820): [client 192.168.178.201:52522] AH01626: authorization result of <RequireAny>: granted, referer: http://192.168.178.35/input/view
[Tue Aug 08 21:07:30.622941 2023] [authz_core:debug] [pid 812] mod_authz_core.c(820): [client 192.168.178.201:52522] AH01626: authorization result of Require all granted: granted, referer: http://192.168.178.35/input/view
[Tue Aug 08 21:07:30.622971 2023] [authz_core:debug] [pid 812] mod_authz_core.c(820): [client 192.168.178.201:52522] AH01626: authorization result of <RequireAny>: granted, referer: http://192.168.178.35/input/view
[Tue Aug 08 21:07:30.651872 2023] [deflate:debug] [pid 812] mod_deflate.c(856): [client 192.168.178.201:52522] AH01384: Zlib: Compressed 2697 to 379 : URL /index.php, referer: http://192.168.178.35/input/view

And lastly the redis log:

                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis 5.0.14 (00000000/0) 32 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                   
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
 |    `-._   `._    /     _.-'    |     PID: 693
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           http://redis.io        
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               

693:M 08 Aug 2023 21:02:24.570 # Server initialized
693:M 08 Aug 2023 21:02:24.570 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
693:M 08 Aug 2023 21:02:24.583 * DB loaded from disk: 0.012 seconds
693:M 08 Aug 2023 21:02:24.583 * Ready to accept connections

The other part of emoncms.log after startup:

2023-08-08 18:29:35.440|INFO|rememberme_model.php|login
2023-08-08 18:29:35.441|INFO|rememberme_model.php|getCookieValues: not present for: 195.96.137.7
2023-08-08 18:29:35.443|WARN|index.php|406 Not Acceptable|indice
2023-08-08 18:29:35.-407|INFO|rememberme_model.php|login
2023-08-08 18:29:35.-406|INFO|rememberme_model.php|getCookieValues: not present for: 195.96.137.7
2023-08-08 18:29:35.-404|WARN|index.php|406 Not Acceptable|index
2023-08-08 18:29:35.-45|INFO|rememberme_model.php|login
2023-08-08 18:29:35.-45|INFO|rememberme_model.php|getCookieValues: not present for: 195.96.137.7
2023-08-08 18:29:35.-43|WARN|index.php|406 Not Acceptable|inicio
2023-08-08 18:29:35.-26|INFO|rememberme_model.php|login
2023-08-08 18:29:35.-26|INFO|rememberme_model.php|getCookieValues: not present for: 195.96.137.7
2023-08-08 18:29:36.125|INFO|rememberme_model.php|login
2023-08-08 18:29:36.125|INFO|rememberme_model.php|getCookieValues: not present for: 195.96.137.7
2023-08-08 18:30:29.-242|WARN|emoncms_mqtt.php|Not connected, retrying connection
2023-08-08 18:30:29.-151|WARN|emoncms_mqtt.php|Connecting to MQTT server: Connection Accepted.: code: 0
2023-08-08 18:34:01.-68|ERROR|index.php|Not Authenticated|admin/log
2023-08-08 18:40:12.-169|WARN|emoncms_mqtt.php|Not connected, retrying connection
2023-08-08 18:40:12.-58|WARN|emoncms_mqtt.php|Connecting to MQTT server: Connection Accepted.: code: 0
2023-08-08 18:46:42.-136|WARN|emoncms_mqtt.php|Not connected, retrying connection
2023-08-08 18:46:42.-35|WARN|emoncms_mqtt.php|Connecting to MQTT server: Connection Accepted.: code: 0
2023-08-08 18:49:37.-132|WARN|emoncms_mqtt.php|Not connected, retrying connection
2023-08-08 18:49:37.-37|WARN|emoncms_mqtt.php|Connecting to MQTT server: Connection Accepted.: code: 0
2023-08-08 19:02:27.-443|INFO|feedwriter.php|Starting feedwriter script
2023-08-08 19:02:27.-397|INFO|demandshaper_run.php|Starting demandshaper service
2023-08-08 19:02:27.-438|INFO|emoncms_mqtt.php|Starting MQTT Input script
2023-08-08 19:02:27.-263|INFO|feedwriter.php|Buffered feed writer daemon started with sleep 300s...
2023-08-08 19:02:27.-247|INFO|feed_model.php|EngineClass() Autoloaded new instance of 'RedisBuffer'.
2023-08-08 19:02:27.-169|WARN|emoncms_mqtt.php|Not connected, retrying connection
2023-08-08 19:02:27.-157|WARN|emoncms_mqtt.php|Connecting to MQTT server: Connection Accepted.: code: 0
2023-08-08 19:02:27.-157|INFO|emoncms_mqtt.php|Subscribed to: emon/# ID - 1
2023-08-08 19:02:27.-146|INFO|emoncms_mqtt.php|Callback subscribed to topic: 
2023-08-08 19:03:12.370|INFO|emoncms_mqtt.php|emon/esp/verwarming 20.062500
2023-08-08 19:03:12.433|INFO|emoncms_mqtt.php|emon/esp/woonkamer 20.562500
2023-08-08 19:03:17.403|INFO|emoncms_mqtt.php|emon/esp/uptime 182639.687500
2023-08-08 19:03:19.-412|INFO|rememberme_model.php|login
2023-08-08 19:03:19.-402|INFO|rememberme_model.php|getCookieValues: not present for: 192.168.178.201
2023-08-08 19:03:19.-147|INFO|emoncms_mqtt.php|emon/SolarEdge/Vermogen 0.0
2023-08-08 19:03:19.-132|INFO|emoncms_mqtt.php|emon/SolarEdge/Energie 50363.852
2023-08-08 19:03:19.-118|INFO|emoncms_mqtt.php|emon/SolarEdge/Vermogen P1 -0.0
2023-08-08 19:03:19.-104|INFO|emoncms_mqtt.php|emon/SolarEdge/Vermogen P2 -0.0
2023-08-08 19:03:19.-91|INFO|emoncms_mqtt.php|emon/SolarEdge/Vermogen P3 -0.0
2023-08-08 19:03:19.-77|INFO|emoncms_mqtt.php|emon/SolarEdge/current 0.0
2023-08-08 19:03:19.-64|INFO|emoncms_mqtt.php|emon/SolarEdge/l1_current 0.0

And all the logs at:

Yes that is the lowest, sorry :frowning:

Potentially your issue. I have seen emoncms overwhelmed by too many topics too quickly. As Emoncms can only process an input at a minimum of 5s, then flooding it every second is pointless.

Thankyou for that replication info above @Rutger0000 that’s useful to know. I wonder if you would get the same issue with emonSD-10Nov22 https://docs.openenergymonitor.org/emonsd/download.html? sounds like you might given that you tried Raspberry Pi OS Lite 2021-03-25

Could the issue be logs filling up or something like that?

Another possibility could be this 10ms delay in emoncms_mqtt perhaps the frequency of the MQTT data is causing the queue to stack up? https://github.com/emoncms/emoncms/blob/master/scripts/services/emoncms_mqtt/emoncms_mqtt.php#L212 You could try reducing it… Another issue might be that emoncms is not processing the messages fast enough beyond that delay specifically, there’s no rate limiting on the local emoncms so it might be overloading it …

Is a very high frequency and volume of MQTT data a common thread for others reporting this issue?

In my case on my Proxmox CT of EmonCMS running on an x86 setup in 64 bit. I only have two OpenEVSE reporting to EmonCMS via MQTT. Those only send every 30 seconds and yet emoncms_mqtt was using up all RAM on the CT as reported. I’ll give the delay reduction a try though, just as soon as I get emoncms reinstalled.

That said, I would think, at least in my case, with a CT running on a Ryzen that there is more than enough “horsepower” behind processing the data coming in. Not sure if there is a built in limit of some sort on how quick it processes the data that might be lifted. Based on what you said and how I understood it, there is not.

Hello @Ryan_S two OpenEVSE’s every 30 seconds should be no problem at all! It sounds like it should work for you so must be something else…

The other thought is that there could be some latency either with redis or mysql that’s causing emoncms_mqtt to run slow? Il see if I can create a test script to push the limit on the post rate and see what happens…