Emoncms_mqtt.php very high ram use

TrystanLea · 18 April 2023 14:43

Nothing of any substance apart from that redis check. How to debug and get to the bottom of this? emoncms_mqtt seems to be running just fine here so it’s difficult for me to dig into it.

Im running Mosquitto 2.0.11 & PHP Module: mosquitto v0.4.0 which seems to be the same as you @Darren. @ullmf @lucwillems what versions are you running?

What sort of MQTT traffic do you all have?

Is redis enabled and running ok? looks like it is for you @Darren.

What do the emoncms inputs look like? it might be worth clearing inputs and devices that are unused…

emonpi.local/device/clean.json
emonpi.local/input/clean.json

borpin · 18 April 2023 15:50

@TrystanLea can you try a fresh build? Could easily be an underlying package that has changed.

Ryan_S · 27 July 2023 16:49

I’m also having this issue and wanted to see if there has been any movement on this.

Ryan_S · 28 July 2023 02:04

Going on your questions. My mosquitto is the same as yours. My inputs are only two OpenEVSE MQTT and two IotaWatts. Yet, I am also experiencing the memory issue with emoncms_mqtt.php.

ullmf · 28 July 2023 06:40

I have not been able to find a solution to the problem so far.
I have tested different Ubuntu versions and also different PHP versions without success.
Furthermore, a lot of data was lost, there were always gaps in the graphics.
After many unsuccessful attempts, I reinstalled the system on an old PI3 and it runs without problems.
This is exactly what I didn’t want, but I found no other solution.

Ryan_S · 28 July 2023 16:32

I think my solution now is to setup InfluxDB and Grafana. It looks like I can do the same things using MQTT with that as I could with EmonCMS. Namely ensuring my two OpenEVSE can follow solar output for charging.

I would happily work with folks to find a solution here so others could go this route with EmonCMS on a server setup, but that requires some back and forth.

borpin · 28 July 2023 20:32

The issue is it isn’t repeatable. I run emoncms on Proxmox quite happily with a huge number of MQTT feeds.

I think you have been using 64bit. Try 32bit 22.04 and see if that works.

Ryan_S · 28 July 2023 21:27

From what I can see, Ubuntu 22.04 doesn’t come in 32 bit. I thought they stopped offering it in 32 bit a couple of years ago.

I guess I should ask. But did you install your LXC using the scripts? I didn’t notice it ask me for 64 or 32 bit during the MQTT install. Looks like that is still 64 bit in the end. If this is an issue with 64 bit I have a feeling it might begin to creep into the Pi world soon enough as I believe the Pi 4 is 64 bit now.

Rutger0000 · 7 August 2023 21:34

I can recreate/experience the same problem the emoncms_mqtt.php high ram usage problem by installing EmonCMS through the scripts at

github.com

openenergymonitor/EmonScripts/blob/master/docs/install.md

# EmonCMS Install Scripts

## Introduction

The [EmonScripts build scripts](https://github.com/openenergymonitor/EmonScripts/) can be used to build a fully fledged emonCMS installation on debian based operating systems, including: installation of LAMP server and related packages, redis, MQTT, emonCMS core, emonCMS modules, and if applicable, emonhub & RaspberryPi support for serial port, and WiFi access point.

We use EmonScripts to build the pre-built emonSD SD card image for the Raspberry Pi. If you prefer to build your own or customise the installation using EmonScripts directly is a good approach.

The installation process is carried out by a series of scripts that install each required component. These can be explored here: [https://github.com/openenergymonitor/EmonScripts/tree/master/install](https://github.com/openenergymonitor/EmonScripts/tree/master/install).

It is **strongly recommended** that you dedicate a single device (NUC/VM/Pi) to emoncms. Using it alongside other software can prove problematic.

**Before starting,** please review the EmonScripts issue list for any new issues that might affect your build. We try to list new issues and notes from recent builds there that might be useful or required to complete a successful build: [https://github.com/openenergymonitor/EmonScripts/issues](https://github.com/openenergymonitor/EmonScripts/issues).

## Building Your Own - Base OS Preparation

### RaspberryPi

To install on a RaspberryPi, a number of tasks are first required. Please follow [these instructions first](rpi-install.md).

This file has been truncated. show original

and

github.com

openenergymonitor/EmonScripts/blob/master/docs/rpi-install.md

# EmonScripts: SD Card Preparation for RaspberryPi

1\. Start by following the first part of this nice guide by Pimoroni: [Setting up a Headless Pi](https://learn.pimoroni.com/article/setting-up-a-headless-pi) to flash the base OS Image. Choose the Lite 32-bit image. Set a username (`pi`) & password, Enable SSH, and add your WiFi credentials if required. **Do not insert into the Pi**.

2\. Eject the SD Card from computer and reinsert. On Windows open 'Disk Management' and look for the SDCard. At the end there will be a lareg area of unallocated space. Right Click on that and select `New Simple Volume`. What you select here does not matter, but do not format it (waste of time!). Eject card and insert into the Pi. Let the Pi do it's first boot (be patient it reboots several times).

3\. Once it has booted SSH into the Pi. e.g. `ssh [email protected]` (password: your password)` or use PuTTY.

4\. You now need to delete the partition you created. It was there to prevent the Pi expanding the `rootfs` on first boot. You cannot easily shrink `rootfs`! You will then create a new partition at the end of the card and then increase `rootfs` to fill the remaining space. Follow the following instructions:

5\. Delete additional partition

```shell
sudo parted /dev/mmcblk0 rm 3
```

6\. Create a new partition at the end - you specify the **start** point and the partition filles to the end of the card. The **example** 20G figure is the **start** point for the data partition and can vary depending on size of card of course. For testing make it small so the mkfs doesn't take an age!! You will get an error message in red - ignore it.

Create partition starting at XX and filling to the end.

This file has been truncated. show original

Using the following Raspberry Pi OS Lite version on a Raspberry Pi 3 Model B Rev 1.2

Raspberry Pi OS Lite

Release date: May 3rd 2023
System: 32-bit
Kernel version: 6.1
Debian version: 11 (bullseye)

I can also recreate it by instead using Raspberry Pi OS Lite 2021-03-25 which is on Debian version: 10 (buster) and Linux 5. Note that for this version of Raspberry Pi OS Lite the packages were first upgraded, but still Debian 10 (buster) and Linux 5.

Problem description
After 20 minutes the Raspberry Pi runs out of memory and freezes.

I installed my previous version of EmonCMS using EmonSD which works perfectly with MQTT.
Old install
On the same Raspberry Pi with the standard EmonSD (versions below)

emonpiRelease: emonSD-08May21,
EmonCMS: low-write 11.3.0, 
OS: Linux 5.10.63-v7+Components: Emoncms Core v11.3.0 | App v2.6.10 | EmonHub Config v2.1.5 | Dashboard v2.3.3 | Device v2.2.2 | Graph v2.2.3 | Network Setup v1.0.2 | WiFi v2.1.1 | Backup v2.3.2 | DemandShaper v2.2.2 | Postprocess v2.2.7 | Sync v2.1.4 | Usefulscripts v2.3.10 | EmonScripts v1.6.7 | RFM2Pi v1.4.1 | Emonhub v2.3.3 | EmonPi v2.9.5.

I am happy to share more information

borpin · 8 August 2023 09:32

@TrystanLea - one for you

borpin · 8 August 2023 09:39

Logs, please (emoncms set to DEBUG and php/apache logs perhaps). Also how many topics are there on the emon base topic?

Ryan_S · 8 August 2023 18:22

Thank you for testing that with a Pi. I didn’t have one handy (still in storage due to moving and renovating the new place) to try that out with. Hoping this gets resolved.

Rutger0000 · 8 August 2023 19:25

I cannot see/find any particular interesting things in the logs.

In the settings.ini of emoncms, I set the log level to INFO.

The lowest log level that I can configure in settings.ini is INFO.

[log]
; Log Level: 1=INFO, 2=WARN, 3=ERROR
level = 1

I am not sure how to configure a lower loglevel here. I tried level = 0, but that does not work.

In my case, there are 20 topics updating every seconds and there are 3 topics updating every minute. The 20 topics every second are fed with data from a Python script running on the same Raspberry Pi.

From tail /var/log/emoncms/emoncms.log

2023-08-08 19:07:37.-125|INFO|emoncms_mqtt.php|emon/SolarEdge/Vermogen P1 -0.0
2023-08-08 19:07:37.-111|INFO|emoncms_mqtt.php|emon/SolarEdge/Vermogen P2 -0.0
2023-08-08 19:07:37.-98|INFO|emoncms_mqtt.php|emon/SolarEdge/Vermogen P3 -0.0
2023-08-08 19:07:37.-84|INFO|emoncms_mqtt.php|emon/SolarEdge/current 0.0
2023-08-08 19:07:37.-71|INFO|emoncms_mqtt.php|emon/SolarEdge/l1_current 0.0
2023-08-08 19:07:37.-57|INFO|emoncms_mqtt.php|emon/SolarEdge/l2_current 0.0
2023-08-08 19:07:37.-44|INFO|emoncms_mqtt.php|emon/SolarEdge/l3_current 0.0
2023-08-08 19:07:37.-21|INFO|emoncms_mqtt.php|emon/SolarEdge/l1n_voltage 232.3
2023-08-08 19:07:37.-08|INFO|emoncms_mqtt.php|emon/SolarEdge/l2n_voltage 232.20000000000002
2023-08-08 19:07:38.005|INFO|emoncms_mqtt.php|emon/SolarEdge/l3n_voltage 232.3
2023-08-08 19:07:38.019|INFO|emoncms_mqtt.php|emon/SolarEdge/frequency 49.97
2023-08-08 19:07:38.032|INFO|emoncms_mqtt.php|emon/SolarEdge/power_apparent 0.0
2023-08-08 19:07:38.045|INFO|emoncms_mqtt.php|emon/SolarEdge/power_reactive 0.0
2023-08-08 19:07:38.059|INFO|emoncms_mqtt.php|emon/SolarEdge/power_factor 0.0
2023-08-08 19:07:38.072|INFO|emoncms_mqtt.php|emon/SolarEdge/current_dc 0.0
2023-08-08 19:07:38.085|INFO|emoncms_mqtt.php|emon/SolarEdge/voltage_dc 23.200000000000003
2023-08-08 19:07:38.098|INFO|emoncms_mqtt.php|emon/SolarEdge/power_dc 0.0
2023-08-08 19:07:38.112|INFO|emoncms_mqtt.php|emon/SolarEdge/temperature 36.32
2023-08-08 19:07:38.-222|INFO|emoncms_mqtt.php|emon/SolarEdge/Vermogen 0.0
2023-08-08 19:07:38.-208|INFO|emoncms_mqtt.php|emon/SolarEdge/Energie 50363.852
2023-08-08 19:07:38.-194|INFO|emoncms_mqtt.php|emon/SolarEdge/Vermogen P1 -0.0
2023-08-08 19:07:38.-180|INFO|emoncms_mqtt.php|emon/SolarEdge/Vermogen P2 -0.0
2023-08-08 19:07:38.-166|INFO|emoncms_mqtt.php|emon/SolarEdge/Vermogen P3 -0.0
2023-08-08 19:07:38.-153|INFO|emoncms_mqtt.php|emon/SolarEdge/current 0.0
2023-08-08 19:07:38.-139|INFO|emoncms_mqtt.php|emon/SolarEdge/l1_current 0.0
2023-08-08 19:07:38.-126|INFO|emoncms_mqtt.php|emon/SolarEdge/l2_current 0.0
2023-08-08 19:07:38.-113|INFO|emoncms_mqtt.php|emon/SolarEdge/l3_current 0.0
2023-08-08 19:07:38.-100|INFO|emoncms_mqtt.php|emon/SolarEdge/l1n_voltage 232.60000000000002
2023-08-08 19:07:38.-86|INFO|emoncms_mqtt.php|emon/SolarEdge/l2n_voltage 231.9
2023-08-08 19:07:38.-73|INFO|emoncms_mqtt.php|emon/SolarEdge/l3n_voltage 232.60000000000002
2023-08-08 19:07:38.-60|INFO|emoncms_mqtt.php|emon/SolarEdge/frequency 49.97
2023-08-08 19:07:38.-47|INFO|emoncms_mqtt.php|emon/SolarEdge/power_apparent 0.0
2023-08-08 19:07:38.-33|INFO|emoncms_mqtt.php|emon/SolarEdge/power_reactive 0.0
2023-08-08 19:07:38.-20|INFO|emoncms_mqtt.php|emon/SolarEdge/power_factor 0.0
2023-08-08 19:07:38.-07|INFO|emoncms_mqtt.php|emon/SolarEdge/current_dc 0.0
2023-08-08 19:07:39.005|INFO|emoncms_mqtt.php|emon/SolarEdge/voltage_dc 22.400000000000002
2023-08-08 19:07:39.019|INFO|emoncms_mqtt.php|emon/SolarEdge/power_dc 0.0
2023-08-08 19:07:39.032|INFO|emoncms_mqtt.php|emon/SolarEdge/temperature 36.29
2023-08-08 19:07:39.-220|INFO|emoncms_mqtt.php|emon/SolarEdge/Vermogen 0.0
2023-08-08 19:07:39.-207|INFO|emoncms_mqtt.php|emon/SolarEdge/Energie 50363.852
2023-08-08 19:07:39.-195|INFO|emoncms_mqtt.php|emon/SolarEdge/Vermogen P1 -0.0
2023-08-08 19:07:39.-183|INFO|emoncms_mqtt.php|emon/SolarEdge/Vermogen P2 -0.0
2023-08-08 19:07:39.-171|INFO|emoncms_mqtt.php|emon/SolarEdge/Vermogen P3 -0.0
2023-08-08 19:07:39.-159|INFO|emoncms_mqtt.php|emon/SolarEdge/current 0.0
2023-08-08 19:07:39.-147|INFO|emoncms_mqtt.php|emon/SolarEdge/l1_current 0.0
2023-08-08 19:07:39.-135|INFO|emoncms_mqtt.php|emon/SolarEdge/l2_current 0.0
2023-08-08 19:07:39.-122|INFO|emoncms_mqtt.php|emon/SolarEdge/l3_current 0.0
2023-08-08 19:07:39.-110|INFO|emoncms_mqtt.php|emon/SolarEdge/l1n_voltage 232.5
2023-08-08 19:07:39.-98|INFO|emoncms_mqtt.php|emon/SolarEdge/l2n_voltage 231.8
2023-08-08 19:07:39.-86|INFO|emoncms_mqtt.php|emon/SolarEdge/l3n_voltage 232.20000000000002
2023-08-08 19:07:39.-74|INFO|emoncms_mqtt.php|emon/SolarEdge/frequency 49.97
2023-08-08 19:07:39.-62|INFO|emoncms_mqtt.php|emon/SolarEdge/power_apparent 0.0
2023-08-08 19:07:39.-50|INFO|emoncms_mqtt.php|emon/SolarEdge/power_reactive 0.0
2023-08-08 19:07:39.-38|INFO|emoncms_mqtt.php|emon/SolarEdge/power_factor 0.0
2023-08-08 19:07:39.-26|INFO|emoncms_mqtt.php|emon/SolarEdge/current_dc 0.0
2023-08-08 19:07:39.-14|INFO|emoncms_mqtt.php|emon/SolarEdge/voltage_dc 21.400000000000002
2023-08-08 19:07:39.-02|INFO|emoncms_mqtt.php|emon/SolarEdge/power_dc 0.0
2023-08-08 19:07:40.009|INFO|emoncms_mqtt.php|emon/SolarEdge/temperature 36.29

More details on the installation, note EmonHUB is disabled

Server Information
-----------------------

Services
	emoncms_mqtt :	 Active Running                      
	feedwriter :	 Active Running - sleep 300s 0 feed points pending write
	service-runner :	 Active Running                      
	redis-server :	 Active Running                      
	mosquitto :	 Active Running                      
	demandshaper :	 Active Running                      
	emonPiLCD :	 Failed loaded failed failed                      
	emonhub :	 Inactive loaded inactive dead                      
Emoncms
	Version :	 low-write 11.3.22
	Git :	 
		URL :	 https://github.com/emoncms/emoncms.git
		Branch :	 * stable
		Describe :	 11.3.22
	Components :	 Emoncms Core v11.3.22 | App v2.7.3 | EmonHub Config v2.1.5 | Dashboard v2.3.3 | Device v2.2.2 | Graph v2.2.3 | Network Setup v1.0.2 | WiFi v2.1.1 | Backup v2.3.2 | DemandShaper v2.2.2 | Postprocess v2.2.7 | Sync v2.1.4 | Usefulscripts v2.3.10 | EmonScripts v1.6.20 | RFM2Pi v1.4.1 | Avrdude-rpi v1.0.1 | Emonhub v2.5.8 | EmonPi v2.9.5

Server
	CPU :	 1 Threads(s) | 4 Core(s) | 1 Sockets(s) | Cortex-A53 | 76.80MIPS | 
	OS :	 Linux 5.10.103-v7+
	Host :	 emonpi | emonpi | (192.168.178.35)
	Date :	 2023-08-08 21:15:44 CEST
	Uptime :	 21:15:44 up 13 min,  1 user,  load average: 0.06, 0.11, 0.09

Memory
	RAM :	 Used: 40.22%
		Total :	 923.21 MB
		Used :	 371.28 MB
		Free :	 551.93 MB
	Swap :	 Used: 0.00%
		Total :	 100 MB
		Used :	 0 B
		Free :	 100 MB

Disk
	 :	 - / :	 Used: 12.94%
		Total :	 19.41 GB
		Used :	 2.51 GB
		Free :	 16.08 GB
		Read Load :	 n/a
		Write Load :	 n/a
		Load Time :	 n/a
	/var/opt/emoncms :	 Used: 0.04%
		Total :	 9.56 GB
		Used :	 3.77 MB
		Free :	 9.07 GB
		Read Load :	 n/a
		Write Load :	 n/a
		Load Time :	 n/a
	/boot :	 Used: 19.14%
		Total :	 252.05 MB
		Used :	 48.24 MB
		Free :	 203.81 MB
		Read Load :	 n/a
		Write Load :	 n/a
		Load Time :	 n/a
	/var/log :	 Used: 9.03%
		Total :	 50 MB
		Used :	 4.52 MB
		Free :	 45.48 MB
		Read Load :	 n/a
		Write Load :	 n/a
		Load Time :	 n/a

HTTP
	Server :	 Apache/2.4.38 (Raspbian) HTTP/1.1 CGI/1.1 80

MySQL
	Version :	 10.3.39-MariaDB-0+deb10u1
	Host :	 127.0.0.1 (127.0.0.1)
	Date :	 2023-08-08 21:15:43 (UTC 02:00‌)
	Stats :	 Uptime: 799  Threads: 12  Questions: 532  Slow queries: 0  Opens: 47  Flush tables: 1  Open tables: 40  Queries per second avg: 0.665

Redis
	Version :	 
		Redis Server :	 5.0.14
		PHP Redis :	 6.0.0-dev
	Host :	 localhost:6379
	Size :	 62 keys (864.45K)
	Uptime :	 0 days

MQTT Server
	Version :	 Mosquitto 1.5.7
	Host :	 localhost:1883 (127.0.0.1)

PHP
	Version :	 8.1.21 (Zend Version 4.1.21)
	Run user :	 User: www-data Group: www-data video Script Owner: pi
	Modules :	 apache2handler calendar Core ctype curl date exif FFI fileinfo filter ftp gd gettext hash iconv json libxml mbstring mosquitto v0.4.0mysqli mysqlnd vmysqlnd 8.1.21openssl pcre PDO pdo_mysql Phar posix readline redis v6.0.0-devReflection session shmop sockets sodium SPL standard sysvmsg sysvsem sysvshm tokenizer Zend OPcache zlib 
Pi
	Model :	 Raspberry Pi 3 Model B Rev 1.2 - 1GB (Sony UK)
	Serial num. :	 3C1E76BB
	CPU Temperature :	 44.01°C
	GPU Temperature :	 44.5°C
	emonpiRelease :	 emonSD-10Nov22
	File-system :	 read-write

Furthermore the full apache2 log (/var/log/apache2/error.log):

[Wed Jul 26 18:12:14.821059 2023] [mpm_event:notice] [pid 1908:tid 1996177984] AH00489: Apache/2.4.38 (Raspbian) configured -- resuming normal operations
[Wed Jul 26 18:12:14.821533 2023] [core:notice] [pid 1908:tid 1996177984] AH00094: Command line: '/usr/sbin/apache2'
[Wed Jul 26 18:12:22.921236 2023] [mpm_event:notice] [pid 1908:tid 1996177984] AH00491: caught SIGTERM, shutting down

which only contains old messages.

The apach2-error.log contains:

AH01626: authorization result of <RequireAny>: granted, referer: http://192.168.178.35/input/view
[Tue Aug 08 21:07:25.651145 2023] [deflate:debug] [pid 812] mod_deflate.c(856): [client 192.168.178.201:52522] AH01384: Zlib: Compressed 2723 to 381 : URL /index.php, referer: http://192.168.178.35/input/view
[Tue Aug 08 21:07:30.588389 2023] [authz_core:debug] [pid 812] mod_authz_core.c(820): [client 192.168.178.201:52522] AH01626: authorization result of Require all granted: granted, referer: http://192.168.178.35/input/view
[Tue Aug 08 21:07:30.588474 2023] [authz_core:debug] [pid 812] mod_authz_core.c(820): [client 192.168.178.201:52522] AH01626: authorization result of <RequireAny>: granted, referer: http://192.168.178.35/input/view
[Tue Aug 08 21:07:30.588799 2023] [authz_core:debug] [pid 812] mod_authz_core.c(820): [client 192.168.178.201:52522] AH01626: authorization result of Require all granted: granted, referer: http://192.168.178.35/input/view
[Tue Aug 08 21:07:30.588829 2023] [authz_core:debug] [pid 812] mod_authz_core.c(820): [client 192.168.178.201:52522] AH01626: authorization result of <RequireAny>: granted, referer: http://192.168.178.35/input/view
[Tue Aug 08 21:07:30.600931 2023] [deflate:debug] [pid 812] mod_deflate.c(856): [client 192.168.178.201:52522] AH01384: Zlib: Compressed 227 to 98 : URL /index.php, referer: http://192.168.178.35/input/view
[Tue Aug 08 21:07:30.622558 2023] [authz_core:debug] [pid 812] mod_authz_core.c(820): [client 192.168.178.201:52522] AH01626: authorization result of Require all granted: granted, referer: http://192.168.178.35/input/view
[Tue Aug 08 21:07:30.622626 2023] [authz_core:debug] [pid 812] mod_authz_core.c(820): [client 192.168.178.201:52522] AH01626: authorization result of <RequireAny>: granted, referer: http://192.168.178.35/input/view
[Tue Aug 08 21:07:30.622941 2023] [authz_core:debug] [pid 812] mod_authz_core.c(820): [client 192.168.178.201:52522] AH01626: authorization result of Require all granted: granted, referer: http://192.168.178.35/input/view
[Tue Aug 08 21:07:30.622971 2023] [authz_core:debug] [pid 812] mod_authz_core.c(820): [client 192.168.178.201:52522] AH01626: authorization result of <RequireAny>: granted, referer: http://192.168.178.35/input/view
[Tue Aug 08 21:07:30.651872 2023] [deflate:debug] [pid 812] mod_deflate.c(856): [client 192.168.178.201:52522] AH01384: Zlib: Compressed 2697 to 379 : URL /index.php, referer: http://192.168.178.35/input/view

And lastly the redis log:

                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis 5.0.14 (00000000/0) 32 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                   
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
 |    `-._   `._    /     _.-'    |     PID: 693
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           http://redis.io        
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               

693:M 08 Aug 2023 21:02:24.570 # Server initialized
693:M 08 Aug 2023 21:02:24.570 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
693:M 08 Aug 2023 21:02:24.583 * DB loaded from disk: 0.012 seconds
693:M 08 Aug 2023 21:02:24.583 * Ready to accept connections

Rutger0000 · 8 August 2023 19:28

The other part of emoncms.log after startup:

2023-08-08 18:29:35.440|INFO|rememberme_model.php|login
2023-08-08 18:29:35.441|INFO|rememberme_model.php|getCookieValues: not present for: 195.96.137.7
2023-08-08 18:29:35.443|WARN|index.php|406 Not Acceptable|indice
2023-08-08 18:29:35.-407|INFO|rememberme_model.php|login
2023-08-08 18:29:35.-406|INFO|rememberme_model.php|getCookieValues: not present for: 195.96.137.7
2023-08-08 18:29:35.-404|WARN|index.php|406 Not Acceptable|index
2023-08-08 18:29:35.-45|INFO|rememberme_model.php|login
2023-08-08 18:29:35.-45|INFO|rememberme_model.php|getCookieValues: not present for: 195.96.137.7
2023-08-08 18:29:35.-43|WARN|index.php|406 Not Acceptable|inicio
2023-08-08 18:29:35.-26|INFO|rememberme_model.php|login
2023-08-08 18:29:35.-26|INFO|rememberme_model.php|getCookieValues: not present for: 195.96.137.7
2023-08-08 18:29:36.125|INFO|rememberme_model.php|login
2023-08-08 18:29:36.125|INFO|rememberme_model.php|getCookieValues: not present for: 195.96.137.7
2023-08-08 18:30:29.-242|WARN|emoncms_mqtt.php|Not connected, retrying connection
2023-08-08 18:30:29.-151|WARN|emoncms_mqtt.php|Connecting to MQTT server: Connection Accepted.: code: 0
2023-08-08 18:34:01.-68|ERROR|index.php|Not Authenticated|admin/log
2023-08-08 18:40:12.-169|WARN|emoncms_mqtt.php|Not connected, retrying connection
2023-08-08 18:40:12.-58|WARN|emoncms_mqtt.php|Connecting to MQTT server: Connection Accepted.: code: 0
2023-08-08 18:46:42.-136|WARN|emoncms_mqtt.php|Not connected, retrying connection
2023-08-08 18:46:42.-35|WARN|emoncms_mqtt.php|Connecting to MQTT server: Connection Accepted.: code: 0
2023-08-08 18:49:37.-132|WARN|emoncms_mqtt.php|Not connected, retrying connection
2023-08-08 18:49:37.-37|WARN|emoncms_mqtt.php|Connecting to MQTT server: Connection Accepted.: code: 0
2023-08-08 19:02:27.-443|INFO|feedwriter.php|Starting feedwriter script
2023-08-08 19:02:27.-397|INFO|demandshaper_run.php|Starting demandshaper service
2023-08-08 19:02:27.-438|INFO|emoncms_mqtt.php|Starting MQTT Input script
2023-08-08 19:02:27.-263|INFO|feedwriter.php|Buffered feed writer daemon started with sleep 300s...
2023-08-08 19:02:27.-247|INFO|feed_model.php|EngineClass() Autoloaded new instance of 'RedisBuffer'.
2023-08-08 19:02:27.-169|WARN|emoncms_mqtt.php|Not connected, retrying connection
2023-08-08 19:02:27.-157|WARN|emoncms_mqtt.php|Connecting to MQTT server: Connection Accepted.: code: 0
2023-08-08 19:02:27.-157|INFO|emoncms_mqtt.php|Subscribed to: emon/# ID - 1
2023-08-08 19:02:27.-146|INFO|emoncms_mqtt.php|Callback subscribed to topic: 
2023-08-08 19:03:12.370|INFO|emoncms_mqtt.php|emon/esp/verwarming 20.062500
2023-08-08 19:03:12.433|INFO|emoncms_mqtt.php|emon/esp/woonkamer 20.562500
2023-08-08 19:03:17.403|INFO|emoncms_mqtt.php|emon/esp/uptime 182639.687500
2023-08-08 19:03:19.-412|INFO|rememberme_model.php|login
2023-08-08 19:03:19.-402|INFO|rememberme_model.php|getCookieValues: not present for: 192.168.178.201
2023-08-08 19:03:19.-147|INFO|emoncms_mqtt.php|emon/SolarEdge/Vermogen 0.0
2023-08-08 19:03:19.-132|INFO|emoncms_mqtt.php|emon/SolarEdge/Energie 50363.852
2023-08-08 19:03:19.-118|INFO|emoncms_mqtt.php|emon/SolarEdge/Vermogen P1 -0.0
2023-08-08 19:03:19.-104|INFO|emoncms_mqtt.php|emon/SolarEdge/Vermogen P2 -0.0
2023-08-08 19:03:19.-91|INFO|emoncms_mqtt.php|emon/SolarEdge/Vermogen P3 -0.0
2023-08-08 19:03:19.-77|INFO|emoncms_mqtt.php|emon/SolarEdge/current 0.0
2023-08-08 19:03:19.-64|INFO|emoncms_mqtt.php|emon/SolarEdge/l1_current 0.0

Rutger0000 · 8 August 2023 19:35

And all the logs at:

borpin · 9 August 2023 11:29

Yes that is the lowest, sorry

Potentially your issue. I have seen emoncms overwhelmed by too many topics too quickly. As Emoncms can only process an input at a minimum of 5s, then flooding it every second is pointless.

TrystanLea · 9 August 2023 22:48

Thankyou for that replication info above @Rutger0000 that’s useful to know. I wonder if you would get the same issue with emonSD-10Nov22 https://docs.openenergymonitor.org/emonsd/download.html? sounds like you might given that you tried Raspberry Pi OS Lite 2021-03-25

Could the issue be logs filling up or something like that?

Another possibility could be this 10ms delay in emoncms_mqtt perhaps the frequency of the MQTT data is causing the queue to stack up? https://github.com/emoncms/emoncms/blob/master/scripts/services/emoncms_mqtt/emoncms_mqtt.php#L212 You could try reducing it… Another issue might be that emoncms is not processing the messages fast enough beyond that delay specifically, there’s no rate limiting on the local emoncms so it might be overloading it …

Is a very high frequency and volume of MQTT data a common thread for others reporting this issue?

Ryan_S · 10 August 2023 02:37

In my case on my Proxmox CT of EmonCMS running on an x86 setup in 64 bit. I only have two OpenEVSE reporting to EmonCMS via MQTT. Those only send every 30 seconds and yet emoncms_mqtt was using up all RAM on the CT as reported. I’ll give the delay reduction a try though, just as soon as I get emoncms reinstalled.

That said, I would think, at least in my case, with a CT running on a Ryzen that there is more than enough “horsepower” behind processing the data coming in. Not sure if there is a built in limit of some sort on how quick it processes the data that might be lifted. Based on what you said and how I understood it, there is not.

TrystanLea · 10 August 2023 09:13

Hello @Ryan_S two OpenEVSE’s every 30 seconds should be no problem at all! It sounds like it should work for you so must be something else…

TrystanLea · 10 August 2023 09:18

The other thought is that there could be some latency either with redis or mysql that’s causing emoncms_mqtt to run slow? Il see if I can create a test script to push the limit on the post rate and see what happens…