Emoncms_mqtt.php very high ram use

Yes that might be the best option, my guess is that we’d need to roll back the php minor version but would need to test…

Thanks for taking a look. Will reinstall and give it a test run tonight if I get a chance. Was thinking it might be related to a package that was called out. Don’t think my Ryzen will care to much about a little more load, but hopefully this bug will get fixed and we can go back to your more efficient code set.

I ran into the same problem after upgrading my RPi 4 from buster to bullseye and upgrading to PHP8.2. After a few hours, my RPi4 was becoming unresponsive. I tracked the problem down to PHP that was taking huge amounts of memory…

Thanks for sharing your solution. It solved my problem. Looking forward for a fix in redis, though.

Have you had a chance to test?

[edit]
My EmonPi is on 8.1.15 and is OK.

pi@emonpi:~ $ sudo apt list --installed | grep php

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

libapache2-mod-php8.1/now 8.1.15-1+0~20230203.33+debian11~1.gbp878288 armhf [installed,upgradable to: 8.1.23-1+0~20230904.54+debian11~1.gbpe6c1a6]
php-common/now 2:93+0~20221211.45+debian11~1.gbpdb4dcc all [installed,upgradable to: 2:93+0~20230409.46+debian11~1.gbpdb4dcc]
php-pear/now 1:1.10.13+submodules+notgz+2022032202-2+0~20221209.38+debian11~1.gbpfd4c1d all [installed,upgradable to: 1:1.10.13+submodules+notgz+2022032202-2+0~20230612.39+debian11~1.gbpfd4c1d]
php-xml/now 2:8.2+93+0~20221211.45+debian11~1.gbpdb4dcc all [installed,upgradable to: 2:8.2+93+0~20230409.46+debian11~1.gbpdb4dcc]
php8.1-cli/now 8.1.15-1+0~20230203.33+debian11~1.gbp878288 armhf [installed,upgradable to: 8.1.23-1+0~20230904.54+debian11~1.gbpe6c1a6]
php8.1-common/now 8.1.15-1+0~20230203.33+debian11~1.gbp878288 armhf [installed,upgradable to: 8.1.23-1+0~20230904.54+debian11~1.gbpe6c1a6]
php8.1-curl/now 8.1.15-1+0~20230203.33+debian11~1.gbp878288 armhf [installed,upgradable to: 8.1.23-1+0~20230904.54+debian11~1.gbpe6c1a6]
php8.1-dev/now 8.1.15-1+0~20230203.33+debian11~1.gbp878288 armhf [installed,upgradable to: 8.1.23-1+0~20230904.54+debian11~1.gbpe6c1a6]
php8.1-gd/now 8.1.15-1+0~20230203.33+debian11~1.gbp878288 armhf [installed,upgradable to: 8.1.23-1+0~20230904.54+debian11~1.gbpe6c1a6]
php8.1-mbstring/now 8.1.15-1+0~20230203.33+debian11~1.gbp878288 armhf [installed,upgradable to: 8.1.23-1+0~20230904.54+debian11~1.gbpe6c1a6]
php8.1-mysql/now 8.1.15-1+0~20230203.33+debian11~1.gbp878288 armhf [installed,upgradable to: 8.1.23-1+0~20230904.54+debian11~1.gbpe6c1a6]
php8.1-opcache/now 8.1.15-1+0~20230203.33+debian11~1.gbp878288 armhf [installed,upgradable to: 8.1.23-1+0~20230904.54+debian11~1.gbpe6c1a6]
php8.1-readline/now 8.1.15-1+0~20230203.33+debian11~1.gbp878288 armhf [installed,upgradable to: 8.1.23-1+0~20230904.54+debian11~1.gbpe6c1a6]
php8.1-xml/now 8.1.15-1+0~20230203.33+debian11~1.gbp878288 armhf [installed,upgradable to: 8.1.23-1+0~20230904.54+debian11~1.gbpe6c1a6]
php8.1/now 8.1.15-1+0~20230203.33+debian11~1.gbp878288 all [installed,upgradable to: 8.1.23-1+0~20230904.54+debian11~1.gbpe6c1a6]
php8.2-common/now 8.2.2-1+0~20230203.13+debian11~1.gbp8ebd00 armhf [installed,upgradable to: 8.2.10-1+0~20230904.33+debian11~1.gbp2dc84c]
php8.2-xml/now 8.2.2-1+0~20230203.13+debian11~1.gbp8ebd00 armhf [installed,upgradable to: 8.2.10-1+0~20230904.33+debian11~1.gbp2dc84c]
pkg-php-tools/oldstable,now 1.40 all [installed,automatic]

Ubuntu 22.04 (Ithink) (OS: Linux 5.15.74-1-pve) - PHP Version: 8.1.2-1ubuntu2.13 (Zend Version 4.1.2)

On the emonpi, 8.1.23 and 8.2.10 is available so potentially an upgrade could break the system (if it is PHP)

pi@emonpi:~ $ sudo apt list --upgradable | grep php

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

libapache2-mod-php8.1/bullseye 8.1.23-1+0~20230904.54+debian11~1.gbpe6c1a6 armhf [upgradable from: 8.1.15-1+0~20230203.33+debian11~1.gbp878288]
php-common/bullseye 2:93+0~20230409.46+debian11~1.gbpdb4dcc all [upgradable from: 2:93+0~20221211.45+debian11~1.gbpdb4dcc]
php-pear/bullseye 1:1.10.13+submodules+notgz+2022032202-2+0~20230612.39+debian11~1.gbpfd4c1d all [upgradable from: 1:1.10.13+submodules+notgz+2022032202-2+0~20221209.38+debian11~1.gbpfd4c1d]
php-xml/bullseye 2:8.2+93+0~20230409.46+debian11~1.gbpdb4dcc all [upgradable from: 2:8.2+93+0~20221211.45+debian11~1.gbpdb4dcc]
php8.1-cli/bullseye 8.1.23-1+0~20230904.54+debian11~1.gbpe6c1a6 armhf [upgradable from: 8.1.15-1+0~20230203.33+debian11~1.gbp878288]
php8.1-common/bullseye 8.1.23-1+0~20230904.54+debian11~1.gbpe6c1a6 armhf [upgradable from: 8.1.15-1+0~20230203.33+debian11~1.gbp878288]
php8.1-curl/bullseye 8.1.23-1+0~20230904.54+debian11~1.gbpe6c1a6 armhf [upgradable from: 8.1.15-1+0~20230203.33+debian11~1.gbp878288]
php8.1-dev/bullseye 8.1.23-1+0~20230904.54+debian11~1.gbpe6c1a6 armhf [upgradable from: 8.1.15-1+0~20230203.33+debian11~1.gbp878288]
php8.1-gd/bullseye 8.1.23-1+0~20230904.54+debian11~1.gbpe6c1a6 armhf [upgradable from: 8.1.15-1+0~20230203.33+debian11~1.gbp878288]
php8.1-mbstring/bullseye 8.1.23-1+0~20230904.54+debian11~1.gbpe6c1a6 armhf [upgradable from: 8.1.15-1+0~20230203.33+debian11~1.gbp878288]
php8.1-mysql/bullseye 8.1.23-1+0~20230904.54+debian11~1.gbpe6c1a6 armhf [upgradable from: 8.1.15-1+0~20230203.33+debian11~1.gbp878288]
php8.1-opcache/bullseye 8.1.23-1+0~20230904.54+debian11~1.gbpe6c1a6 armhf [upgradable from: 8.1.15-1+0~20230203.33+debian11~1.gbp878288]
php8.1-readline/bullseye 8.1.23-1+0~20230904.54+debian11~1.gbpe6c1a6 armhf [upgradable from: 8.1.15-1+0~20230203.33+debian11~1.gbp878288]
php8.1-xml/bullseye 8.1.23-1+0~20230904.54+debian11~1.gbpe6c1a6 armhf [upgradable from: 8.1.15-1+0~20230203.33+debian11~1.gbp878288]
php8.1/bullseye 8.1.23-1+0~20230904.54+debian11~1.gbpe6c1a6 all [upgradable from: 8.1.15-1+0~20230203.33+debian11~1.gbp878288]
php8.2-common/bullseye 8.2.10-1+0~20230904.33+debian11~1.gbp2dc84c armhf [upgradable from: 8.2.2-1+0~20230203.13+debian11~1.gbp8ebd00]
php8.2-xml/bullseye 8.2.10-1+0~20230904.33+debian11~1.gbp2dc84c armhf [upgradable from: 8.2.2-1+0~20230203.13+debian11~1.gbp8ebd00]

@TrystanLea - can you try an emonSD base image and an upgrade and see what happens?

[edit2]
Looks like 8.3 is available in Bullseye. Perhaps skipping forward will solve it.

Another developer has replied on the issue I created on the phpredis repo and has tracked the issue down to a specific commit in the phpredis library and/or using redis 6.0 Redis pipeline memory leak? · Issue #2376 · phpredis/phpredis · GitHub, he has created a number of other related issues including Segmentation Fault while just calling `->exec()` · Issue #2390 · phpredis/phpredis · GitHub.

The interim solution might be to install redis 5.3.7

In trying to install php 8.3 before reading the above reply suggesting that the redis version was the issue. I seem to have broken my phpredis library. I cant install it, I get the error:

Configuring for:
PHP Api Version:         20230831
Zend Module Api No:      20230831
Zend Extension Api No:   420230831
cp: cannot stat 'shtool': No such file or directory
cp: cannot stat 'config.guess': No such file or directory
cp: cannot stat 'config.sub': No such file or directory
cp: cannot stat 'ltmain.sh': No such file or directory
chmod: cannot access '/home/pi/phpredis/build/shtool': No such file or directory
shtool at '/home/pi/phpredis/build/shtool' does not exist or is not executable.
Make sure that the file exists and is executable and then rerun this script.

and now Im struggling to recover my image from that position! argh :slight_smile:

If anyone else can try switching to redis 5.3.7 that would be much appreciated!

It must be a combination of the two as this combination is fine.

REDIS

Redis Server 6.0.16
PHP Redis 6.0.0-dev

PHP

Version 8.1.15 (Zend Version 4.1.15)

The PHPRedis developers have updated the issue to say that the issue is now fixed (September 23th) Redis pipeline memory leak? · Issue #2376 · phpredis/phpredis · GitHub

I’ve been testing a new image build for about 3 week and it seems to have been stable. Testing the 200 input every 1s script above works fine without memory build up so I think we can safely close this issue now.

1 Like