Emoncms_mqtt.php very high ram use

Emoncms
59

I built a new system with EmonScripts today and was able to replicate this issue quite clearly. I ultimately tracked the issue down to a piece of code in Modules/input/input_model.php, function redis_get_inputs.

    $pipe = $this->redis->multi(Redis::PIPELINE);
    foreach ($inputids as $id) {
        $pipe->hGetAll("input:$id");
    }
    $result = $pipe->exec();

There appears to be a memory leak within the latest version of the phpredis library when using either hGetAll or hMGet with php 8.1.23. I will create an issue on the phpredis repo to ask about this, I cant see any reports of the same issue elsewhere…

I’ve changed the implementation back to the less CPU efficient implementation that does not use a pipeline and that at least keeps memory stable.

The fix , hopefully temporary is available in a emoncms branch called ‘fix_emoncms_mqtt_memory_leak’

cd /var/www/emoncms
git checkout fix_emoncms_mqtt_memory_leak

CPU does appear to be 4x as high unfortunately, in my test of 50x inputs every second it’s the difference between about 5% of CPU and 20% for the emoncms_mqtt process.

2 Likes