Multi-instance service-runner

Emoncms
41

There is one instance of redis in the web view at emoncms/index.php and then instances of redis in each background process. The background processes: feedwriter, phpmqtt_input dont interact with service-runner. If they did interact a sequential switch between prefix contexts should still be fine.

grep -e "new Redis" -rnw emoncms
emoncms/Modules/packetgen/copyto_raspberrpi_module/raspberrypi_run.php:28:  $redis = new Redis();
emoncms/index.php:33:        $redis = new Redis();
emoncms/scripts/feedwriter.php:26:        $redis = new Redis();
emoncms/scripts/export_montly_cron.php:34:        $redis = new Redis();
emoncms/scripts/phpmqtt_input.php:77:        $redis = new Redis();
emoncms/scripts/input_queue_processor.php:28:        $redis = new Redis();
1 Like
42

I’ve just been doing some testing with blpop (the blocking list pop) and while it works perfectly to remove the need to continually poll redis, it will only actually work if we end up with a single service-runner key regardless of how many EmonCMS instances use that redis.

It’s a fairly simple change to the python code and it completely removes the polling required which has two benefits:

  1. drastically reduced CPU load (although it is pretty low now with the 5 times per second polling)
  2. Immediate execution of the first flag popped off the list. (subsequent flags pushed to the list wait for execution of the first to complete)

So if someone ever makes the required changes in the other parts of EmonCMS to force a single service-runner key to be used, we should definitely switch to using blpop in the service-runner.

2 Likes