Post Process stuck in Queue

Thomas_Adeyinka · 2023-06-26T10:55:24.773Z

Hello all, I am trying to retro-actively generate kWh logs from power feed. I selected power to kWh and chose the input feed and selected the postporcess name. I selected from start and clicked start.

The response I got was “added to queue” It has been in queue every since, over 5 days from now. I have added other processes and same thing. (See screenshot)

What could I be doing wrong?

post-processor2858×1278 200 KB

Server Information

Services

emoncms_mqtt :- Activating loaded activating auto-restart
feedwriter :- Activating loaded activating auto-restart
service-runner :- Active Running
redis-server :- Active Running
emonhub :- Not found or not installed
emonPiLCD :- Not found or not installed
mosquitto :- Not found or not installed
demandshaper :- Not found or not installed

Emoncms

Version :- 11.3.21
Git :-
- URL :- GitHub - emoncms/emoncms: Web-app for processing, logging and visualising energy, temperature and other environmental data
- Branch :- * stable
- Describe :- 11.0.5-151-gc726cf01
Components :- Emoncms Core v11.3.21 | App v2.7.2 | Backup v2.3.2 | Dashboard v2.3.3 | Device v2.2.2 | Graph v2.2.3 | Postprocess v2.4.6 | EmonScripts v1.3.9

Server

Machine :- QEMU Compute Instance/Standard PC (Q35 + ICH9, 2009), BIOS
CPU :- 1 Threads(s) | 1 Core(s) | 1 Sockets(s) | AMD EPYC 7601 32-Core Processor | 2199.998MHz | 4399.99MIPS |
OS :- Linux 4.19.0-24-amd64
Host :- localhost | localhost | (172.104.130.37)
Date :- 2023-06-26 11:53:23 WAT
Uptime :- 11:53:23 up 4 days, 1:34, 1 user, load average: 0.02, 0.05, 0.04

Memory

RAM :- Used: 58.53%
- Total :- 987.32 MB
- Used :- 577.86 MB
- Free :- 409.46 MB
Swap :- Used: 80.98%
- Total :- 512 MB
- Used :- 414.64 MB
- Free :- 97.36 MB

Disk

**** :- - / :- Used: 57.42%
- Total :- 24.1 GB
- Used :- 13.84 GB
- Free :- 9.02 GB
- Read Load :- 383.99 B/s
- Write Load :- 2.82 KB/s
- Load Time :- 25 days 7 hours 13 mins

HTTP

Server :- Apache/2.4.38 (Debian) HTTP/1.0 CGI/1.1 80

MySQL

Version :- 10.5.19-MariaDB-1:10.5.19+maria~deb10
Host :- localhost (127.0.0.1)
Date :- 2023-06-26 11:53:23 (UTC 01:00‌)
Stats :- Uptime: 351276 Threads: 9 Questions: 803079 Slow queries: 0 Opens: 30 Open tables: 23 Queries per second avg: 2.286

Redis

Version :-
- Redis Server :- 5.0.14
- PHP Redis :- 5.3.7
Host :- localhost:6379
Size :- 408 keys (925.00K)
Uptime :- 4 days

PHP

Version :- 8.0.29 (Zend Version 4.0.29)
Run user :- User: www-data Group: www-data Script Owner: www-data
Modules :- apache2handler calendar Core ctype curl date dom v20031129exif FFI fileinfo filter ftp gettext hash iconv igbinary v3.2.14json libxml mbstring mysqli mysqlnd vmysqlnd 8.0.29openssl pcre PDO pdo_mysql Phar posix readline redis v5.3.7Reflection session shmop SimpleXML sockets sodium SPL standard sysvmsg sysvsem sysvshm tokenizer xml xmlreader xmlwriter xsl Zend OPcache zlib

Client Information

HTTP

Browser :- Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/114.0
Language :- en-US,en;q=0.5

Window

Size :- 1728 x 944

Screen

Resolution :- 1728 x 1117

borpin · 2023-06-26T13:20:17.773Z

That doesn’t look good.

Thomas_Adeyinka · 2023-06-26T19:03:35.394Z

Thanks for the pointer.
I decided to start with the emoncms_mqtt script, discovered I did not have mqtt module for php installed but after installing it, I still could not get this to work.

I get this from systemctl


emoncms_mqtt.service - Emoncms emoncms_mqtt script
   Loaded: loaded (/var/www/emoncms/scripts/services/emoncms_mqtt/emoncms_mqtt.service; enabled; vendor preset: enabled)
   Active: inactive (dead) since Mon 2023-06-26 20:01:02 WAT; 31s ago
     Docs: https://github.com/emoncms/emoncms/blob/master/docs/RaspberryPi/MQTT.md
  Process: 23756 ExecStartPre=/bin/mkdir -p ${LOG_PATH} (code=exited, status=0/SUCCESS)
  Process: 23757 ExecStartPre=/bin/chown ${USER} ${LOG_PATH} (code=exited, status=0/SUCCESS)
  Process: 23758 ExecStartPre=/bin/touch ${LOG_PATH}/emoncms-mqtt.log (code=exited, status=0/SUCCESS)
  Process: 23759 ExecStartPre=/bin/chmod 666 ${LOG_PATH}/emoncms-mqtt.log (code=exited, status=0/SUCCESS)
  Process: 23760 ExecStart=/usr/bin/php /var/www/emoncms/scripts/services/emoncms_mqtt/emoncms_mqtt.php (code=exited, status=0/SUCCESS)
 Main PID: 23760 (code=exited, status=0/SUCCESS)
Jun 26 20:01:02 localhost systemd[1]: Started Emoncms emoncms_mqtt script.
Jun 26 20:01:02 localhost emoncms_mqtt[23760]: PHP Notice:  Undefined index: phpfiwa in /var/www/emoncms/process_settings.php on line 83
Jun 26 20:01:02 localhost emoncms_mqtt[23760]: PHP Notice:  Trying to access array offset on value of type null in /var/www/emoncms/process_settings.php on line 85

Thomas_Adeyinka · 2023-06-26T19:05:27.463Z

I had to downgrade php to 7.4 since mqtt php module would not build on php8.0. There is very little information on how to setip emoncms-mqtt module

borpin · 2023-06-26T20:37:11.059Z

Are you using emonscripts to build this system? I’m guessing not. That is the starting point for your issues. It all works fine on PHP8.x when built with the scripts.

Thomas_Adeyinka · 2023-06-26T20:55:44.228Z

It turned out to be a case of fixing the chown user in the script settings from “pi” and enabling the service in settings.ini. Both services are running fine now

Server Information

Services

emoncms_mqtt :- Active Running
feedwriter :- Active Running - sleep 60s 0 feed points pending write
service-runner :- Active Running
redis-server :- Active Running
emonhub :- Not found or not installed
emonPiLCD :- Not found or not installed
mosquitto :- Not found or not installed
demandshaper :- Not found or not installed

Emoncms

Version :- low-write 11.3.21
Git :-
- URL :- GitHub - emoncms/emoncms: Web-app for processing, logging and visualising energy, temperature and other environmental data
- Branch :- * stable
- Describe :- 11.0.5-151-gc726cf01
Components :- Emoncms Core v11.3.21 | App v2.7.2 | Backup v2.3.2 | Dashboard v2.3.3 | Device v2.2.2 | Graph v2.2.3 | Postprocess v2.4.6 | EmonScripts v1.3.9

Server

Machine :- QEMU Compute Instance/Standard PC (Q35 + ICH9, 2009), BIOS
CPU :- 1 Threads(s) | 1 Core(s) | 1 Sockets(s) | AMD EPYC 7601 32-Core Processor | 2199.998MHz | 4399.99MIPS |
OS :- Linux 4.19.0-24-amd64
Host :- localhost | localhost | (172.104.130.37)
Date :- 2023-06-26 21:55:08 WAT
Uptime :- 21:55:08 up 4 days, 11:36, 2 users, load average: 0.04, 0.03, 0.00

Memory

RAM :- Used: 58.03%
- Total :- 987.32 MB
- Used :- 572.94 MB
- Free :- 414.38 MB
Swap :- Used: 87.50%
- Total :- 512 MB
- Used :- 448.01 MB
- Free :- 63.99 MB

Disk

**** :- - / :- Used: 56.95%
- Total :- 24.1 GB
- Used :- 13.73 GB
- Free :- 9.14 GB
- Read Load :- 577.87 B/s
- Write Load :- 3.2 KB/s
- Load Time :- 25 days 17 hours 15 mins

HTTP

Server :- Apache/2.4.38 (Debian) HTTP/1.0 CGI/1.1 80

MySQL

Version :- 5.5.5-10.5.19-MariaDB-1:10.5.19+maria~deb10
Host :- localhost (127.0.0.1)
Date :- 2023-06-26 21:55:08 (UTC 01:00‌)
Stats :- Uptime: 387381 Threads: 6 Questions: 964134 Slow queries: 0 Opens: 31 Open tables: 24 Queries per second avg: 2.488

Redis

Version :-
- Redis Server :- 5.0.14
- PHP Redis :- 5.3.7
Host :- localhost:6379
Size :- 446 keys (970.47K)
Uptime :- 4 days

MQTT Server

Version :- Mosquitto ?
Host :- localhost:1883 (127.0.0.1)

PHP

Version :- 7.4.33 (Zend Version 3.4.0)
Run user :- User: www-data Group: www-data Script Owner: www-data
Modules :- apache2handler calendar Core ctype curl date dom v20031129exif FFI fileinfo filter ftp gettext hash iconv igbinary v3.2.14json libxml mbstring mosquitto v0.4.0mysqli mysqlnd vmysqlnd 7.4.33openssl pcre PDO pdo_mysql Phar posix readline redis v5.3.7Reflection session shmop SimpleXML sockets sodium SPL standard sysvmsg sysvsem sysvshm tokenizer xml xmlreader xmlwriter xsl Zend OPcache zlib

Client Information

HTTP

Browser :- Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/114.0
Language :- en-US,en;q=0.5

Window

Size :- 1728 x 943

Screen

Resolution :- 1728 x 1117

Thomas_Adeyinka · 2023-06-26T20:59:31.871Z

Original problem with post process module being all stock in queue persist though.

Thomas_Adeyinka · 2023-06-26T21:23:22.444Z

Manually running

sudo php postprocess_run.php

in console solved the problem and the process were calc in less than 20s. I would be marking this as solved.

borpin · 2023-06-27T06:03:06.527Z

Thomas_Adeyinka:

fixing the chown user in the script settings from “pi”

Which script?