Looks to be a nice step-up. A BIG 
Interesting feature of Discourse is that if you paste a link into the subject, it picks up the Title tag from the link, and adds in a link under the conversation subject 
.
Looks great, now that we have a build script (albeit alpha) once the new OS image is available we can test it relatively easily 
itâs already available as of today.
Interesting that RPF have decided to leap frog both the Debian and Raspbian releases and release an image based on Debian Buster which is the Debian âtestingâ branch not the âstableâ release. Normally we have to wait months while the changes trickle through from Debian to Raspbian and then on to the RPF Raspbian image. Debian is apparently launching Buster on July 7th
The new 2019-06-20-raspbian-buster(-lite) images will bring lots of changes in the dependencies we use for emoncms, eg php7.3, redis5.5 and mosquitto1.5.7. The big question here is whether the php extensions for redis and mosquitto will cope.
pi@raspberrypi:~ $ uname -a
Linux raspberrypi 4.19.50-v7+ #896 SMP Thu Jun 20 16:11:44 BST 2019 armv7l GNU/Linux
pi@raspberrypi:~ $ sudo apt-get install --dry-run apache2 redis-server mariadb-server php mosquitto
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
apache2-bin apache2-data apache2-utils galera-3 gawk libaio1 libapache2-mod-php7.3 libapr1 libaprutil1 libaprutil1-dbd-sqlite3 libaprutil1-ldap
libbrotli1 libcgi-fast-perl libcgi-pm-perl libconfig-inifiles-perl libdbd-mysql-perl libdbi-perl libencode-locale-perl libev4 libfcgi-perl
libhiredis0.14 libhtml-parser-perl libhtml-tagset-perl libhtml-template-perl libhttp-date-perl libhttp-message-perl libio-html-perl libjansson4
libjemalloc2 liblua5.1-0 liblua5.2-0 liblwp-mediatypes-perl libmariadb3 libreadline5 libsigsegv2 libsnappy1v5 libsodium23 libterm-readkey-perl
libtimedate-perl liburi-perl libuv1 libwebsockets8 lsof lua-bitop lua-cjson mariadb-client-10.3 mariadb-client-core-10.3 mariadb-common
mariadb-server-10.3 mariadb-server-core-10.3 mysql-common php-common php7.3 php7.3-cli php7.3-common php7.3-json php7.3-opcache php7.3-readline
redis-tools socat ssl-cert
Suggested packages:
apache2-doc apache2-suexec-pristine | apache2-suexec-custom www-browser gawk-doc php-pear libclone-perl libmldbm-perl libnet-daemon-perl
libsql-statement-perl libdata-dump-perl libipc-sharedcache-perl libwww-perl mailx mariadb-test tinyca apparmor ruby-redis openssl-blacklist
The following NEW packages will be installed:
apache2 apache2-bin apache2-data apache2-utils galera-3 gawk libaio1 libapache2-mod-php7.3 libapr1 libaprutil1 libaprutil1-dbd-sqlite3 libaprutil1-ldap
libbrotli1 libcgi-fast-perl libcgi-pm-perl libconfig-inifiles-perl libdbd-mysql-perl libdbi-perl libencode-locale-perl libev4 libfcgi-perl
libhiredis0.14 libhtml-parser-perl libhtml-tagset-perl libhtml-template-perl libhttp-date-perl libhttp-message-perl libio-html-perl libjansson4
libjemalloc2 liblua5.1-0 liblua5.2-0 liblwp-mediatypes-perl libmariadb3 libreadline5 libsigsegv2 libsnappy1v5 libsodium23 libterm-readkey-perl
libtimedate-perl liburi-perl libuv1 libwebsockets8 lsof lua-bitop lua-cjson mariadb-client-10.3 mariadb-client-core-10.3 mariadb-common mariadb-server
mariadb-server-10.3 mariadb-server-core-10.3 mosquitto mysql-common php php-common php7.3 php7.3-cli php7.3-common php7.3-json php7.3-opcache
php7.3-readline redis-server redis-tools socat ssl-cert
0 upgraded, 66 newly installed, 0 to remove and 0 not upgraded.
Inst libapr1 (1.6.5-1 Raspbian:testing [armhf])
Inst libaprutil1 (1.6.1-4 Raspbian:testing [armhf])
Inst libaprutil1-dbd-sqlite3 (1.6.1-4 Raspbian:testing [armhf])
Inst libaprutil1-ldap (1.6.1-4 Raspbian:testing [armhf])
Inst libbrotli1 (1.0.7-2 Raspbian:testing [armhf])
Inst libjansson4 (2.12-1 Raspbian:testing [armhf])
Inst liblua5.2-0 (5.2.4-1.1 Raspbian:testing [armhf])
Inst apache2-bin (2.4.38-3 Raspbian:testing [armhf])
Inst apache2-data (2.4.38-3 Raspbian:testing [all])
Inst apache2-utils (2.4.38-3 Raspbian:testing [armhf])
Inst apache2 (2.4.38-3 Raspbian:testing [armhf])
Inst libsigsegv2 (2.12-2 Raspbian:testing [armhf])
Conf libsigsegv2 (2.12-2 Raspbian:testing [armhf])
Inst gawk (1:4.2.1+dfsg-1 Raspbian:testing [armhf])
Inst mysql-common (5.8+1.0.5 Raspbian:testing [all])
Inst mariadb-common (1:10.3.15-1 Raspbian:testing [all])
Inst galera-3 (25.3.25-2 Raspbian:testing [armhf])
Inst libdbi-perl (1.642-1+b1 Raspbian:testing [armhf])
Inst lsof (4.91+dfsg-1 Raspbian:testing [armhf])
Inst libconfig-inifiles-perl (3.000001-1 Raspbian:testing [all])
Inst libaio1 (0.3.112-3 Raspbian:testing [armhf])
Inst libreadline5 (5.2+dfsg-3 Raspbian:testing [armhf])
Inst libsnappy1v5 (1.1.7-1 Raspbian:testing [armhf])
Inst mariadb-client-core-10.3 (1:10.3.15-1 Raspbian:testing [armhf])
Inst mariadb-client-10.3 (1:10.3.15-1 Raspbian:testing [armhf])
Inst mariadb-server-core-10.3 (1:10.3.15-1 Raspbian:testing [armhf])
Inst socat (1.7.3.2-2 Raspbian:testing [armhf])
Conf mysql-common (5.8+1.0.5 Raspbian:testing [all])
Conf mariadb-common (1:10.3.15-1 Raspbian:testing [all])
Inst mariadb-server-10.3 (1:10.3.15-1 Raspbian:testing [armhf])
Inst libhiredis0.14 (0.14.0-3 Raspbian:testing [armhf])
Inst libjemalloc2 (5.1.0-3 Raspbian:testing [armhf])
Inst liblua5.1-0 (5.1.5-8.1 Raspbian:testing [armhf])
Inst lua-bitop (1.0.2-5 Raspbian:testing [armhf])
Inst lua-cjson (2.1.0+dfsg-2.1 Raspbian:testing [armhf])
Inst redis-tools (5:5.0.3-4 Raspbian:testing [armhf])
Inst redis-server (5:5.0.3-4 Raspbian:testing [armhf])
Inst php-common (2:69 Raspbian:testing [all])
Inst php7.3-common (7.3.4-2 Raspbian:testing [armhf])
Inst php7.3-json (7.3.4-2 Raspbian:testing [armhf])
Inst php7.3-opcache (7.3.4-2 Raspbian:testing [armhf])
Inst php7.3-readline (7.3.4-2 Raspbian:testing [armhf])
Inst libsodium23 (1.0.17-1 Raspbian:testing [armhf])
Inst php7.3-cli (7.3.4-2 Raspbian:testing [armhf])
Inst libapache2-mod-php7.3 (7.3.4-2 Raspbian:testing [armhf])
Inst libhtml-tagset-perl (3.20-3 Raspbian:testing [all])
Inst liburi-perl (1.76-1 Raspbian:testing [all])
Inst libhtml-parser-perl (3.72-3+b2 Raspbian:testing [armhf])
Inst libcgi-pm-perl (4.40-1 Raspbian:testing [all])
Inst libfcgi-perl (0.78-2+b2 Raspbian:testing [armhf])
Inst libcgi-fast-perl (1:2.13-1 Raspbian:testing [all])
Inst libmariadb3 (1:10.3.15-1 Raspbian:testing [armhf])
Inst libdbd-mysql-perl (4.050-2 Raspbian:testing [armhf])
Inst libencode-locale-perl (1.05-1 Raspbian:testing [all])
Inst libev4 (1:4.25-1 Raspbian:testing [armhf])
Inst libhtml-template-perl (2.97-1 Raspbian:testing [all])
Inst libtimedate-perl (2.3000-2 Raspbian:testing [all])
Inst libhttp-date-perl (6.02-1 Raspbian:testing [all])
Inst libio-html-perl (1.001-1 Raspbian:testing [all])
Inst liblwp-mediatypes-perl (6.02-1 Raspbian:testing [all])
Inst libhttp-message-perl (6.18-1 Raspbian:testing [all])
Inst libterm-readkey-perl (2.38-1 Raspbian:testing [armhf])
Inst libuv1 (1.24.1-1 Raspbian:testing [armhf])
Inst libwebsockets8 (2.0.3-3 Raspbian:testing [armhf])
Inst mariadb-server (1:10.3.15-1 Raspbian:testing [all])
Inst mosquitto (1.5.7-1 Raspbian:testing [armhf])
Inst php7.3 (7.3.4-2 Raspbian:testing [all])
Inst php (2:7.3+69 Raspbian:testing [all])
Inst ssl-cert (1.0.39 Raspbian:testing [all])
Conf libapr1 (1.6.5-1 Raspbian:testing [armhf])
Conf libaprutil1 (1.6.1-4 Raspbian:testing [armhf])
Conf libaprutil1-dbd-sqlite3 (1.6.1-4 Raspbian:testing [armhf])
Conf libaprutil1-ldap (1.6.1-4 Raspbian:testing [armhf])
Conf libbrotli1 (1.0.7-2 Raspbian:testing [armhf])
Conf libjansson4 (2.12-1 Raspbian:testing [armhf])
Conf liblua5.2-0 (5.2.4-1.1 Raspbian:testing [armhf])
Conf apache2-bin (2.4.38-3 Raspbian:testing [armhf])
Conf apache2-data (2.4.38-3 Raspbian:testing [all])
Conf apache2-utils (2.4.38-3 Raspbian:testing [armhf])
Conf apache2 (2.4.38-3 Raspbian:testing [armhf])
Conf gawk (1:4.2.1+dfsg-1 Raspbian:testing [armhf])
Conf galera-3 (25.3.25-2 Raspbian:testing [armhf])
Conf libdbi-perl (1.642-1+b1 Raspbian:testing [armhf])
Conf lsof (4.91+dfsg-1 Raspbian:testing [armhf])
Conf libconfig-inifiles-perl (3.000001-1 Raspbian:testing [all])
Conf libaio1 (0.3.112-3 Raspbian:testing [armhf])
Conf libreadline5 (5.2+dfsg-3 Raspbian:testing [armhf])
Conf libsnappy1v5 (1.1.7-1 Raspbian:testing [armhf])
Conf mariadb-client-core-10.3 (1:10.3.15-1 Raspbian:testing [armhf])
Conf mariadb-client-10.3 (1:10.3.15-1 Raspbian:testing [armhf])
Conf mariadb-server-core-10.3 (1:10.3.15-1 Raspbian:testing [armhf])
Conf socat (1.7.3.2-2 Raspbian:testing [armhf])
Conf mariadb-server-10.3 (1:10.3.15-1 Raspbian:testing [armhf])
Conf libhiredis0.14 (0.14.0-3 Raspbian:testing [armhf])
Conf libjemalloc2 (5.1.0-3 Raspbian:testing [armhf])
Conf liblua5.1-0 (5.1.5-8.1 Raspbian:testing [armhf])
Conf lua-bitop (1.0.2-5 Raspbian:testing [armhf])
Conf lua-cjson (2.1.0+dfsg-2.1 Raspbian:testing [armhf])
Conf redis-tools (5:5.0.3-4 Raspbian:testing [armhf])
Conf redis-server (5:5.0.3-4 Raspbian:testing [armhf])
Conf php-common (2:69 Raspbian:testing [all])
Conf php7.3-common (7.3.4-2 Raspbian:testing [armhf])
Conf php7.3-json (7.3.4-2 Raspbian:testing [armhf])
Conf php7.3-opcache (7.3.4-2 Raspbian:testing [armhf])
Conf php7.3-readline (7.3.4-2 Raspbian:testing [armhf])
Conf libsodium23 (1.0.17-1 Raspbian:testing [armhf])
Conf php7.3-cli (7.3.4-2 Raspbian:testing [armhf])
Conf libapache2-mod-php7.3 (7.3.4-2 Raspbian:testing [armhf])
Conf libhtml-tagset-perl (3.20-3 Raspbian:testing [all])
Conf liburi-perl (1.76-1 Raspbian:testing [all])
Conf libhtml-parser-perl (3.72-3+b2 Raspbian:testing [armhf])
Conf libcgi-pm-perl (4.40-1 Raspbian:testing [all])
Conf libfcgi-perl (0.78-2+b2 Raspbian:testing [armhf])
Conf libcgi-fast-perl (1:2.13-1 Raspbian:testing [all])
Conf libmariadb3 (1:10.3.15-1 Raspbian:testing [armhf])
Conf libdbd-mysql-perl (4.050-2 Raspbian:testing [armhf])
Conf libencode-locale-perl (1.05-1 Raspbian:testing [all])
Conf libev4 (1:4.25-1 Raspbian:testing [armhf])
Conf libhtml-template-perl (2.97-1 Raspbian:testing [all])
Conf libtimedate-perl (2.3000-2 Raspbian:testing [all])
Conf libhttp-date-perl (6.02-1 Raspbian:testing [all])
Conf libio-html-perl (1.001-1 Raspbian:testing [all])
Conf liblwp-mediatypes-perl (6.02-1 Raspbian:testing [all])
Conf libhttp-message-perl (6.18-1 Raspbian:testing [all])
Conf libterm-readkey-perl (2.38-1 Raspbian:testing [armhf])
Conf libuv1 (1.24.1-1 Raspbian:testing [armhf])
Conf libwebsockets8 (2.0.3-3 Raspbian:testing [armhf])
Conf mariadb-server (1:10.3.15-1 Raspbian:testing [all])
Conf mosquitto (1.5.7-1 Raspbian:testing [armhf])
Conf php7.3 (7.3.4-2 Raspbian:testing [all])
Conf php (2:7.3+69 Raspbian:testing [all])
Conf ssl-cert (1.0.39 Raspbian:testing [all])
As I raised elsewhere, I donât think the mosquitto one will support PHP7.3. That extension seems not be receiving any maintenance and there is no obvious alternative (that I have found).
indeed, hence the âbig questionâ!
MQTT in general doesnât seem well supported for PHP, IMO MQTT in emoncms is way more trouble than itâs worth. The Python MQTT (paho) will always be bang up to date for Mosquito as they are both by Eclipse, no reliance on 3rd parties. I think either MQTT in emoncms should be dropped and develop emonhub further to be the MQTT front end for emoncms OR just write a Python service to replace the PHP service, but the latter means you have 2 Python services doing the same thing. I have toyed with the idea of a redis interfacer for emonhub so that it can write directly to emoncms input queueâs somehow, using emonhub you could have as many MQTT interfacers set up as you liked, so subscribing to some obscure IoT devices MQTT topics would easy, rather than forcing all devices to publish to emoncmsâs chosen base topic.
Perhaps a new thread? I do not understand the inner workings of the message/data handling between emonhub and emoncms. I do suspect that a python script to provide an interface (in and out) between emoncms and an MQTT broker is going to be required. Whether that is via emonhub or not I could not say.
However, for now, I suspect âBusterâ will have to remain unsupported.
[edit]
Any reason not to pass a JSON string via Redis?
Iâm making a lot of use of MQTT as part of the demandshaper module so I would be keen for a PHP solution to this and Im not convinced we should throw it out more generally. Il make a note to investigate with PHP7.3 . Has anyone tested the latest Mosquitto-PHP with 7.3? looks like 7 might now be supported? GitHub - mgdm/Mosquitto-PHP: A wrapper for the Eclipse Mosquitto⢠MQTT client library for PHP.
Looks like there has been a small amount of movement in April; first for 2 years. I was sure Iâd read some issues about it and 7.3 but perhaps I am wrong.
Comment about the RPi 4 itself and their manufacturing strategy: https://eandt.theiet.org/content/articles/2019/06/raspberry-pi-4-unveiled-interview-with-founder-eben-upton
There were issues with Mosquitto 1.6.2
Feed from one input stops updating, others feeds continue to work
due to the older images using the Mosquitto repos rather than Debianâs, we are currently using 1.4.10, Buster uses 1.5.7, i seem to recall that when the raspbian repo was at 1.5.? users (Brian?) were rolling back to the 1.4.10 due to some other errors, where between 1.4.10 and 1.6.2 things fall over i donât know.
Yeah, initial php7 support came in at the end of 2016, little has happened since then, only 2 commits since the end of 2017!
A couple of those links above do use 7.3, but the issues were with the Mosquitto version not so much the PHP version AFAIK.
The extension hasnât been updated since March 2017 on PECL, php 7.3 didnât release until 2018 so if it is compatible itâs more chance than anything.
I wasnât suggesting dropping MQTT support, just changing the way itâs done. You maybe flogging a dead horse with support for MQTT in PHP, even if PHP7.3 does work with Mosquitto 1.5.7, what happens when we reach Mosquitto 1.6.2 (already known to be an issue)? or php7.4, scheduled to be released on November 21, 2019?
Yes Iâm with you on that. MQTT publishing already uses Redis (IIRC). Why not do the same both ways via a python script and use JSON as the message format? Just a thought.
The issue is that emoncms has no way to clear the queue once that python script puts the mqtt message there. So you need a emoncms_input.php service to process that redis queue so rather than
emonhub > mosquitto-broker > emoncms-python-service > redis > emoncms-php-input-service > emoncms
it would make more sense to go
emonhub > redis > emoncms-php-input-service > emoncms
and use emonhub for all the MQTT subscribing (and publishing)
Note the Ethernet port has swapped sides so you will need a new EmonPi faceplate.
Interesting reading. Especially the line about the lack of stabilty at 7 nm.
Il make a note that the MQTT support might need reviewing. This kind of thing is always a potential issue with any dependency. When I get a chance I will test v1.6.2 and see if I can replicate the issues seen.
Long story short: Downloading EmonSD and using it on rpi4, will it work? Does fully updating it, allow to use emoncms and others(mosquito, redis, etc) work ok, or does need some work? Sorry to ask but after reading all the posts, i got more confused.
I think the answer is no. EmonSD Oct18 is based on raspbian stretch, a new image will be needed based on debian buster. If you have a pi4 at hand you could try the build script and let us know if it works? EmonSD build script progress update and beta release I havenât personally tried it yet with the pi4.
Weâve got a Pi4 here in our hands! Weâre currently testing building our image on Buster which is still in a beta version for another week or so.
The Pi4 is not available to purchase in more than one-offs until Sep 2019, therefore we are unable to order qtys for reselling until then. Not a big issue since the Pi3B+ is still more and capable of running Emoncms. We will do some testing to ensure when we do eventually transition to Pi4 itâs a smooth stable transition.