Yes and no, it is possible to do that, but that puts a lot of strain on emoncms and the mosquitto broker as each “publish to mqtt” process is a new connection and disconnection per value, it was never a “good” solution but a easy fix to provide “some” form of MQTT output.
I agree, but that small group does seem to be the emonpi/emonsd target audience.
Agreed, that was what I was getting at with
As above, it’s not ideal to use the “publish to mqtt” process, that was really implemented just as a way to publish calculated data eg “energy use” which might be calculated from solar and grid so unavailable unless sourced from emoncms, not for publishing every input for other services to find which is why we ended up with what we have.
Indeed, in fact I would even say any MQTT will fall short of reliable via the MQTT input due to the connection issues and the lack of a session id.
The http bulk api from emonhub ensures all data arrives safely and that it is timestamped as well as possible (rfm transmission delays still cause timestamping errors even though the emontx/emonTH interval is fixed).
I’m not sure that is entirely the case, both have strengths and weaknesses, certainly the way they are implemented in emoncms http wins hands down on all fronts, but mqtt has the potential to be better for us as is is better suited to frequent and small payloads eg data values that can be shared, perfect for most IoT projects as it has a much smaller overhead and can easily share the data with multiple softwares etc.
I have been eager to use MQTT since I/we first found out about it, but I have not implemented any MQTT yet as I find the current offering somewhat disappointing and a bit of a step backwards in reliability (and accuracy), I want to embrace mqtt because it’s a step forward with more positives than negatives, not just have MQTT because it’s MQTT.
That could be a plus, but it isn’t a big gain because any form of buffer must be on the data source’s end of a potential outage/disconnection/disruption etc, so for the MQTT buffering to help it must have an uninterruptible connection with the source to buffer data to send onwards at a later time, so (exaggerated for demonstration) every device needs to run a local MQTT broker and every target needs to then subscribe to each remote broker, or have a network of brokers, which can undermine the simplicity of the MQTT model.
eg if emoncms.org was also an MQTT broker, it would be of little use buffering data from your local devices once they have reached the server, it needs to buffer locally so that during a network outage the data is buffered at the device end until the network is up.
It would benefit emoncms.org input queues and also if there was data coming out of emoncms that a remote device might miss due to a temporary interruption, so there benefits but that is not a key benefit to our current models. There should be no good reason to need to buffer MQTT on an emonPi where the emoncms server and MQTT broker are both located on one machine, except arguably during updates etc. It also duplicates a part of emonhub’s intended role, but only for MQTT, not for other services eg HTTP.
emoncms is a reaction led software, it reacts to new data arrival and it reacts to requests for data or webpages etc. emonhub is an active service that is constantly looping through time, it has the mechanisms to constantly monitor several MQTT topics (and many other things) and to prepare, buffer and deliver data to emoncms (via mqtt or http or both or perhaps even some other way eg direct php calls). It could also handle getting data from emoncms (via http calls or “publish to mqtt” processes etc) and distribute that data via MQTT (or other mediums) so that emoncms doesn’t need to get bogged down with multiple connections, MQTT or otherwise, the desire for a direct MQTT implementation in emoncms is hampering any advances using MQTT IMO.
sorry for the disappearing act @borpin - I haven’t had a reoccurrence of the problem yet, but my setup has been up and down a bit due to having work done at home… I’ll keep monitoring and will report back in a week or two - thanks for asking!