The MQTT implementation in the development branch was only a proposed MQTT interfacers implementation, the basic framework is there as a PoC but there is no real processing of the data in either as there was never a proper discussion on the format we were aiming for. This is essentially an incomplete generic mqtt interfacer, but it is bi-directional. With mqtt emonhub doesn’t need to perform a “read” function, it just needs to maintain the mqtt connection and when a message comes in the “on_message” function is called (in this instance all it does is prints a log message), only when publishing a topic does the emonhub code need to call send, hence why it looks like a “send only” interfacer because the interfacers “read” function is redundant for mqtt.
But you are right there is no buffering, that is because at the time I was undecided about whether to add the buffering to the interfacers (doing away with reporters) or to retain the buffered output-only reporters.
Essentially yes, The interfacers have always been bi-directional, the JeeInterfacer has sent out the emonLCD time since day one and it has also configured the rfm2pi using serial output. The reporters have always had buffering and “happened” to also only be one-way.
When putting together the routing code for the experimental version I retained reporters (and included them in the routing) as we din not have any equivalent interfacers yet and my experimental version was compatible with existing emonhub installs with reporters (and will still need to be) so that we could introduce new interfacers that potentially replaced the reporters. However, whilst experimenting with reporter-like interfacers I discovered that there may well be a benefit to keeping the distinction between non-buffered 2 way and buffered one-way interfacers/reporters.
Yes! Although the priority here is that the connection between emonhub and emoncms is as efficient and reliable as possible. Using buffered data and confirmed bulk delivery over MQTT or HTTP(S).
I understand why buffered output may seem unnecessary when you are thinking about MQTT because you only picture emonhub and emoncms on the same machine. My view on the connection between emonhub and emoncms is that it could well be on the same machine but it could also be LAN or WAN connected, the physical location of the components doesn’t need to dictate the connections characteristics, if we make a connection that works regardless of the location of emonhub and emoncms relative to each other it will suit all cases including a local install.
In fact if you really wanted a connection that assumed emoncms and emonhub were on the same machine and that nothing would ever inturupt the connection between them, you could use a non-buffered, QoS1 per key topic (2 way local realtime “status” only) type interfacer.
But I fail to see why we would need to force a situation where a frame of data is split up and published on across several different topics individually only so emoncms can them subscribe to the base topic and process all those individual connections, rather than just passing the complete frame to emoncms in one go, MQTT is supposed to provide a light weight framework, which it fails to do it you are increasing the workload at both ends by forcing per key topics unnecessarily. The per key topic tree can be supplied separately (if required) by emonhub for local consumption.
Plus. making users understand why a local emoncms uses a different interfacer to a remote emoncms would be tricky and supporting 2 methods depending on the location of emoncms is too complex.
Always using a reporter (or reporter-like interfacer) when connecting to any and all emoncms instance(s), regardless of whether it’s mqtt or http(s) is straight forward, very clear and suits all scenarios with one solution.
This was a consideration and possibly even the favorite solution during initial development, but I thought that may be overly complicated and at that time I did not feel it was a decision that was needed at that point in time since I was retaining reporters for compatibility and the framework was more important than any individual interfacer implementations. Since then (over the last 2 years) it has become increasingly obvious they need to be distinctly different. Even if we did implement a “switch” setting I would concider making that a harcoded switch that was the only difference between a “mqtt-local” and “mqtt-buffered” interfacers so it was clear what the differences are, this ongoing discussion I think is testament to the need for that.
I think the best way forward would be to retain reporters (initially at least) to retain backwards compatibility with the original version. Then if the reporters are there, whilst still supporting the existing emon-pi mqtt and http interfacers we can transition emon-pi users to using the reporters (or all emonhub users to reporter-like interfacers) to provide a more streamlined buffered-bulk-confirmed-delivery to emoncms.
I will answer the questions about the “run” “add” and “send” methods in another post as I want to keep discussion about desired functionality separate to the coding, they are different discussions and I want to avoid the structure of the Python functions defining the desired functionality of emonhub, it needs to be the other way around.