I have the emonPi Solar PV bundle monitoring my energy generation and usage.
I want to feed the twice-per-second Modbus-RTU power readings (sniffing the PV inverter’s closed loop control meter) into the emonPi via MQTT. I have node-Red on a separate Raspberry Pi (which I call nrPi) grabbing that reading, and feeding into MQTT, along with a bunch of once-per-10s readings sampled from the inverter’s WiFi interface. The MQTT feed is then picked up on the emonPi for input to emoncms.
The slower readings are working reliably; but this fast one, created as a PHPFINA 5s window feed, is missing about 30% of values, even though I would have thought at least one of the ten samples originated in a 5s window should have reached the PHPFINA feed.
Is there a debug tool for MQTT where I can read what is leaving the nrPi, and/or arriving at the emonPi. I’ve seen the emonhub log for the Arduino shield in the emonpi - I’m looking for something like that in MQTT land. Does it exist, and could I use it for this?
Additionally, is it reasonable to batter the PHPFINA tool so fast as 10x the feed rate? Would I do better to run some kind of averaging/decimation on the high rate samples before handing them over to emonhub?
Hi,
I use MQTT Explorer on Windows. You can monitor the MQTT broker with it.
I have been made aware on this forum in following post Node Red MQTT formatting thread, that it is wiser to send MQTT data in one big packet as many smaller ones.
I use at least 3 sources á 8-10 values with 10 seconds updates efficiently since.
Maybe it helps also in your case.
Can you run mosquitto_sub on the nrPi to see what messages the MQTT server is sending?
To monitor MQTT network traffic between the nrPi and emonPi you could possibly use tcpdump, restricted to capturing packets on port 1883 (IIRC). It’s a bit low-level, but often works well if you want to grub about in the details of the traffic.
The receiving process will not cope with that frequency. I have seen this previously with the Emoncms MQTT system. It is easily overloaded. It is better to send one MQTT message with lots of inputs at once (JSON), than separate MQTT messages for each input.
If you want it, I suggest you buffer the values and send as a single HTTP message. Even then, if the EmonCMS input is only logging every 5S what is the point of sending it twice a second?
Thank you @bekesizl , @JasF and @borpin Clearly I’m abusing the tools, and will adjust my targets.
The original motive in sampling power flows at 500ms rather than 5s interval was to observe the inverter’s transient response on an emonPi dashboard. I have already done that in a node-red dashboard, so I’ll probably just stop there.
I’ve also now successfully tried MQTT Explorer, which was helpful - I think my main missing samples issue is in the node-red flow on nrPi, as this tool saw no more samples than the emonPi.
I assume from the lack of response on any MQTT receive log on emoncms (cf the emonhub Arduino log) that there is no such tool, and I just need to look at the inputs / feeds directly. Is that a fair conclusion?
If you have set emoncms log-level to debug, you should see messages stating the MQTT messages are being received.
You can also configure the Mosquitto Broker on the EmonPi to log, and then you can view those messages as well. Using this I was able to see that the Broker was receiving/publishing messages but that the emoncms MQTT script was not processing all messages (low QOS).