Emonhub- Edit Mosquitto update times

drc · 2020-06-27T12:53:52.828Z

Hi All, I’ve nearly perfected my integration and as part of it I am hoping to take readings from power1 CT clamp over MQTT for further processing in a remote server.

I am able to remotely access the values but it seems the power1 feed over MQTT updates only every 5 seconds, I was hoping to reduce that to one second. I am not writing this data to an SD card so I’m not too worried about corruption!

I’ve looked into the Mosquitto settings and I can’t find information about rate throttling, is the MQTT rate limited due to the readings on the emonpi sensor, in a “if the sensor doesn’t provide an update don’t issue a new MQTT message” way? Is there any way for me to get a more responsive message over MQTT, even if it’s the same value? I’ve checked all of the board and I couldnt find anything on it! Any help would be greatly appreciated!

EDIT: it seems the json feed has similar update rates, is there any way to reduce either update times?

drc · 2020-06-28T15:34:51.781Z

EDIT 2: It looks like the ATmega firmware would need a recompile to increase the frequency of reporting from the CT clamps, can anyone comment on if this is a big task?

Robert.Wall · 2020-06-28T16:38:54.345Z

Which firmware exactly are you using, and where does it reside? Are you implying the “emon” front end of an emonPi, or something else - because emonHub also handles data incoming from emonTx’s as well.

drc · 2020-06-28T17:21:33.635Z

Hi Robert,

Thanks for taking the time to reply. After spending some time to get to grips with how the system works a bit better, it seems what I would like to do is increase the rate that the ATmega in the emon pi sends sensor updates to the emonHub over its communication channel.

I would assume that the bandwidth on the serial interface is able to handle the increased packet rate and something needs to be edited on the ATmega firmware to make the increase in refresh speed a possibility. I won’t be writing data to the emoncms database, I simply want to tap off the more frequent updates for my own use through either JSON or MQTT.

I was hoping to create a software power control box for my commodity hardware inverter so I could have more control flexibility, but an update rate of 5 seconds means that I wont have the speed of control that I need.

The information displayed on the following page suggests that the emonPi will only update every five seconds, this is the rate that I would like to increase:

Guide | OpenEnergyMonitor

2. Log Locally

Log data locally to emonPi

Any suggestion how would be greatly appreciated!

Robert.Wall · 2020-06-28T18:41:45.464Z

You will need to download the “front end” software, emonLib and the other supporting libraries and set everything up using the Arduino IDE as explained in Learn→Electricity Monitoring→Using the Arduino IDE

You need to bear in mind that emonLib measures the two channels in sequence, and takes 10 cycles plus the calculation overhead for each, and it has to deal with temperature inputs and radio traffic at the same time, all of which put a limit on how fast you can go. I don’t think 5 s would be a problem; I wouldn’t like to say where you might run into problems, I’d guess in the 1 - 2 s area.

What you need to change is
TIME_BETWEEN_READINGS= 5000; in the file “src.ino” (I think - I’ve thrown all the individual files into one - it’s much easier to find things like that ),

Then you must compile it, transfer the compiled file (.hex) to your emonPi, and from there upload it into the front end.

My old notes say about the procedure for uploading:
"If you do want to change the firmware, it’s best to compile the Arduino sketch on your PC then transfer the .hex to the emonPi and upload directly from the RaspberryPi using avrdude. Emonhub will need to be stopped to free up the serial port. See example script to upload .hex https://github.com/openenergymonitor/emonpi/blob/master/firmware/compiled/update

#!/bin/bash
sudo service emonhub stop
avrdude -v -c arduino -p ATMEGA328P -P /dev/ttyAMA0 -b 115200 -U flash:w:/home/pi/emonpi/firmware/compiled/latest.hex
sudo service emonhub start

(That file might not be in that location any more, and obviously, where the file ends up when you transfer it will affect that command!)

I don’t know a reason not to increase the serial baud rate, if that helps (but then, I know very little about the RPi).

At present, you are tied to using the discrete sample version of emonLib. On its own, the new emonLibCM will run down to below 1 s reporting interval. I have a non-radio “emonPiCM” under test, but that’s some time away from being released as it needs significant changes to emonHub to integrate it into the system. (To develop that software, I took the “emon” pcb off the Pi and treated it like an emonTx - which is essentially what it is.)

borpin · 2020-06-28T19:41:27.654Z

Paul will throw his hands up in horror at the suggestion, so I’ll say it quietly, I’ve installed PlatformIO on a Pi and successfully compiled the sketch & uploaded the file to an emonTX via the serial interface, so it should work for the emonPi. EmonTX to Rpi - Direct Serial Connection - #48 by borpin.

A word of warning, on the emonPi by default, the firmware is uploaded on update. You can select not to, but you have to select not to.

drc · 2020-06-30T10:33:59.610Z

Thanks, both of you for your contributions. @borpin I’ve been looking though the guide here: Part 1/3: PlatformIO open-source embedded development ecosystem - Blog | OpenEnergyMonitor

And it seems quite straightforward (well, relatively…!). To do this in the safest way possible would you recommend something like the following: create a new SD card image with the PlatforIO on it and swap out my current SD (that contains all my data) and upload the file that way? This way I shouldn’t cause too much harm!

borpin · 2020-06-30T10:36:18.226Z

I’d take a data backup, but installing PIO should not impact the existing setup. That link I posted to my Serial project showed how I did it, may be better than the guide and shows how to generate the hex. Some of this may have been included in the repo on GitHub.

The extra stuff I did are in the maser of thie CM repo GitHub - openenergymonitor/EmonTxV3CM: EmonTxV3 Continuous Monitoring Firmware (Default shipped EmonTxV3 firmware)

Just clone it to your home folder and edit/compile/upload from there.

drc · 2020-06-30T23:28:21.412Z

Hi again,

I’ve been able to edit the change, compile it and run it on the microcontroller. I had some problems along the way but persistence paid off. Things to note on the emonpi are that the baud speed is 38400 and the reset pin is pin 7. I currently have emonpi set to update every 2.5 seconds but I will update this with further info if I push it faster.

I actually find the connection between the pi and the arduino quite tempermental, it only rarely works, I sometimes had to use:

sudo avrdude -v -p atmega328p -C /root/.platformio/packages/tool-avrdude/avrdude.conf -c arduino -b 38400 -D -P "/dev/ttyAMA0" -U flash:w:.pio/build/emonpi/firmware.hex:i

Has anyone else found this?

Here are my notes for anyone who wants to increase the speed of updates from emonpi that are sent to mqtt:

sudo su
systemctl stop emonhub
sudo pip install -U urllib3
sudo pip install -U platformio
cd /opt/openenergymonitor/emonpi/
git pull origin master
cd /opt/openenergymonitor/emonpi/firmware/src

EDIT CHANGES AND SAVE > TIME_BETWEEN_READINGS= 5000; in:

touch extra_script.py
nano extra_script.py

and paste >>

Import("env", "projenv")
from shutil import copyfile

def save_hex(*args, **kwargs):
    print("Copying hex output to project directory...")
    target = str(kwargs['target'][0])
    copyfile(target, 'output.hex')
    print("Done.")

env.AddPostAction("$BUILD_DIR/${PROGNAME}.hex", save_hex)   #post action for the target hex

cd /opt/openenergymonitor/emonpi/firmware
/root/.platformio/penv/bin/platformio project init
nano platformio.ini

add reference to extra_script.py >

[env:emontx_pi]
platform = atmelavr
framework = arduino
board = uno
build_flags = ${common.build_flags}
lib_deps = ${common.lib_deps_external}
monitor_speed = ${common.monitor_speed}
extra_scripts = post:extra_script.py

cd /opt/openenergymonitor/emonpi/firmware
sudo platformio run

open a parallel connection to view communication between the pi and the arduino:

miniterm --rtscts /dev/ttyAMA0 38400 

sudo platformio run -t upload
sudo service emonhub start

drc · 2020-07-01T11:56:02.781Z

If you don’t monitor using miniterm the upload seems to happen reliably. I’ve been able to increase the update to 4hz which is more than I need but I’ll keep an eye on it to ensure stability for a few days. Thanks everyone, I think this issue is resolved now.

borpin · 2020-07-01T12:14:20.912Z

drc:

If you don’t monitor using miniterm the upload seems to happen reliably.

It will. It’s not a good idea to have 2 different processes monitoring the same serial port.

drc · 2021-10-04T18:12:57.266Z

Here is a summary to do the same task after you have maybe upgraded the emonpi:

cd /opt/openenergymonitor/emonpi/firmware/src
sudo nano src.ino >> EDIT TIME_BETWEEN_READINGS= 5000;
cd ..
sudo systemctl stop emonhub
sudo platformio run -t upload
sudo systemctl start emonhub

Edit - formatted text. Moderator, BT

Viggz · 2022-03-24T14:36:27.832Z

how do you install platformio on the emonpi ?

borpin · 2022-03-25T09:59:52.413Z

GitHub

GitHub - openenergymonitor/EmonTxV3CM: EmonTxV3 Continuous Monitoring...

EmonTxV3 Continuous Monitoring Firmware (Default shipped EmonTxV3 firmware) - GitHub - openenergymonitor/EmonTxV3CM: EmonTxV3 Continuous Monitoring Firmware (Default shipped EmonTxV3 firmware)