Mosquitto_pub crashing MQTT on every message

[cidirome]

Hi there.

I’m trying to send messages to my emonpi_like system and I’m having problems.
Whenever I try to send any message, like

send null
mosquitto_pub -h 127.0.0.1 -u emonpi -P emonpimqtt2016 -t emonhub/tx/5/values -n -d

send one value
mosquitto_pub -h 127.0.0.1 -u emonpi -P emonpimqtt2016 -t emonhub/tx/5/values -m 10

send some values
mosquitto_pub -h 127.0.0.1 -u emonpi -P emonpimqtt2016 -t emonhub/tx/5/values -m 10,111,33 -d

send some values as a string (this is how it is working on my old pi, but it is possible that there may have been tweaks made by me to accept only bytes)
mosquitto_pub -h 127.0.0.1 -u emonpi -P emonpimqtt2016 -t emonhub/tx/5/values -m "10,111,33" -d

The result is always the same (log excerpt):

2022-04-17 12:43:51,452 WARNING  MQTT       Exception caught in MQTT thread. Traceback (most recent call last):
  File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 31, in wrapper
    return func(*args)
  File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 126, in run
    self.action()
  File "/opt/openenergymonitor/emonhub/src/interfacers/EmonHubMqttInterfacer.py", line 214, in action
    self._mqttc.loop(0)
  File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 1181, in loop
    rc = self.loop_read(max_packets)
  File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 1572, in loop_read
    rc = self._packet_read()
  File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 2310, in _packet_read
    rc = self._packet_handle()
  File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 2936, in _packet_handle
    return self._handle_publish()
  File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 3216, in _handle_publish
    self._handle_on_message(message)
  File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 3444, in _handle_on_message
    self.on_message(self, self._userdata, message)
  File "/opt/openenergymonitor/emonhub/src/interfacers/EmonHubMqttInterfacer.py", line 264, in on_message
    realdata = payload.split(",")
TypeError: a bytes-like object is required, not 'str'

2022-04-17 12:43:51,556 WARNING  MainThread MQTT thread is dead.
2022-04-17 12:43:51,557 WARNING  MainThread Attempting to restart thread MQTT (thread has been restarted 26 times...)

Any ideas of what is causing this?

Cheers.

Formatted text - BT, Moderator

[cidirome]

I’ve found this info

The ‘node only format’ is used with the emoncms Nodes Module (now deprecated on Emoncms V9+) and the emonPiLCD python service.

on this page: [emonhub/configuration.md at master · openenergymonitor/emonhub · GitHub]

Can this be the reason why it is presently not working?

Cheers.

[borpin (Brian Orpin)]

I think the 2 things are unrelated.

Using mosquitto_pub means you are bypassing emonhub.

@cidirome

TypeError: a bytes-like object is required, not 'str'

Is the issue here.

Can you include some log preceding the error?

Can you post your emonhub.cfg file?

@cidirome

I’m trying to send messages to my emonpi_like system

What exactly is your setup?

[cidirome]

@borpin, thank you for your answer and to look at may situation.

Here is the requested information:

Log preceding the error

2022-04-22 13:40:44,898 WARNING  RFM2Pi     3903 RX data length: 13 is not valid for datacodes ['h', 'h', 'h', 'h', 'h', 'h', 'L', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'L', 'L', 'b', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'h']
2022-04-22 13:40:45,424 INFO     MQTT       Publishing 'node' formatted msg
2022-04-22 13:40:47,473 INFO     MQTT       Publishing 'node' formatted msg
2022-04-22 13:40:47,834 WARNING  RFM2Pi     3906 Discarded RX frame 'string too short' : []
2022-04-22 13:40:49,511 INFO     MQTT       Publishing 'node' formatted msg
2022-04-22 13:40:49,956 WARNING  RFM2Pi     3909 RX data length: 13 is not valid for datacodes ['h', 'h', 'h', 'h', 'h', 'h', 'L', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'L', 'L', 'b', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'h']
2022-04-22 13:40:50,440 INFO     MQTT       Publishing 'node' formatted msg
2022-04-22 13:40:50,473 WARNING  RFM2Pi     3911 Discarded RX frame 'string too short' : []
2022-04-22 13:40:51,480 INFO     MQTT       Publishing 'node' formatted msg
2022-04-22 13:40:51,806 WARNING  MQTT       Exception caught in MQTT thread. Traceback (most recent call last):
  File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 31, in wrapper
    return func(*args)
  File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 126, in run
    self.action()
  File "/opt/openenergymonitor/emonhub/src/interfacers/EmonHubMqttInterfacer.py", line 214, in action
    self._mqttc.loop(0)
  File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 1181, in loop
    rc = self.loop_read(max_packets)
  File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 1572, in loop_read
    rc = self._packet_read()
  File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 2310, in _packet_read
    rc = self._packet_handle()
  File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 2936, in _packet_handle
    return self._handle_publish()
  File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 3216, in _handle_publish
    self._handle_on_message(message)
  File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 3444, in _handle_on_message
    self.on_message(self, self._userdata, message)
  File "/opt/openenergymonitor/emonhub/src/interfacers/EmonHubMqttInterfacer.py", line 264, in on_message
    realdata = payload.split(",")
TypeError: a bytes-like object is required, not 'str'

2022-04-22 13:40:51,849 WARNING  MainThread MQTT thread is dead.
2022-04-22 13:40:51,850 WARNING  MainThread Attempting to restart thread MQTT (thread has been restarted 0 times...)
2022-04-22 13:40:51,851 INFO     MainThread Setting RFM2Pi quiet: 1 (1q)
2022-04-22 13:40:52,852 INFO     MainThread Setting RFM2Pi calibration: 230V (1p)
2022-04-22 13:40:53,854 INFO     MainThread Creating EmonHubMqttInterfacer 'MQTT'
2022-04-22 13:40:53,857 INFO     MainThread Setting MQTT node_format_enable: 1
2022-04-22 13:40:53,857 INFO     MainThread Setting MQTT nodevar_format_enable: 1
2022-04-22 13:40:53,858 INFO     MainThread Setting MQTT nodevar_format_basetopic: emon/
2022-04-22 13:40:53,858 INFO     MainThread Setting MQTT node_JSON_enable: 1
2022-04-22 13:40:53,860 INFO     MainThread Setting emoncmsorg senddata: 1
2022-04-22 13:40:53,860 INFO     MainThread Setting emoncmsorg sendstatus: 1
2022-04-22 13:40:54,160 INFO     MQTT       Connecting to MQTT Server
2022-04-22 13:40:54,263 INFO     MQTT       connection status: Connection successful

Please ignore the messages “WARNING RFM2Pi 3909 RX data length: 13 is not valid” because they are relative to active devices I have in the premisses that are picked up by this one (that is only a test one). It a message that goes from the active EmonPi(HUB) to the EmonFrigo and is picked up by this test one.

emonhub.conf

[interfacers]
### This interfacer manages the RFM12Pi/RFM69Pi/emonPi module
[[RFM2Pi]]
    Type = EmonHubJeeInterfacer
    [[[init_settings]]]
        com_port = /dev/ttyAMA0
        com_baud = 38400                        # 9600 for old RFM12Pi
    [[[runtimesettings]]]
        pubchannels = ToEmonCMS,
        subchannels = ToRFM12,

        group = 210
        frequency = 433
        baseid = 5                              # emonPi / emonBase nodeID
        calibration = 230V                      # (UK/EU: 230V, US: 110V)
        quiet = true                            # Disable quite mode (default enabled) to enable RF packet debugging, show packets which fail crc
        # interval =  300                         # Interval to transmit time to emonGLCD (seconds)


[[MQTT]]

    Type = EmonHubMqttInterfacer
    [[[init_settings]]]
        mqtt_host = 127.0.0.1
        mqtt_port = 1883
        mqtt_user = emonpi
        mqtt_passwd = emonpimqtt2016

    [[[runtimesettings]]]
        pubchannels = ToRFM12,
        subchannels = ToEmonCMS,

        # emonhub/rx/10/values format
        # Use with emoncms Nodes module
        node_format_enable = 1
        node_format_basetopic = emonhub/

        # emon/emontx/power1 format - use with Emoncms MQTT input
        # http://github.com/emoncms/emoncms/blob/master/docs/RaspberryPi/MQTT.md
        nodevar_format_enable = 1
        nodevar_format_basetopic = emon/

        # Single JSON payload published  - use with Emoncms MQTT
        node_JSON_enable = 1
        node_JSON_basetopic = emon/

[[emoncmsorg]]
    Type = EmonHubEmoncmsHTTPInterfacer
    [[[init_settings]]]
    [[[runtimesettings]]]
        pubchannels = ToRFM12,
        subchannels = ToEmonCMS,
        url = https://emoncms.org
        apikey = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
        senddata = 1                    # Enable sending data to Emoncms.org
        sendstatus = 1                  # Enable sending WAN IP to Emoncms.org MyIP > https://emoncms.org/myip/list
        sendinterval= 30                # Bulk send interval to Emoncms.org in seconds

#######################################################################
#######################          Nodes          #######################
#######################################################################

[nodes]

## See config user guide: https://github.com/openenergymonitor/emonhub/blob/emon-pi/conf/emonhub.conf

[[5]]
    nodename = MyEmonPi4CT
    firmware = MyEmonPi_NewBoard_2022.ino.eightanaloginputs.ino
    hardware = emonpi_V0.1
    [[[rx]]]
        names = Grid, Solar, W_Heater, EV, TotalPower, Vrms,      noSamples, LowestNoSamples, minEnergy, maxEnergy,     T1, T2, T3, T4, T5, T6,   Wh_Import, Wh_Export, TriaqOn,     Permanent, Worse, Best, V0, V1, V2, V3, V4, V5, V6, V7, V8, V9, TotalEnergy
        datacodes = h, h, h, h, h, h,    L, h, h, h,      h, h, h, h, h, h,      L, L, b,     h, h, h, h, h, h, h, h, h, h, h, h, h, h
        scales = 1, 1, 1, 1, 1, 0.01, 1, 1, 1, 1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 1,1, 1, 1,1,1,1,1,1,1,1,1,1,1,1,0.01, 1
        units = W, W, W, W, V, Samp, Samp, J, J, C, C, C, C, C, C, Wh, Wh, (0:Off / 1:On), uS, uS, uS
    [[[tx]]]
        names = O
        units = X
        scales = 1
        datacodes = L

[[10]]
    nodename = EmonFrigo
    firmware = Frigo_RMF_SoftSPI.ino
    hardware = emonFrigo
    [[[rx]]]
        names = power1, power2, TotalPower, Vrms,      noSamples, LowestNoSamples, T1, T2, T3, T4,       RainLevel, Worse, Best, V0, V1, V2, V3, V4, V5, V6, V7, V8, V9
        datacodes = h, h, h, h,                                          L,                 h,                           h, h, h, h,                 h,              L,          L,      h, h, h, h, h, h, h, h, h, h
        scales = 1, 1, 1, 0.01,                                           1,                 1,                           0.1, 0.1, 0.1, 0.1,     1,              1,          1, 1,1,1,1,1,1,1,1,1,1
        units = W, W, W, V,                                               Samp,          Samp,                    C, C, C, C,                L,              uS,       uS
    [[[tx]]]
        names = O
        units = X
        scales = 1
        datacodes = L


[[6]]
    nodename = emontxshield
    [[[rx]]]
       names = power1, power2, power3, power4, vrms
       datacode = h
       scales = 1,1,1,1,0.01
       units = W,W,W,W,V

[[7]]
   nodename = emontx4
   [[[rx]]]
      names = power1, power2, power3, power4, vrms, temp1, temp2, temp3, temp4, temp5, temp6, pulse
      datacodes = h,h,h,h,h,h,h,h,h,h,h,L
      scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
      units = W,W,W,W,V,C,C,C,C,C,C,p

[[8]]
    nodename = emontx3
    [[[rx]]]
       names = power1, power2, power3, power4, vrms, temp1, temp2, temp3, temp4, temp5, temp6, pulse
       datacodes = h,h,h,h,h,h,h,h,h,h,h,L
       scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
       units = W,W,W,W,V,C,C,C,C,C,C,p

[[9]]

Maybe this is also usefull:

root@emonpi:~# mosquitto_pub -h 127.0.0.1 -u emonpi -P emonpimqtt2016 -t emonhub/tx/5/values -n -d
Client mosqpub|1464-emonpi sending CONNECT
Client mosqpub|1464-emonpi received CONNACK (0)
Client mosqpub|1464-emonpi sending PUBLISH (d0, q0, r0, m1, 'emonhub/tx/5/values', ... (0 bytes))
Client mosqpub|1464-emonpi sending DISCONNECT

root@emonpi:~# mosquitto_pub -h 127.0.0.1 -u emonpi -P emonpimqtt2016 -t emonhub/tx/5/values -m 10 -d
Client mosqpub|1536-emonpi sending CONNECT
Client mosqpub|1536-emonpi received CONNACK (0)
Client mosqpub|1536-emonpi sending PUBLISH (d0, q0, r0, m1, 'emonhub/tx/5/values', ... (2 bytes))
Client mosqpub|1536-emonpi sending DISCONNECT

[borpin (Brian Orpin)]

@borpin

What exactly is your setup?

What are you interfacing to? It looks like a custom RF packet.

What is the data format you are receiving?

My suspicion is your node definition is wrong - @Robert.Wall may be able to help more than me on that front.

You should also set the log to debug, with;

loglevel = DEBUG

[Robert.Wall]

@borpin

My suspicion is your node definition is wrong - @Robert.Wall may be able to help more than me on that front.

If I knew ANYTHING about MQTT, I might be able to help. But as I’ve never needed to know anything about it, I don’t.

@cidirome
You must not have two nodes with the same ID on the system. If you have, you need to change the ID of one of them, and change the entry in emonhub.conf to suit.
If you don’t want Node 13 to be picked up by emonHub, delete or comment out its definition in the file.
Most official OEM sketches have the node definition in the comments at the top of the sketch. You can copy a correct entry for the node in emonhub.conf from there. If you have modified the sketch, the onus is on you to create a valid definition to use in emonhub.conf. See Github for details.

[borpin (Brian Orpin)]

@Robert.Wall

If I knew ANYTHING about MQTT, I might be able to help. But as I’ve never needed to know anything about it,

I don’t think it is an MQTT problem. I think the data codes are not being decoded correctly and so the data being sent to the MQTT Interfacer is then not of the correct format.

[edit]
@cidirome try putting a try catch around this line and send the realdata object to the debugger.

openenergymonitor/emonhub/blob/f286e18e4d7ebda126ce1b06a193edd0c5d4431e/src/interfacers/EmonHubMqttInterfacer.py#L264

[borpin (Brian Orpin)]

@cidirome

pubchannels = ToRFM12,

I think this is your problem.

Take this line out as this is the MQTT interfacer listening for messages from the Broker and trying to interpret them.

[Robert.Wall]

Surely, the first step is indeed to acquire valid data. I know of no sketch that sends as much data as I can see - indeed more than 66 bytes is invalid in JeeLib - and I suspect that limit might well apply elsewhere, but I don’t know. I can well imagine that it’s been assumed that no more than 66 bytes will appear. It would be worth truncating the long data blocks to see if the problem goes away.

[borpin (Brian Orpin)]

@Robert.Wall

Surely, the first step is indeed to acquire valid data.

Yes, and don’t call me Shirley (just in case … https://youtu.be/KM2K7sV-K74)

[borpin (Brian Orpin)]

@cidirome Ok, I’m with you now, you are trying to send values to emonhub.

If you send it to the topic for Node 5, I think the message must match the node definition and it is expecting the message as a byte format (I think). I have not used this.

I’m not clear why you need to send the MQTT message to emonhub.

If you want to send values to EmonCMS via MQTT, use the topic emon in the message.

[Robert.Wall]

Shirley is a suburb of Solihull, West Midlands. I thought you were in Edinburgh - and I’ve got a friend in Eton Terrace, so if you are, watch out…

[Bill.Thomson]

@borpin

Don’t call me Shirley

[borpin (Brian Orpin)]

@cidirome I have managed to get into one of my systems and checked what I do (rather than rely on memory).

A minimal MQTT setup where emonhub receives data (RFM, Serial or whatever) and sends it on to an MQTT Broker for Emoncms to pick up. Note you can have multiple MQTT sections to send data to multiple MQTT Brokers.

[[MQTT2]]
    Type = EmonHubMqttInterfacer
    [[[init_settings]]]
        mqtt_host = 192.168.x.x
        mqtt_port = 1883
        mqtt_user = xxxxx
        mqtt_passwd = xxxxx

    [[[runtimesettings]]]
        subchannels = ToEmonCMS,

        node_format_enable = 0
        nodevar_format_enable = 0
        node_JSON_enable = 1
        node_JSON_basetopic = emon/

[cidirome]

Hi.
I don’t know exactly what is happening.
But it seems that anything I try to send crashes the MQTT.
The configuration is very similar to my old EmonPi (that is still working), just a more recent image on the SD card and a more recent Raspberry.
Note that I want to send values to EmonPi, basically all I need is to send a serial messages in the jeelibs (I think it is the correct name) packet format to the emonPi firmware. My firmware will interpret it or relay it to the RFM if the destination is not its own node id.
If there is other options besides mosquito_pub, I can use them, as long as I can use the linux command line.
Cheers.

[borpin (Brian Orpin)]

@cidirome

Note that I want to send values to EmonPi, basically all I need is to send a serial messages in the jeelibs (I think it is the correct name) packet format to the emonPi firmware. My firmware will interpret it or relay it to the RFM if the destination is not its own node id.

So you are sending RFM packets from your device to the RFM card on the EmonPi?

You then want emonhub to decode these and send to emoncms?

Correct?

Please post the current emonhub.cfg file.

Have you set LogLevel to DEBUG?

[cidirome]

Hi there.

Sorry for taking so long to answer: I’m between jobs and has to pause this project for a while.
Meanwhile I already updated the bootloader of the 328P to the new one that already programs @115200 baudrate.

Here is the emonhub.cfg

#######################################################################
#######################      emonhub.conf     #########################
#######################################################################
### emonHub configuration file, for info see documentation:
### https://github.com/openenergymonitor/emonhub/blob/emon-pi/configuration.md
#######################################################################
#######################    emonHub  settings    #######################
#######################################################################

[hub]
### loglevel must be one of DEBUG, INFO, WARNING, ERROR, and CRITICAL
#loglevel = WARNING
loglevel = DEBUG
#loglevel = INFO
### Uncomment this to also send to syslog
# use_syslog = yes
#######################################################################
#######################       Interfacers       #######################
#######################################################################

[interfacers]
### This interfacer manages the RFM12Pi/RFM69Pi/emonPi module
[[RFM2Pi]]
    Type = EmonHubJeeInterfacer
    [[[init_settings]]]
        com_port = /dev/ttyAMA0
        com_baud = 38400                        # 9600 for old RFM12Pi
    [[[runtimesettings]]]
        pubchannels = ToEmonCMS,
        subchannels = ToRFM12,

        group = 210
        frequency = 433
        baseid = 5                              # emonPi / emonBase nodeID
        calibration = 230V                      # (UK/EU: 230V, US: 110V)
        quiet = true                            # Disable quite mode (default enabled) to enable RF packet debugging, show packets which fail crc
        # interval =  300                         # Interval to transmit time to emonGLCD (seconds)


[[MQTT]]

    Type = EmonHubMqttInterfacer
    [[[init_settings]]]
        mqtt_host = 127.0.0.1
        mqtt_port = 1883
        mqtt_user = emonpi
        mqtt_passwd = emonpimqtt2016

    [[[runtimesettings]]]
        pubchannels = ToRFM12,
        subchannels = ToEmonCMS,

        # emonhub/rx/10/values format
        # Use with emoncms Nodes module
        node_format_enable = 1
        node_format_basetopic = emonhub/

        # emon/emontx/power1 format - use with Emoncms MQTT input
        # http://github.com/emoncms/emoncms/blob/master/docs/RaspberryPi/MQTT.md
        nodevar_format_enable = 1
        nodevar_format_basetopic = emon/

        # Single JSON payload published  - use with Emoncms MQTT
        node_JSON_enable = 1
        node_JSON_basetopic = emon/

[[emoncmsorg]]
    Type = EmonHubEmoncmsHTTPInterfacer
    [[[init_settings]]]
    [[[runtimesettings]]]
        pubchannels = ToRFM12,
        subchannels = ToEmonCMS,
        url = https://emoncms.org
        apikey = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
        senddata = 1                    # Enable sending data to Emoncms.org
        sendstatus = 1                  # Enable sending WAN IP to Emoncms.org MyIP > https://emoncms.org/myip/list
        sendinterval= 30                # Bulk send interval to Emoncms.org in seconds

#######################################################################
#######################          Nodes          #######################
#######################################################################

[nodes]

## See config user guide: https://github.com/openenergymonitor/emonhub/blob/emon-pi/conf/emonhub.conf

[[5]]
    nodename = MyEmonPi4CT
    firmware = MyEmonPi_NewBoard_2022.ino.eightanaloginputs.ino
    hardware = emonpi_V0.1
    [[[rx]]]
        names = Grid, Solar, W_Heater, EV, TotalPower, Vrms,      noSamples, LowestNoSamples, minEnergy, maxEnergy,     T1, T2, T3, T4, T5, T6,   Wh_Import, Wh_Export, TriaqOn,     Permanent, Worse, Best, V0, V1, V2, V3, V4, V5, V6, V7, V8, V9, TotalEnergy
        datacodes = h, h, h, h, h, h,    L, h, h, h,      h, h, h, h, h, h,      L, L, b,     h, h, h, h, h, h, h, h, h, h, h, h, h, h
        scales = 1, 1, 1, 1, 1, 0.01, 1, 1, 1, 1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 1,1, 1, 1,1,1,1,1,1,1,1,1,1,1,1,0.01, 1
        units = W, W, W, W, V, Samp, Samp, J, J, C, C, C, C, C, C, Wh, Wh, (0:Off / 1:On), uS, uS, uS
    [[[tx]]]
        names = O
        units = X
        scales = 1
        datacodes = L

[[10]]
    nodename = EmonFrigo
    firmware = Frigo_RMF_SoftSPI.ino
    hardware = emonFrigo
    [[[rx]]]
        names = power1, power2, TotalPower, Vrms,      noSamples, LowestNoSamples, T1, T2, T3, T4,       RainLevel, Worse, Best, V0, V1, V2, V3, V4, V5, V6, V7, V8, V9
        datacodes = h, h, h, h,                                          L,                 h,                           h, h, h, h,                 h,              L,          L,      h, h, h, h, h, h, h, h, h, h
        scales = 1, 1, 1, 0.01,                                           1,                 1,                           0.1, 0.1, 0.1, 0.1,     1,              1,          1, 1,1,1,1,1,1,1,1,1,1
        units = W, W, W, V,                                               Samp,          Samp,                    C, C, C, C,                L,              uS,       uS
    [[[tx]]]
        names = O
        units = X
        scales = 1
        datacodes = L


[[6]]
    nodename = emontxshield
    [[[rx]]]
       names = power1, power2, power3, power4, vrms
       datacode = h
       scales = 1,1,1,1,0.01
       units = W,W,W,W,V

[[7]]
   nodename = emontx4
   [[[rx]]]
      names = power1, power2, power3, power4, vrms, temp1, temp2, temp3, temp4, temp5, temp6, pulse
      datacodes = h,h,h,h,h,h,h,h,h,h,h,L
      scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
      units = W,W,W,W,V,C,C,C,C,C,C,p

[[8]]
    nodename = emontx3
    [[[rx]]]
       names = power1, power2, power3, power4, vrms, temp1, temp2, temp3, temp4, temp5, temp6, pulse
       datacodes = h,h,h,h,h,h,h,h,h,h,h,L
       scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
       units = W,W,W,W,V,C,C,C,C,C,C,p

[[9]]
   nodename = emontx2
   [[[rx]]]
      names = power1, power2, power3, power4, vrms, temp1, temp2, temp3, temp4, temp5, temp6, pulse
      datacodes = h,h,h,h,h,h,h,h,h,h,h,L
      scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
      units = W,W,W,W,V,C,C,C,C,C,C,p

[[11]]
    nodename = 3phase
    [[[rx]]]
       names = powerL1, powerL2, powerL3, power4, Vrms, temp1, temp2, temp3, temp4, temp5, temp6, pulse
       datacodes = h,h,h,h,h,h,h,h,h,h,h,L
       scales = 1,1,1,1,0.01,0.01,0.01,0.01,0.01,0.01,0.01,1
       units = W,W,W,W,V,C,C,C,C,C,C,p

[[12]]
    nodename = 3phase2
    [[[rx]]]
       names = powerL1, powerL2, powerL3, power4, Vrms, temp1, temp2, temp3, temp4, temp5, temp6, pulse
       datacodes = h,h,h,h,h,h,h,h,h,h,h,L
       scales = 1,1,1,1,0.01,0.01,0.01,0.01,0.01,0.01,0.01,1
       units = W,W,W,W,V,C,C,C,C,C,C,p

[[13]]
    nodename = 3phase3
    [[[rx]]]
       names = powerL1, powerL2, powerL3, power4, Vrms, temp1, temp2, temp3, temp4, temp5, temp6, pulse
       datacodes = h,h,h,h,h,h,h,h,h,h,h,L
       scales = 1,1,1,1,0.01,0.01,0.01,0.01,0.01,0.01,0.01,1
       units = W,W,W,W,V,C,C,C,C,C,C,p

[[14]]
    nodename = 3phase4
    [[[rx]]]
       names = powerL1, powerL2, powerL3, power4, Vrms, temp1, temp2, temp3, temp4, temp5, temp6, pulse
       datacodes = h,h,h,h,h,h,h,h,h,h,h,L
       scales = 1,1,1,1,0.01,0.01,0.01,0.01,0.01,0.01,0.01,1
       units = W,W,W,W,V,C,C,C,C,C,C,p

[[15]]
    nodename = emontx3cm15
    [[[rx]]]
       names = MSG, Vrms, P1, P2, P3, P4, E1, E2, E3, E4, T1, T2, T3, pulse
       datacodes = L,h,h,h,h,h,l,l,l,l,h,h,h,L
       scales = 1,0.01,1,1,1,1,1,1,1,1,0.01,0.01,0.01,1
       units = n,V,W,W,W,W,Wh,Wh,Wh,Wh,C,C,C,p
       whitening = 1

[[16]]
    nodename = emontx3cm16
    [[[rx]]]
       names = MSG, Vrms, P1, P2, P3, P4, E1, E2, E3, E4, T1, T2, T3, pulse
       datacodes = L,h,h,h,h,h,l,l,l,l,h,h,h,L
       scales = 1,0.01,1,1,1,1,1,1,1,1,0.01,0.01,0.01,1
       units = n,V,W,W,W,W,Wh,Wh,Wh,Wh,C,C,C,p
       whitening = 1

[[19]]
   nodename = emonth1
   [[[rx]]]
      names = temperature, external temperature, humidity, battery
      datacode = h
      scales = 0.1,0.1,0.1,0.1
      units = C,C,%,V

[[20]]
   nodename = emonth2
   [[[rx]]]
      names = temperature, external temperature, humidity, battery
      datacode = h
      scales = 0.1,0.1,0.1,0.1
      units = C,C,%,V

[[21]]
   nodename = emonth3
   [[[rx]]]
      names = temperature, external temperature, humidity, battery
      datacode = h
      scales = 0.1,0.1,0.1,0.1
      units = C,C,%,V

[[22]]
   nodename = emonth4
   [[[rx]]]
      names = temperature, external temperature, humidity, battery
      datacode = h
      scales = 0.1,0.1,0.1,0.1
      units = C,C,%,V

[[23]]
    nodename = emonth5
    [[[rx]]]
       names = temperature, external temperature, humidity, battery, pulsecount
       datacodes = h,h,h,h,L
       scales = 0.1,0.1,0.1,0.1,1
       units = C,C,%,V,p

[[24]]
    nodename = emonth6
    [[[rx]]]
       names = temperature, external temperature, humidity, battery, pulsecount
       datacodes = h,h,h,h,L
       scales = 0.1,0.1,0.1,0.1,1
       units = C,C,%,V,p

[[25]]
    nodename = emonth7
    [[[rx]]]
       names = temperature, external temperature, humidity, battery, pulsecount
       datacodes = h,h,h,h,L
       scales = 0.1,0.1,0.1,0.1,1
       units = C,C,%,V,p

[[26]]
    nodename = emonth8
    [[[rx]]]
       names = temperature, external temperature, humidity, battery, pulsecount
       datacodes = h,h,h,h,L
       scales = 0.1,0.1,0.1,0.1,1
       units = C,C,%,V,p

Some log with Debug activated, the crassing comand is:
mosquitto_pub -h 127.0.0.1 -u emonpi -P emonpimqtt2016 -t emonhub/tx/5/values -m 10 -d

2022-05-02 22:24:35,242 DEBUG    RFM2Pi     148 Sent to channel(start)' : ToEmonCMS
2022-05-02 22:24:35,242 DEBUG    RFM2Pi     148 Sent to channel(end)' : ToEmonCMS
2022-05-02 22:24:35,327 DEBUG    MQTT       Publishing: emon/MyEmonPi4CT/Grid 0
2022-05-02 22:24:35,328 DEBUG    MQTT       Publishing: emon/MyEmonPi4CT/Solar 0
2022-05-02 22:24:35,329 DEBUG    MQTT       Publishing: emon/MyEmonPi4CT/W_Heater 0
2022-05-02 22:24:35,330 DEBUG    MQTT       Publishing: emon/MyEmonPi4CT/EV 0
2022-05-02 22:24:35,331 DEBUG    MQTT       Publishing: emon/MyEmonPi4CT/TotalPower 0
2022-05-02 22:24:35,333 DEBUG    MQTT       Publishing: emon/MyEmonPi4CT/Vrms 247.63
2022-05-02 22:24:35,334 DEBUG    MQTT       Publishing: emon/MyEmonPi4CT/noSamples 9613
2022-05-02 22:24:35,335 DEBUG    MQTT       Publishing: emon/MyEmonPi4CT/LowestNoSamples 38
2022-05-02 22:24:35,336 DEBUG    MQTT       Publishing: emon/MyEmonPi4CT/minEnergy 0
2022-05-02 22:24:35,337 DEBUG    MQTT       Publishing: emon/MyEmonPi4CT/maxEnergy 110
2022-05-02 22:24:35,338 DEBUG    MQTT       Publishing: emon/MyEmonPi4CT/T1 0
2022-05-02 22:24:35,339 DEBUG    MQTT       Publishing: emon/MyEmonPi4CT/T2 0
2022-05-02 22:24:35,340 DEBUG    MQTT       Publishing: emon/MyEmonPi4CT/T3 0
2022-05-02 22:24:35,341 DEBUG    MQTT       Publishing: emon/MyEmonPi4CT/T4 0
2022-05-02 22:24:35,342 DEBUG    MQTT       Publishing: emon/MyEmonPi4CT/T5 0
2022-05-02 22:24:35,343 DEBUG    MQTT       Publishing: emon/MyEmonPi4CT/T6 0
2022-05-02 22:24:35,345 DEBUG    MQTT       Publishing: emon/MyEmonPi4CT/Wh_Import 0
2022-05-02 22:24:35,346 DEBUG    MQTT       Publishing: emon/MyEmonPi4CT/Wh_Export 0
2022-05-02 22:24:35,347 DEBUG    MQTT       Publishing: emon/MyEmonPi4CT/TriaqOn 0
2022-05-02 22:24:35,348 DEBUG    MQTT       Publishing: emon/MyEmonPi4CT/Permanent 20001
2022-05-02 22:24:35,349 DEBUG    MQTT       Publishing: emon/MyEmonPi4CT/Worse 88
2022-05-02 22:24:35,350 DEBUG    MQTT       Publishing: emon/MyEmonPi4CT/Best 28
2022-05-02 22:24:35,350 DEBUG    MQTT       Publishing: emon/MyEmonPi4CT/V0 0
2022-05-02 22:24:35,351 DEBUG    MQTT       Publishing: emon/MyEmonPi4CT/V1 0
2022-05-02 22:24:35,352 DEBUG    MQTT       Publishing: emon/MyEmonPi4CT/V2 0
2022-05-02 22:24:35,353 DEBUG    MQTT       Publishing: emon/MyEmonPi4CT/V3 0
2022-05-02 22:24:35,353 DEBUG    MQTT       Publishing: emon/MyEmonPi4CT/V4 500
2022-05-02 22:24:35,354 DEBUG    MQTT       Publishing: emon/MyEmonPi4CT/V5 0
2022-05-02 22:24:35,355 DEBUG    MQTT       Publishing: emon/MyEmonPi4CT/V6 0
2022-05-02 22:24:35,356 DEBUG    MQTT       Publishing: emon/MyEmonPi4CT/V7 0
2022-05-02 22:24:35,356 DEBUG    MQTT       Publishing: emon/MyEmonPi4CT/V8 611
2022-05-02 22:24:35,357 DEBUG    MQTT       Publishing: emon/MyEmonPi4CT/V9 50.01
2022-05-02 22:24:35,358 DEBUG    MQTT       Publishing: emon/MyEmonPi4CT/TotalEnergy -26
2022-05-02 22:24:35,359 INFO     MQTT       Publishing 'node' formatted msg
2022-05-02 22:24:35,359 DEBUG    MQTT       Publishing: emonhub/rx/5/values 0,0,0,0,0,247.63,9613,38,0,110,0,0,0,0,0,0,0,0,0,20001,88,28,0,0,0,0,500,0,0,0,611,50.01,-26
2022-05-02 22:24:35,360 DEBUG    MQTT       Publishing: emon/MyEmonPi4CT {"Grid": 0, "Solar": 0, "W_Heater": 0, "EV": 0, "TotalPower": 0, "Vrms": 247.63, "noSamples": 9613, "LowestNoSamples": 38, "minEnergy": 0, "maxEnergy": 110, "T1": 0, "T2": 0, "T3": 0, "T4": 0, "T5": 0, "T6": 0, "Wh_Import": 0, "Wh_Export": 0, "TriaqOn": 0, "Permanent": 20001, "Worse": 88, "Best": 28, "V0": 0, "V1": 0, "V2": 0, "V3": 0, "V4": 500, "V5": 0, "V6": 0, "V7": 0, "V8": 611, "V9": 50.01, "TotalEnergy": -26, "time": 1651526675.2375312}
2022-05-02 22:24:36,983 WARNING  MQTT       Exception caught in MQTT thread. Traceback (most recent call last):
  File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 31, in wrapper
    return func(*args)
  File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 126, in run
    self.action()
  File "/opt/openenergymonitor/emonhub/src/interfacers/EmonHubMqttInterfacer.py", line 214, in action
    self._mqttc.loop(0)
  File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 1181, in loop
    rc = self.loop_read(max_packets)
  File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 1572, in loop_read
    rc = self._packet_read()
  File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 2310, in _packet_read
    rc = self._packet_handle()
  File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 2936, in _packet_handle
    return self._handle_publish()
  File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 3216, in _handle_publish
    self._handle_on_message(message)
  File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 3444, in _handle_on_message
    self.on_message(self, self._userdata, message)
  File "/opt/openenergymonitor/emonhub/src/interfacers/EmonHubMqttInterfacer.py", line 264, in on_message
    realdata = payload.split(",")
TypeError: a bytes-like object is required, not 'str'

2022-05-02 22:24:37,166 WARNING  MainThread MQTT thread is dead.
2022-05-02 22:24:37,166 WARNING  MainThread Attempting to restart thread MQTT (thread has been restarted 1 times...)
2022-05-02 22:24:37,167 INFO     MainThread Setting RFM2Pi quiet: 1 (1q)
2022-05-02 22:24:37,254 DEBUG    RFM2Pi     device settings updated: E i5 g210 @ 433 MHz
2022-05-02 22:24:38,169 INFO     MainThread Setting RFM2Pi calibration: 230V (1p)
2022-05-02 22:24:38,259 DEBUG    RFM2Pi     device settings updated: E i5 g210 @ 433 MHz
2022-05-02 22:24:39,171 INFO     MainThread Creating EmonHubMqttInterfacer 'MQTT'
2022-05-02 22:24:39,173 DEBUG    MainThread Setting MQTT pubchannels: ['ToRFM12']
2022-05-02 22:24:39,174 DEBUG    MainThread Setting MQTT subchannels: ['ToEmonCMS']
2022-05-02 22:24:39,174 INFO     MainThread Setting MQTT node_format_enable: 1
2022-05-02 22:24:39,175 INFO     MainThread Setting MQTT nodevar_format_enable: 1
2022-05-02 22:24:39,175 INFO     MainThread Setting MQTT nodevar_format_basetopic: emon/
2022-05-02 22:24:39,175 INFO     MainThread Setting MQTT node_JSON_enable: 1
2022-05-02 22:24:39,177 INFO     MainThread Setting emoncmsorg senddata: 1
2022-05-02 22:24:39,177 INFO     MainThread Setting emoncmsorg sendstatus: 1
2022-05-02 22:24:40,276 DEBUG    RFM2Pi     149 NEW FRAME : OK 5 0 0 0 0 0 0 0 0 0 0 189 96 141 37 0 0 38 0 0 0 110 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 33 78 88 0 28 0 0 0 0 0 0 0 0 0 244 1 0 0 0 0 0 0 104 2 138 19 230 255 (-0)
2022-05-02 22:24:40,278 DEBUG    RFM2Pi     149 Timestamp : 1651526680.275959
2022-05-02 22:24:40,279 DEBUG    RFM2Pi     149 From Node : 5
2022-05-02 22:24:40,280 DEBUG    RFM2Pi     149    Values : [0, 0, 0, 0, 0, 247.65, 9613, 38, 0, 110, 0, 0, 0, 0, 0, 0, 0, 0, 0, 20001, 88, 28, 0, 0, 0, 0, 500, 0, 0, 0, 616, 50.02, -26]
2022-05-02 22:24:40,280 DEBUG    RFM2Pi     149 Sent to channel(start)' : ToEmonCMS
2022-05-02 22:24:40,281 DEBUG    RFM2Pi     149 Sent to channel(end)' : ToEmonCMS
2022-05-02 22:24:40,480 INFO     MQTT       Connecting to MQTT Server
2022-05-02 22:24:40,582 INFO     MQTT       connection status: Connection successful
2022-05-02 22:24:40,583 DEBUG    MQTT       CONACK => Return code: 0
2022-05-02 22:24:40,684 INFO     MQTT       on_subscribe
2022-05-02 22:24:41,637 DEBUG    emoncmsorg Buffer size: 7

[borpin (Brian Orpin)]

Why are you trying to publish a value to emonhub?

@cidirome

        # emonhub/rx/10/values format
        # Use with emoncms Nodes module
        node_format_enable = 1
        node_format_basetopic = emonhub/

You do not need this - note it says it is for the nodes module - which has long gone. This is in effect the input to emonhub via MQTT.

@cidirome

        # emon/emontx/power1 format - use with Emoncms MQTT input
        # http://github.com/emoncms/emoncms/blob/master/docs/RaspberryPi/MQTT.md
        nodevar_format_enable = 1
        nodevar_format_basetopic = emon/

        # Single JSON payload published  - use with Emoncms MQTT
        node_JSON_enable = 1
        node_JSON_basetopic = emon/

Either or here. you do not need both. You can see from the debug log, you are sending the same data to emoncms twice.

I suggest enabling JSON and disabling the nodevar format - you will only have one publish.

[cidirome]

@borpin

You do not need this - note it says it is for the nodes module - which has long gone. This is in effect the input to emonhub via MQTT.

Hi.
Thank you for taking the time to analyse my data.

The final destination is “EmonPi” (node 5, in quotes because it is a modified EmonPi), not EmonHub itself. (In fact I also need to send commands to the other nodes too, but this is the beginning)

As I said before:

Note that I want to send values to EmonPi, basically all I need is to send a serial messages in the jeelibs (I think it is the correct name) packet format to the emonPi firmware. My firmware will interpret it or relay it to the RFM if the destination is not its own node id.
If there is other options besides mosquito_pub, I can use them, as long as I can use the linux command line.

I can adapt to other kinds of commands, but this is what I had working on my old system, and I couldn’t find information about how it is done in the current EmonPi SD image.

Cheers.

[borpin (Brian Orpin)]

Ok, sorry, I do not know enough about your system and what you are trying to do to help.

Fundamentally the problem is in the error message;

@cidirome

TypeError: a bytes-like object is required, not 'str'

This may be because between the 2 systems the Python script has moved to Python3 and this has only just come to light because it is rarely used.

@bwduncan one for you perhaps?