Emontx4 and PI4 USB problems

menocar · 23 February 2023 14:39

Hi,

I’m having an issue with reliably getting data from a emontx4 over usb. When connected to a laptop if the serial over USB works reliably with the full messages include all the configuration when first connected.

When connected to a raspberry pi 4 the USB link is always recognised, but config data never shows in the emonhub logs and often when connected I get 3 burst of data and then no more. I have tried changing the USB lead, which doesn’t seem to change the result. Moving the usb lead around might be having an impact, but I’m not certain.

Attached below are some of the emonhub logs. You can see on the message 2023-02-23 14:32:08,581, where the config reporting should be there are some strange characters. Makes me wonder if I have a noise issue.

The raspberry PI also has some 1 wire temperature sensors and is power using a POE hat.

Any help gratefully received.

2023-02-23 14:32:04,248 DEBUG    USB        Opening serial port: /dev/serial/by-id/usb-Silicon_Labs_CP2102N_USB_to_UART_Bridge_Controller_7ebfffea5787ec118f33df3719c2d21c-if00-port0 @ 115200 bits/s
2023-02-23 14:32:08,581 DEBUG    USB        w!$!;0w!$MSG:366,Vrms:261.78,P1:-170,P2:145,P3:10,P4:302,P5:120,P6:61,E1:-128852,E2:348152,E3:80508,E4:233878,E5:86280,E6:113709,T1:0.00,T2:0.00,T3:0.00,pulse:300768
2023-02-23 14:32:18,326 DEBUG    USB        125 NEW FRAME : MSG:367,Vrms:262.93,P1:-171,P2:145,P3:9,P4:308,P5:125,P6:62,E1:-128853,E2:348153,E3:80508,E4:233879,E5:86280,E6:113709,T1:0.00,T2:0.00,T3:0.00,pulse:300768
2023-02-23 14:32:18,327 DEBUG    USB        125 Timestamp : 1677162738.325987
2023-02-23 14:32:18,327 DEBUG    USB        125 From Node : emonTx4
2023-02-23 14:32:18,328 DEBUG    USB        125    Values : [367, 262.93, -171, 145, 9, 308, 125, 62, -128853, 348153, 80508, 233879, 86280, 113709, 0, 0, 0, 300768, 44, 95.65217391304348]
2023-02-23 14:32:18,328 DEBUG    USB        125 Sent to channel(start)' : ToEmonCMS
2023-02-23 14:32:18,329 DEBUG    USB        125 Sent to channel(end)' : ToEmonCMS

Edit - formatted text. Moderator, BT

borpin · 26 February 2023 09:19

I’m not sure what config data you are expecting.

Can you post your emonhub config, please?

Is the Pi running and emonSD image?

menocar · 27 February 2023 13:46

Hi,

When emontx USB comms works well (everytime connected to laptop or occasionally with pi) before the first message with data in I get information like the following.

2023-02-27 13:33:06,090 DEBUG    USB        Band 433 MHz, Group 210, Node 17, 7 dBm
2023-02-27 13:33:06,191 DEBUG    USB        Calibration:
2023-02-27 13:33:06,293 DEBUG    USB        vCal = 807.86
2023-02-27 13:33:06,395 DEBUG    USB        assumedV = 240.00
2023-02-27 13:33:06,497 DEBUG    USB        i1Cal = 150.15
2023-02-27 13:33:06,599 DEBUG    USB        i1Lead = 3.20
2023-02-27 13:33:06,701 DEBUG    USB        i2Cal = 150.15

I’m not worried about seeing the config being reported, it is just another symptom. The main issue is that many times I don’t get the config data and then only get a few messages with data in before silence from the USB in the emonhub logs. The emontx is still flashing at suitable intervals. Removing and reconnecting the USB lead usual results in a few more samples. Moving the usb lead around and reconnecting sometimes stablises things and I then get a continuous stream of samples for a few days.

The PI isn’t running an emonSD image. It has manually installed emoncms and hub. The emonhub is working fine for other data feed in using the socket interface.

Thanks.

Emonhub config below

#######################################################################
#######################      emonhub.conf     #########################
#######################################################################
### emonHub configuration file, for info see documentation:
### https://github.com/openenergymonitor/emonhub/blob/emon-pi/configuration.md
#######################################################################
#######################    emonHub  settings    #######################
#######################################################################

[hub]
### loglevel must be one of DEBUG, INFO, WARNING, ERROR, and CRITICAL
loglevel = DEBUG
### Uncomment this to also send to syslog
# use_syslog = yes
#######################################################################
#######################       Interfacers       #######################
#######################################################################

[interfacers]

[[USB]]
    Type = EmonHubOEMInterfacer
    [[[init_settings]]]
        com_port = /dev/serial/by-id/usb-Silicon_Labs_CP2102N_USB_to_UART_Bridge_Controller_7ebfffea5787ec118f33df3719c2d21c-if00-port0
        com_baud = 115200
    [[[runtimesettings]]]
        pubchannels = ToEmonCMS,
        nodename = emonTx4

### This interfacer manages the RFM12Pi/RFM69Pi/emonPi module
#[[RFM2Pi]]
#    Type = EmonHubJeeInterfacer
#    [[[init_settings]]]
#        com_port = /dev/ttyAMA0
#        com_baud = 38400                        # 9600 for old RFM12Pi
#    [[[runtimesettings]]]
#        pubchannels = ToEmonCMS,
#        subchannels = ToRFM12,
#
#        group = 210
#        frequency = 433
#        baseid = 5                              # emonPi / emonBase nodeID
#        calibration = 230V                      # (UK/EU: 230V, US: 110V)
#        quiet = true                            # Disable quite mode (default enabled) to enable RF packet debugging, show packets which fail crc
#        # interval =  300                         # Interval to transmit time to emonGLCD (seconds)

[[Socket]]

    Type = EmonHubSocketInterfacer

    [[[init_settings]]]

        port_nb = 50011

    [[[runtimesettings]]]
        pubchannels = ToEmonCMS,
        subchannels = ToRFM12,

#[[MQTT]]
#
#    Type = EmonHubMqttInterfacer
#    [[[init_settings]]]
#        mqtt_host = 127.0.0.1
#        mqtt_port = 1883
#        mqtt_user = emonpi
#        mqtt_passwd = emonpimqtt2016
#
#    [[[runtimesettings]]]
#        pubchannels = ToRFM12,
#        subchannels = ToEmonCMS,
#
#        # emonhub/rx/10/values format
#        # Use with emoncms Nodes module
#        node_format_enable = 1
#        node_format_basetopic = emonhub/
#
#        # emon/emontx/power1 format - use with Emoncms MQTT input
#        # http://github.com/emoncms/emoncms/blob/master/docs/RaspberryPi/MQTT.md
#        nodevar_format_enable = 1
#        nodevar_format_basetopic = emon/
#
#        # Single JSON payload published  - use with Emoncms MQTT
#        node_JSON_enable = 0
#        node_JSON_basetopic = emon/

#[[emoncmsorg]]
#    Type = EmonHubEmoncmsHTTPInterfacer
#    [[[init_settings]]]
#    [[[runtimesettings]]]
#       pubchannels = ToRFM12,
#       subchannels = ToEmonCMS,
#       url = https://emoncms.org
#       apikey = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
#       senddata = 1                    # Enable sending data to Emoncms.org
#       sendstatus = 1                  # Enable sending WAN IP to Emoncms.org MyIP > https://emoncms.org/myip/list
#       interval= 30                # Bulk send interval to Emoncms.org in seconds

[[my-internal-emoncms-server]]
    Type = EmonHubEmoncmsHTTPInterfacer
    [[[init_settings]]]
    [[[runtimesettings]]]
        pubchannels = ToRFM12,
        subchannels = ToEmonCMS,
        url = https://pi4.horsley.me.uk
        apikey = 59029d7de92265ee48b84c0a72fe534a
        senddata = 1                    # my-awesome-emoncms-server
        sendstatus = 0                  # Enable sending WAN IP to Emoncms MyIP$
        sendinterval= 15                # Bulk send interval to post in seconds
        timestamped = True              # Add timestamp to data sent


#######################################################################
#######################          Nodes          #######################
#######################################################################

[nodes]

## See config user guide: https://github.com/openenergymonitor/emonhub/blob/emon-pi/conf/emonhub.conf

[[5]]
    nodename = emonpi
    [[[rx]]]
        names = power1,power2,power1pluspower2,vrms,t1,t2,t3,t4,t5,t6,pulsecount
        datacodes = h, h, h, h, h, h, h, h, h, h, L
        scales = 1,1,1,0.01,0.1,0.1,0.1,0.1,0.1,0.1,1
        units = W,W,W,V,C,C,C,C,C,C,p

[[6]]
    nodename = emontxshield
    [[[rx]]]
       names = power1, power2, power3, power4, vrms
       datacode = h
       scales = 1,1,1,1,0.01
       units = W,W,W,W,V

[[7]]
   nodename = emontx4
   [[[rx]]]
      names = power1, power2, power3, power4, vrms, temp1, temp2, temp3, temp4, temp5, temp6, pulse
      datacodes = h,h,h,h,h,h,h,h,h,h,h,L
      scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
      units = W,W,W,W,V,C,C,C,C,C,C,p

[[8]]
    nodename = emontx3
    [[[rx]]]
       names = power1, power2, power3, power4, vrms, temp1, temp2, temp3, temp4, temp5, temp6, pulse
       datacodes = h,h,h,h,h,h,h,h,h,h,h,L
       scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
       units = W,W,W,W,V,C,C,C,C,C,C,p

[[9]]
   nodename = emontx2
   [[[rx]]]
      names = power1, power2, power3, power4, vrms, temp1, temp2, temp3, temp4, temp5, temp6, pulse
      datacodes = h,h,h,h,h,h,h,h,h,h,h,L
      scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
      units = W,W,W,W,V,C,C,C,C,C,C,p

[[10]]
    nodename = emontx1
    [[[rx]]]
       names = power1, power2, power3, power4, vrms, temp1, temp2, temp3, temp4, temp5, temp6, pulse
       datacodes = h,h,h,h,h,h,h,h,h,h,h,L
       scales = 1,1,1,1,0.01,0.01,0.01,0.01,0.01,0.01,0.01,1
       units = W,W,W,W,V,C,C,C,C,C,C,p

[[11]]
    nodename = 3phase
    [[[rx]]]
       names = powerL1, powerL2, powerL3, power4, Vrms, temp1, temp2, temp3, temp4, temp5, temp6, pulse
       datacodes = h,h,h,h,h,h,h,h,h,h,h,L
       scales = 1,1,1,1,0.01,0.01,0.01,0.01,0.01,0.01,0.01,1
       units = W,W,W,W,V,C,C,C,C,C,C,p

[[12]]
    nodename = 3phase2
    [[[rx]]]
       names = powerL1, powerL2, powerL3, power4, Vrms, temp1, temp2, temp3, temp4, temp5, temp6, pulse
       datacodes = h,h,h,h,h,h,h,h,h,h,h,L
       scales = 1,1,1,1,0.01,0.01,0.01,0.01,0.01,0.01,0.01,1
       units = W,W,W,W,V,C,C,C,C,C,C,p

[[13]]
    nodename = 3phase3
    [[[rx]]]
       names = powerL1, powerL2, powerL3, power4, Vrms, temp1, temp2, temp3, temp4, temp5, temp6, pulse
       datacodes = h,h,h,h,h,h,h,h,h,h,h,L
       scales = 1,1,1,1,0.01,0.01,0.01,0.01,0.01,0.01,0.01,1
       units = W,W,W,W,V,C,C,C,C,C,C,p

[[14]]
    nodename = 3phase4
    [[[rx]]]
       names = powerL1, powerL2, powerL3, power4, Vrms, temp1, temp2, temp3, temp4, temp5, temp6, pulse
       datacodes = h,h,h,h,h,h,h,h,h,h,h,L
       scales = 1,1,1,1,0.01,0.01,0.01,0.01,0.01,0.01,0.01,1
       units = W,W,W,W,V,C,C,C,C,C,C,p

[[15]]
    nodename = emontx3cm15
    [[[rx]]]
       names = MSG, Vrms, P1, P2, P3, P4, E1, E2, E3, E4, T1, T2, T3, pulse
       datacodes = L,h,h,h,h,h,l,l,l,l,h,h,h,L
       scales = 1,0.01,1,1,1,1,1,1,1,1,0.01,0.01,0.01,1
       units = n,V,W,W,W,W,Wh,Wh,Wh,Wh,C,C,C,p
       whitening = 1

[[16]]
    nodename = emontx3cm16
    [[[rx]]]
       names = MSG, Vrms, P1, P2, P3, P4, E1, E2, E3, E4, T1, T2, T3, pulse
       datacodes = L,h,h,h,h,h,l,l,l,l,h,h,h,L
       scales = 1,0.01,1,1,1,1,1,1,1,1,0.01,0.01,0.01,1
       units = n,V,W,W,W,W,Wh,Wh,Wh,Wh,C,C,C,p
       whitening = 1

[[17]]
    nodename = emonTx4_17
    [[[rx]]]
        names = MSG, Vrms, P1, P2, P3, P4, P5, P6, E1, E2, E3, E4, E5, E6, T1, T2, T3, pulse
        datacodes = L, h, h, h, h, h, h, h, l, l, l, l, l, l, h, h, h, L
        scales = 1.0, 0.01, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.01, 0.01, 0.01, 1.0
        units = n, V, W, W, W, W, W, W, Wh, Wh, Wh, Wh, Wh, Wh, C, C, C, p

[[18]]
    nodename = 1wire
    [[[rx]]]
       datacode = h
       scale = 0.1
       unit = C

[[19]]
   nodename = emonth1
   [[[rx]]]
      names = temperature, external temperature, humidity, battery
      datacode = h
      scales = 0.1,0.1,0.1,0.1
      units = C,C,%,V

[[20]]
   nodename = emonth2
   [[[rx]]]
      names = temperature, external temperature, humidity, battery
      datacode = h
      scales = 0.1,0.1,0.1,0.1
      units = C,C,%,V

[[21]]
   nodename = emonth3
   [[[rx]]]
      names = temperature, external temperature, humidity, battery
      datacode = h
      scales = 0.1,0.1,0.1,0.1
      units = C,C,%,V

[[22]]
   nodename = emonth4
   [[[rx]]]
      names = temperature, external temperature, humidity, battery
      datacode = h
      scales = 0.1,0.1,0.1,0.1
      units = C,C,%,V

[[23]]
    nodename = emonth5
    [[[rx]]]
       names = temperature, external temperature, humidity, battery, pulsecount
       datacodes = h,h,h,h,L
       scales = 0.1,0.1,0.1,0.1,1
       units = C,C,%,V,p

[[24]]
    nodename = emonth6
    [[[rx]]]
       names = temperature, external temperature, humidity, battery, pulsecount
       datacodes = h,h,h,h,L
       scales = 0.1,0.1,0.1,0.1,1
       units = C,C,%,V,p

[[25]]
    nodename = emonth7
    [[[rx]]]
       names = temperature, external temperature, humidity, battery, pulsecount
       datacodes = h,h,h,h,L
       scales = 0.1,0.1,0.1,0.1,1
       units = C,C,%,V,p

[[26]]
    nodename = emonth8
    [[[rx]]]
       names = temperature, external temperature, humidity, battery, pulsecount
       datacodes = h,h,h,h,L
       scales = 0.1,0.1,0.1,0.1,1
       units = C,C,%,V,p

[[27]]
    nodename = hmnetwork
    [[[rx]]]
       names = water_demand, t1, d1, t2, d2, t3, d3, t4, d4, t5, d5
       datacodes = B, h, B, h, B, h, B, h, B, h, B
       scales = 1,0.1,1,0.1,1,0.1,1,0.1,1,0.1,1
       units = d, C, d, C, d, C, d, C, d, C, d
[[28]]
    nodename = boiler
    [[[rx]]]
       datacode = h
       scale = 0.1
       unit = C

[APIkey corrupted for your security - Moderator (RW)]

borpin · 27 February 2023 13:52

If you stop emonhub and connect via miniterm instead, what do you see?

i.e. is it an issue with emonhub or an issue with the serial comms?

[edit]
On my emonPi (Pi 3B) I get this.

 $ python -m serial.tools.miniterm --rtscts /dev/ttyUSB0 115200
--- Miniterm on /dev/ttyUSB0  115200,8,N,1 ---
--- Quit: Ctrl+] | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
emonTx V4 CM Continuous Monitoring V1.5.4
OpenEnergyMonitor.org
Loaded EEPROM config
Settings:
Band 433 MHz, Group 210, Node 15, 7 dBm
Calibration:
vCal = 807.86
assumedV = 240.00
i1Cal = 150.15
i1Lead = 3.20
i2Cal = 150.15
i2Lead = 3.20
i3Cal = 150.15
i3Lead = 3.20
i4Cal = 60.06
i4Lead = 3.20
i5Cal = 60.06
i5Lead = 3.20
i6Cal = 60.06
i6Lead = 3.20
datalog = 9.80
pulses = 1
pulse period = 100
RF off
temp_enable = 0
JSON Format on
Reference voltage calibration: 1.0235
No temperature sensors detected, disabling temperature
AC present - Real Power calc enabled
{"MSG":1,"Vrms":248.46,"P1":0,"P2":1,"P3":317,"P4":126,"P5":0,"P6":0,"E1":-445,"E2":230,"E3":289612,"E4":115341,"E5":-25,"E6":-351,"T1":0.00,"T2":0.00,"T3":0.00,"pulse":2185}
{"MSG":2,"Vrms":248.84,"P1":0,"P2":0,"P3":309,"P4":123,"P5":0,"P6":0,"E1":-445,"E2":230,"E3":289613,"E4":115341,"E5":-25,"E6":-351,"T1":0.00,"T2":0.00,"T3":0.00,"pulse":2185}
{"MSG":3,"Vrms":249.47,"P1":0,"P2":0,"P3":310,"P4":123,"P5":0,"P6":0,"E1":-445,"E2":230,"E3":289613,"E4":115341,"E5":-25,"E6":-351,"T1":0.00,"T2":0.00,"T3":0.00,"pulse":2185}

I’ll leave it running and see what happens

@Robert.Wall @TrystanLea - each time I start the serial interface via miniterm it seems to reset the emonTX4 (starts from message 1). Is that correct?

Robert.Wall · 27 February 2023 14:37

Miniterm is obviously sending a reset. That’s what I originally intended to happen on a restart. @TrystanLea might have changed it since.

I’m wondering whether sending the startup config data is the cause of @menocar’s problem, because that information was never intended to be sent to emonHub/emonCMS. It should really be turned off by selecting a more appropriate data format. But equally, emonHub should be capable of rejecting it without ill effect.

I’ve not yet found the time to check exactly what Trystan’s V4 sketch does - it might well be failing to properly parse data that the Pi is erroneously sending it, which is what’s causing it to shut down. I suggest looking into that possibility.

borpin · 27 February 2023 14:42

I have a feeling the python may be resetting / opening a new connection occasionally, causing the data stream to reset, so yes, the setup data might be the cause. Never seen it before when connecting via the UART, connecting via the USB-A/USB-C port might be the issue.

Is it easy to disable that ‘reset’ function in code?

Robert.Wall · 27 February 2023 14:52

I don’t know Python, so I can’t say. It’s probably not the language anyway, but the comms utility of the Pi’s OS that handles the USB interface.

Is the Pi echoing all the config data back to the V4? If some bright spark thought it would always be a human at the end of the USB cable and thought they needed to see what they’d typed, this will cause no end of trouble. (i.e. Like the ESP8266 does, making it necessary to cut the cable in the outgoing direction.)

borpin · 27 February 2023 14:55

It doesn’t send it back, if that is what you mean.

Could the TX4 ignore any reset sent except during boot?

Robert.Wall · 27 February 2023 14:57

I don’t know how a reset sent via USB gets to the Atmel-DB processor.

borpin · 27 February 2023 21:16

@menocar - ~~so there definitely is an issue, don’t know what.~~ @TrystanLea

Ignore that.

borpin · 27 February 2023 21:31

Odd, I changed the config and changed it back and now it works. Most odd.

[[USB]]
    Type = EmonHubOEMInterfacer
    [[[init_settings]]]
        com_port = /dev/ttyUSB0
        com_baud = 115200
    [[[runtimesettings]]]
        pubchannels = ToEmonCMS,

        nodename = emonTX4test

Running a grep on the log so see if I get any resets.

borpin · 27 February 2023 21:35

I’ll bet it is a power issue. You are powering the TX4 via the USB from a Pi powered by a POE Hat.

Not enough power to do all that.

menocar · 28 February 2023 08:04

Thanks for looking at this.

I’m using a hardwired emonVS, so I assumed that was powering the TX4.

Does the
2023-02-27 13:33:06,395 DEBUG USB assumedV = 240.00
suggest that the TX4 isn’t being powered by the emonVS?

menocar · 28 February 2023 08:15

@borpin I connected from the Pi via miniterm. The output is below. Having said that I’m not seeing the issue today. When I’m at home this evening I will see if I can recreate the problem with the usb lead and capture some more logs from emonhub and miniterm.

pi@pi4:~ $ python -m serial.tools.miniterm --rtscts /dev/ttyUSB1 115200
--- Miniterm on /dev/ttyUSB1  115200,8,N,1 ---
--- Quit: Ctrl+] | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
emonTx V4 CM Continuous Monitoring V1.5.4
OpenEnergyMonitor.org
Loaded EEPROM config
Settings:
Band 433 MHz, Group 210, Node 17, 7 dBm
Calibration:
vCal = 807.86
assumedV = 240.00
i1Cal = 150.15
i1Lead = 3.20
i2Cal = 150.15
i2Lead = 3.20
i3Cal = 300.30
i3Lead = 3.20
i4Cal = 60.06
i4Lead = 3.20
i5Cal = 60.06
i5Lead = 3.20
i6Cal = 60.06
i6Lead = 3.20
datalog = 9.80
pulses = 1
pulse period = 100
RF off
temp_enable = 0
JSON Format Off
Reference voltage calibration: 1.0270
No temperature sensors detected, disabling temperature
AC present - Real Power calc enabled
MSG:1,Vrms:260.55,P1:-79,P2:266,P3:62,P4:88,P5:3,P6:109,E1:-139150,E2:377307,E3:88286,E4:256287,E5:97833,E6:120396,T1:0.00,T2:0.00,T3:0.00,pulse:324011
MSG:2,Vrms:258.12,P1:-78,P2:262,P3:61,P4:87,P5:3,P6:107,E1:-139150,E2:377307,E3:88286,E4:256287,E5:97833,E6:120396,T1:0.00,T2:0.00,T3:0.00,pulse:324011
MSG:3,Vrms:259.14,P1:-77,P2:269,P3:61,P4:87,P5:3,P6:112,E1:-139150,E2:377308,E3:88286,E4:256287,E5:97833,E6:120396,T1:0.00,T2:0.00,T3:0.00,pulse:324012

Robert.Wall · 28 February 2023 11:44

“Assumed voltage” was the voltage that will be used for apparent power should the sampled (therefore accurate) mains voltage not be available. If the voltage sample is available at power-up, this is always used as it allows real power to be calculated.
“was the voltage that will be used” because I have yet to check the latest V4 version is the same as the V3.4 software. So I don’t know whether printing that means it’s telling you the value that IS being used (which almost means the V4 is not powered by the Vs) or it’s telling you the value that will be used when the V4 is not powered by the Vs.

borpin · 28 February 2023 17:00

Ah, no. I thought that was only printed if there wasn’t a reference voltage, but I do have that message and I do have a Vs.

I have been running for over 7,000 messages following the emonhub log, scrolled back as far as I could but all the messages are fine.

I wonder if the cable has a fault.

Otherwise, something else in your setup is causing the issue I’d suggest.