Serial connection TxV3 to EmonPi

I seem to be having a bit of a problem interfacing my EmonTxV3 to my EmonPi. I had thought things were working, but now that I have some power running through my CT1, I see it’s not decoding the data correctly. It’s interpreting the first CT as the node number.

EmonHubLog: (2017-07-06 22:46:59,563 INFO MainThread EmonHub emonHub 'emon-pi' variant v1 - Pastebin.com)

2017-07-06 22:46:59,563 INFO     MainThread EmonHub emonHub 'emon-pi' variant v1.2
2017-07-06 22:46:59,564 INFO     MainThread Opening hub...
2017-07-06 22:46:59,565 INFO     MainThread Logging level set to DEBUG
2017-07-06 22:46:59,565 INFO     MainThread Creating EmonHubTx3eInterfacer 'SerialTx3e' 
2017-07-06 22:46:59,568 DEBUG    MainThread Opening serial port: /dev/ttyAMA0 @ 115200 bits/s
2017-07-06 22:46:59,569 DEBUG    MainThread Setting SerialTx3e pubchannels: ['ToEmonCMS']
2017-07-06 22:46:59,570 DEBUG    MainThread Interfacer: Subscribed to channel' : ch2
2017-07-06 22:46:59,572 INFO     MainThread Creating EmonHubMqttInterfacer 'MQTT' 
2017-07-06 22:46:59,575 INFO     MainThread MQTT Init mqtt_host=127.0.0.1 mqtt_port=1883 mqtt_user=emonpi
2017-07-06 22:46:59,579 DEBUG    MainThread MQTT Subscribed to channel' : ToEmonCMS
2017-07-06 22:46:59,581 DEBUG    SerialTx3e 1 NEW FRAME : ct1:84,ct2:0,ct3:0,ct4:0,vrms:10627,pulse:0
2017-07-06 22:46:59,582 DEBUG    SerialTx3e 1 Timestamp : 1499381219.58
2017-07-06 22:46:59,583 DEBUG    SerialTx3e 1 From Node : 84
2017-07-06 22:46:59,583 DEBUG    SerialTx3e 1    Values : [0, 0, 0, 10627, 0]
2017-07-06 22:46:59,584 DEBUG    SerialTx3e 1 Sent to channel(start)' : ToEmonCMS
2017-07-06 22:46:59,585 INFO     MainThread Creating EmonHubEmoncmsHTTPInterfacer 'emoncmsorg' 
2017-07-06 22:46:59,586 DEBUG    SerialTx3e 1 Sent to channel(end)' : ToEmonCMS
2017-07-06 22:46:59,588 DEBUG    MainThread emoncmsorg Subscribed to channel' : ToEmonCMS
2017-07-06 22:46:59,684 INFO     MQTT       Connecting to MQTT Server
2017-07-06 22:46:59,788 INFO     MQTT       Connecting to MQTT Server
2017-07-06 22:46:59,892 INFO     MQTT       Connecting to MQTT Server
2017-07-06 22:46:59,995 INFO     MQTT       Connecting to MQTT Server
2017-07-06 22:47:00,098 INFO     MQTT       Connecting to MQTT Server
2017-07-06 22:47:00,201 INFO     MQTT       Connecting to MQTT Server
2017-07-06 22:47:00,304 INFO     MQTT       Connecting to MQTT Server
2017-07-06 22:47:00,407 INFO     MQTT       Connecting to MQTT Server
2017-07-06 22:47:00,510 INFO     MQTT       Connecting to MQTT Server
2017-07-06 22:47:00,613 INFO     MQTT       Connecting to MQTT Server
2017-07-06 22:47:00,716 INFO     MQTT       Connecting to MQTT Server
2017-07-06 22:47:00,819 INFO     MQTT       Connecting to MQTT Server
2017-07-06 22:47:00,922 INFO     MQTT       Connecting to MQTT Server
2017-07-06 22:47:01,025 INFO     MQTT       Connecting to MQTT Server
2017-07-06 22:47:01,128 INFO     MQTT       Connecting to MQTT Server
2017-07-06 22:47:01,231 INFO     MQTT       Connecting to MQTT Server
2017-07-06 22:47:01,334 INFO     MQTT       Connecting to MQTT Server
2017-07-06 22:47:01,436 INFO     MQTT       Connecting to MQTT Server
2017-07-06 22:47:01,439 INFO     MQTT       connection status: Connection successful
2017-07-06 22:47:01,440 DEBUG    MQTT       CONACK => Return code: 0
2017-07-06 22:47:01,542 INFO     MQTT       on_subscribe
2017-07-06 22:47:09,527 DEBUG    SerialTx3e 2 NEW FRAME : ct1:73,ct2:0,ct3:0,ct4:0,vrms:10608,pulse:0
2017-07-06 22:47:09,528 DEBUG    SerialTx3e 2 Timestamp : 1499381229.53
2017-07-06 22:47:09,529 DEBUG    SerialTx3e 2 From Node : 73
2017-07-06 22:47:09,529 DEBUG    SerialTx3e 2    Values : [0, 0, 0, 10608, 0]
2017-07-06 22:47:09,530 DEBUG    SerialTx3e 2 Sent to channel(start)' : ToEmonCMS
2017-07-06 22:47:09,531 INFO     SerialTx3e Publishing: emon/73/1 0
2017-07-06 22:47:09,532 INFO     SerialTx3e Publishing: emon/73/2 0
2017-07-06 22:47:09,534 INFO     SerialTx3e Publishing: emon/73/3 0
2017-07-06 22:47:09,535 INFO     SerialTx3e Publishing: emon/73/4 10608
2017-07-06 22:47:09,536 INFO     SerialTx3e Publishing: emon/73/5 0
2017-07-06 22:47:09,538 INFO     SerialTx3e Publishing: emon/73/rssi 0
2017-07-06 22:47:09,539 INFO     SerialTx3e Publishing: emonhub/rx/73/values 0,0,0,10608,0
2017-07-06 22:47:09,540 INFO     SerialTx3e Publishing: emonhub/rx/73/rssi 0
2017-07-06 22:47:09,542 DEBUG    SerialTx3e 2 adding frame to buffer => [1499381229.527619, 73, 0, 0, 0, 10608, 0]
2017-07-06 22:47:09,542 DEBUG    SerialTx3e 2 Sent to channel(end)' : ToEmonCMS
2017-07-06 22:47:19,451 DEBUG    SerialTx3e 3 NEW FRAME : ct1:41,ct2:0,ct3:0,ct4:0,vrms:10626,pulse:0
2017-07-06 22:47:19,452 DEBUG    SerialTx3e 3 Timestamp : 1499381239.45
2017-07-06 22:47:19,452 DEBUG    SerialTx3e 3 From Node : 41
2017-07-06 22:47:19,453 DEBUG    SerialTx3e 3    Values : [0, 0, 0, 10626, 0]
2017-07-06 22:47:19,454 DEBUG    SerialTx3e 3 Sent to channel(start)' : ToEmonCMS
2017-07-06 22:47:19,455 INFO     SerialTx3e Publishing: emon/41/1 0
2017-07-06 22:47:19,456 INFO     SerialTx3e Publishing: emon/41/2 0
2017-07-06 22:47:19,457 INFO     SerialTx3e Publishing: emon/41/3 0
2017-07-06 22:47:19,459 INFO     SerialTx3e Publishing: emon/41/4 10626
2017-07-06 22:47:19,460 INFO     SerialTx3e Publishing: emon/41/5 0
2017-07-06 22:47:19,461 INFO     SerialTx3e Publishing: emon/41/rssi 0
2017-07-06 22:47:19,463 INFO     SerialTx3e Publishing: emonhub/rx/41/values 0,0,0,10626,0
2017-07-06 22:47:19,464 INFO     SerialTx3e Publishing: emonhub/rx/41/rssi 0
[interfacers]
### This interfacer manages the EmonTx3 ESP format serial
[[SerialTx3e]]
     Type = EmonHubTx3eInterfacer
      [[[init_settings]]]
           com_port= /dev/ttyAMA0
           com_baud = 115200
      [[[runtimesettings]]]
           pubchannels = ToEmonCMS,

I used the emonSD-07Nov16 image for a RPi3B I had lying around. Then I did the following because it seemed recommended.

$ sudo rpi-rw
$ sudo apt-get update
$ sudo apt-get dist-upgrade
$ sudo reboot
$ sudo apt-get clean all

Thanks,
Nathan

I think the emonPi is decoding the data correctly, it is the emonTx that is not sending the correct data. If you look at the “Direct Serial” sketch (emonTxV3_DirectSerial.ino), you’ll see that the NodeID is the first data item sent, followed by the powers and anything else you need.

  // Print to serial
    
    Serial.print(nodeID); Serial.print(' ');
    Serial.print(ct1.realPower); Serial.print(' '); 
    Serial.print(ct2.realPower); Serial.print(' '); 
    Serial.print(ct3.realPower); Serial.print(' '); 
    Serial.print(ct4.realPower); Serial.print(' '); 
    Serial.println(ct1.Vrms);

I just received this TxV3 in the mail yesterday, so I had assumed I was getting the latest firmware (which I did V2.9). And before I purchased it I read on the wiki that:

For emonTx V3 firmware V2.5+ then no firmware change is required…

I didn’t see a noRF firmware option, when I was ordering my EmonTx. But, yeah I can see that the current firmware does not send the nodeID:

  if (DEBUG==1) {
    Serial.print("ct1:"); Serial.print(emontx.power1);
    Serial.print(",ct2:"); Serial.print(emontx.power2);
    Serial.print(",ct3:"); Serial.print(emontx.power3);
    Serial.print(",ct4:"); Serial.print(emontx.power4);
    Serial.print(",vrms:"); Serial.print(emontx.Vrms);
    Serial.print(",pulse:"); Serial.print(emontx.pulseCount);
    if (DS18B20_STATUS==1){
      for(byte j=0;j<numSensors;j++){
        Serial.print(",t"); Serial.print(j); Serial.print(":");
        Serial.print(emontx.temp[j]);
      }
    }
    Serial.println();
    delay(50);
  }

But it also seems to me that adding a nodeoffset to the [interfacer] (emonhub/EmonHubTx3eInterfacer.py at emon-pi · openenergymonitor/emonhub · GitHub) might fix this,
although, I’m new to python:

    # Parse the ESP format string
    payload=[]
    if f.startswith('ct1:'):
      for item in f.split(',')[:]:
        payload.append(item.split(':')[1])
    f=payload

    if int(self._settings['nodeoffset']):
        c.nodeid = int(self._settings['nodeoffset'])
        c.realdata = f
    else:
        c.nodeid = int(f[0])
        c.realdata = f[1:]

Ahh! I think that’s where the confusion seeps in, The EmonHubTx3eInterfacer doesn’t use the emonTxV3_DirectSerial.ino, sketch nor the established “serial-direct” format to transfer data. It uses the emonesp serial format and is intended to work with a stock emonTx3.4 without changing the firmware.

The original EmonHubSerialInterfacer uses the established “serial-direct” format which is space separated values with a leading node id.

The newer EmonHubTx3eInterfacer I understand to expect key:value pairs and only work with post v2.4 (discrete_sampling?) emonTx firmware. v2.4 works at 9600 baud but from v2.5 the baud needs to be 115200.

The nodeid of the device appears to not be used as the emonTx doesn’t pass it out in the serial.print

and the EmonHubTx3eInterfacer seems to rely on the “nodeoffset” setting being used to define a node id with no link to the actual node id of the device. ie if you had an RFM2Pi connected and also hooked up the emonTx via a usb-serial adapter you could have the same data arriving via rfm using it’s FW node id and also via the usb-serial port as whatever node id you choose and duplicate the data.

This code also means if the nodeoffset is not defined it will fallback to using the first value, as if it were a “serial-direct” format with leading node id.

Ideally the node id should be passed out of the emonTx to avoid confusion, at the very least the EmonHubTx3eInterfacer needs editing to not work if the nodeoffset isn’t used or default to a specific node id “0” perhaps neither is ideal but I doubt we will see the node id being passed via serial if the emonESP doesn’t need it

perhaps the EmonHubTx3eInterfacer should default to using “emonTx3e” as the node name unless a new setting “nodename” is defined, but that may need some changes to the emonhub core to implement.

SO! Long story, short, I think you need to define a nodeoffset = 8 in the EmonHubTx3eInterfacer [[[runtimesettings]]].

In fact, as the node id in the sketch is now user selectable via the serial connection, the EmonHubTx3eInterfacer should be able to edit that node id (and the other settings) the same way it can on a rfm2pi or emonpi,

Sorry, looks like I wasn’t fast enough in my edit of my previous post. Thank you for the very quick response. I tried editing my emonhub.conf:

[interfacers]
### This interfacer manages the EmonTx3 ESP format serial
[[SerialTx3e]]
     Type = EmonHubTx3eInterfacer
      [[[init_settings]]]
           com_port= /dev/ttyAMA0
           com_baud = 115200
      [[[runtimesettings]]]
           pubchannels = ToEmonCMS,
           nodeoffset = 8

But now I get:

2017-07-07 15:30:06,831 DEBUG    SerialTx3e 3 NEW FRAME : ct1:0,ct2:0,ct3:0,ct4:0,vrms:10689,pulse:0
2017-07-07 15:30:06,832 WARNING  SerialTx3e 3 RX data length: 6 is not valid for datacodes ['h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'L']

I guess that was to be expected too!

The emonESP payload is dynamically sized depending on how many temp sensors you have, where as the original RFM payload is fixed at 6 temp sensors, since the project has previously defined the emonTxv3.4 payload in emonhub.conf nodes as fixed at 26bytes, the variable length emonESP payload is also “real values” so the datacodes are not required and datacode = 0 (as in zero decoding required) should be used instead.

Or just commenting out the datacode(s) line should also work as the default value is “0”.

Huzzah!

[[8]]
    nodename = emontx3
    [[[rx]]]
       names = power1, power2, power3, power4, vrms, temp1, temp2, temp3, temp4, temp5, temp6, pulse
       #datacodes = h,h,h,h,h,h,h,h,h,h,h,L
       datacode = 0
       scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
       units =W,W,W,W,V,C,C,C,C,C,C,p

Thanks for the help Paul

Since you do not appear to be using any temp sensors, you should trim that further to just

[[8]]
    nodename = emontx3
    [[[rx]]]
       names = power1, power2, power3, power4, vrms, pulse
       datacode = 0
       scales = 1,1,1,1,0.01,1
       units =W,W,W,W,V,p

so that the pulse lines up again, but if you are not using pulse either it won’t matter much I guess.

Ooh I’ll have to try that this weekend, i presume you just use the standard sketch and use the EmonHubTx3eInterfacer in my config.

Regards
Dave