How to map data captured over Input API to `pubchannels`?

mnbf9rca · 2022-12-22T17:13:37.424Z

I’ve got a hildebrand IHD which exposes an MQTT endpoint. I took a python script i found on this community and updated it to the data structure that i’m seeing on MQTT (you can see that in Github). Additionally, i have an emonTx4 connected via serial (USB).

Both these feeds work fine, and I can see both devices and the relevant inputs in my emonpi:

image852×870 38.8 KB

I’d like the data to show up on emoncms org. I can see the data from emonTx4 (albeit without labels), but the Glow data never appears:

I think the trouble is that i dont know how to create an interfacer to map the incoming Glow data (sent over an HTTP request to the Input API) to the ToEmonCMS channel. I’ve looked at the examples in github, and even at the source, but i can’t tell how to do it.

Here’s my config file:

#######################################################################
#######################      emonhub.conf     #########################
#######################################################################
### emonHub configuration file, for info see documentation:
### https://github.com/openenergymonitor/emonhub/blob/emon-pi/configuration.md
#######################################################################
#######################    emonHub  settings    #######################
#######################################################################

[hub]
### loglevel must be one of DEBUG, INFO, WARNING, ERROR, and CRITICAL
loglevel = DEBUG
autoconf = 1

### Uncomment this to also send to syslog
# use_syslog = yes
#######################################################################
#######################       Interfacers       #######################
#######################################################################

[interfacers]

[[USB]]
    Type = EmonHubOEMInterfacer
    [[[init_settings]]]
        com_port = /dev/serial/by-id/usb-Silicon_Labs_CP2102N_USB_to_UART_Bridge_Controller_ea1a6ed75887ec11a374e33719c2d21c-if00-port0
        com_baud = 115200
    [[[runtimesettings]]]
        pubchannels = ToEmonCMS,
        nodename = emonTx4

### This interfacer manages the RFM12Pi/RFM69Pi/emonPi module
#[[RFM2Pi]]
#    Type = EmonHubJeeInterfacer
#    [[[init_settings]]]
#        com_port = /dev/ttyAMA0
#        com_baud = 38400                        # 9600 for old RFM12Pi
#    [[[runtimesettings]]]
#        pubchannels = ToEmonCMS,
#        subchannels = ToRFM12,
#
#        group = 210
#        frequency = 433
#        baseid = 5                              # emonPi / emonBase nodeID
#        calibration = 230V                      # (UK/EU: 230V, US: 110V)
#        quiet = true                            # Disable quite mode (default enabled) to enable RF packet debugging, show packets which fail crc
#        # interval =  300                         # Interval to transmit time to emonGLCD (seconds)


[[MQTT]]

    Type = EmonHubMqttInterfacer
    [[[init_settings]]]
        mqtt_host = 127.0.0.1
        mqtt_port = 1883
        mqtt_user = emonpi
        mqtt_passwd = emonpimqtt2016

    [[[runtimesettings]]]
        pubchannels = ToRFM12,
        subchannels = ToEmonCMS,

        # emonhub/rx/10/values format
        # Use with emoncms Nodes module
        node_format_enable = 1
        node_format_basetopic = emonhub/

        # emon/emontx/power1 format - use with Emoncms MQTT input
        # http://github.com/emoncms/emoncms/blob/master/docs/RaspberryPi/MQTT.md
        nodevar_format_enable = 1
        nodevar_format_basetopic = emon/

        # Single JSON payload published  - use with Emoncms MQTT
        node_JSON_enable = 0
        node_JSON_basetopic = emon/


[[emoncmsorg]]
    Type = EmonHubEmoncmsHTTPInterfacer
    [[[init_settings]]]
    [[[runtimesettings]]]
        pubchannels = ToRFM12,
        subchannels = ToEmonCMS,
        url = https://emoncms.org
        apikey = <removed>
        senddata = 1                    # Enable sending data to Emoncms.org
        sendstatus = 1                  # Enable sending WAN IP to Emoncms.org MyIP > https://emoncms.org/myip/list
        sendinterval= 30                # Bulk send interval to Emoncms.org in seconds

#######################################################################
#######################          Nodes          #######################
#######################################################################

[nodes]

## See config user guide: https://github.com/openenergymonitor/emonhub/blob/emon-pi/conf/emonhub.conf

[[5]]
    nodename = emonpi
    [[[rx]]]
        names = power1,power2,power1pluspower2,vrms,t1,t2,t3,t4,t5,t6,pulsecount
        datacodes = h, h, h, h, h, h, h, h, h, h, L
        scales = 1,1,1,0.01,0.1,0.1,0.1,0.1,0.1,0.1,1
        units = W,W,W,V,C,C,C,C,C,C,p

[[6]]
    nodename = emontxshield
    [[[rx]]]
       names = power1, power2, power3, power4, vrms
       datacode = h
       scales = 1,1,1,1,0.01
       units = W,W,W,W,V

[[7]]
   nodename = emontx4
   [[[rx]]]
      names = power1, power2, power3, power4, vrms, temp1, temp2, temp3, temp4, temp5, temp6, pulse
      datacodes = h,h,h,h,h,h,h,h,h,h,h,L
      scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
      units = W,W,W,W,V,C,C,C,C,C,C,p

[[8]]
    nodename = emontx3
    [[[rx]]]
       names = power1, power2, power3, power4, vrms, temp1, temp2, temp3, temp4, temp5, temp6, pulse
       datacodes = h,h,h,h,h,h,h,h,h,h,h,L
       scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
       units = W,W,W,W,V,C,C,C,C,C,C,p

[[9]]
   nodename = emontx2
   [[[rx]]]
      names = power1, power2, power3, power4, vrms, temp1, temp2, temp3, temp4, temp5, temp6, pulse
      datacodes = h,h,h,h,h,h,h,h,h,h,h,L
      scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
      units = W,W,W,W,V,C,C,C,C,C,C,p

[[10]]
    nodename = emontx1
    [[[rx]]]
       names = power1, power2, power3, power4, vrms, temp1, temp2, temp3, temp4, temp5, temp6, pulse
       datacodes = h,h,h,h,h,h,h,h,h,h,h,L
       scales = 1,1,1,1,0.01,0.01,0.01,0.01,0.01,0.01,0.01,1
       units = W,W,W,W,V,C,C,C,C,C,C,p

[[11]]
    nodename = 3phase
    [[[rx]]]
       names = powerL1, powerL2, powerL3, power4, Vrms, temp1, temp2, temp3, temp4, temp5, temp6, pulse
       datacodes = h,h,h,h,h,h,h,h,h,h,h,L
       scales = 1,1,1,1,0.01,0.01,0.01,0.01,0.01,0.01,0.01,1
       units = W,W,W,W,V,C,C,C,C,C,C,p

[[12]]
    nodename = 3phase2
    [[[rx]]]
       names = powerL1, powerL2, powerL3, power4, Vrms, temp1, temp2, temp3, temp4, temp5, temp6, pulse
       datacodes = h,h,h,h,h,h,h,h,h,h,h,L
       scales = 1,1,1,1,0.01,0.01,0.01,0.01,0.01,0.01,0.01,1
       units = W,W,W,W,V,C,C,C,C,C,C,p

[[13]]
    nodename = 3phase3
    [[[rx]]]
       names = powerL1, powerL2, powerL3, power4, Vrms, temp1, temp2, temp3, temp4, temp5, temp6, pulse
       datacodes = h,h,h,h,h,h,h,h,h,h,h,L
       scales = 1,1,1,1,0.01,0.01,0.01,0.01,0.01,0.01,0.01,1
       units = W,W,W,W,V,C,C,C,C,C,C,p

[[14]]
    nodename = 3phase4
    [[[rx]]]
       names = powerL1, powerL2, powerL3, power4, Vrms, temp1, temp2, temp3, temp4, temp5, temp6, pulse
       datacodes = h,h,h,h,h,h,h,h,h,h,h,L
       scales = 1,1,1,1,0.01,0.01,0.01,0.01,0.01,0.01,0.01,1
       units = W,W,W,W,V,C,C,C,C,C,C,p

[[15]]
    nodename = emontx3cm15
    [[[rx]]]
       names = MSG, Vrms, P1, P2, P3, P4, E1, E2, E3, E4, T1, T2, T3, pulse
       datacodes = L,h,h,h,h,h,l,l,l,l,h,h,h,L
       scales = 1,0.01,1,1,1,1,1,1,1,1,0.01,0.01,0.01,1
       units = n,V,W,W,W,W,Wh,Wh,Wh,Wh,C,C,C,p
       whitening = 1

[[16]]
    nodename = emontx3cm16
    [[[rx]]]
       names = MSG, Vrms, P1, P2, P3, P4, E1, E2, E3, E4, T1, T2, T3, pulse
       datacodes = L,h,h,h,h,h,l,l,l,l,h,h,h,L
       scales = 1,0.01,1,1,1,1,1,1,1,1,0.01,0.01,0.01,1
       units = n,V,W,W,W,W,Wh,Wh,Wh,Wh,C,C,C,p
       whitening = 1

[[17]]
    nodename = emonTx4_17
    [[[rx]]]
        names = MSG, Vrms, P1, P2, P3, P4, P5, P6, E1, E2, E3, E4, E5, E6, T1, T2, T3, pulse
        datacodes = L, h, h, h, h, h, h, h, l, l, l, l, l, l, h, h, h, L
        scales = 1.0, 0.01, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.01, 0.01, 0.01, 1.0
        units = n, V, W, W, W, W, W, W, Wh, Wh, Wh, Wh, Wh, Wh, C, C, C, p

[[19]]
   nodename = emonth1
   [[[rx]]]
      names = temperature, external temperature, humidity, battery
      datacode = h
      scales = 0.1,0.1,0.1,0.1
      units = C,C,%,V

[[20]]
   nodename = emonth2
   [[[rx]]]
      names = temperature, external temperature, humidity, battery
      datacode = h
      scales = 0.1,0.1,0.1,0.1
      units = C,C,%,V

[[21]]
   nodename = emonth3
   [[[rx]]]
      names = temperature, external temperature, humidity, battery
      datacode = h
      scales = 0.1,0.1,0.1,0.1
      units = C,C,%,V

[[22]]
   nodename = emonth4
   [[[rx]]]
      names = temperature, external temperature, humidity, battery
      datacode = h
      scales = 0.1,0.1,0.1,0.1
      units = C,C,%,V

[[23]]
    nodename = emonth5
    [[[rx]]]
       names = temperature, external temperature, humidity, battery, pulsecount
       datacodes = h,h,h,h,L
       scales = 0.1,0.1,0.1,0.1,1
       units = C,C,%,V,p

[[24]]
    nodename = emonth6
    [[[rx]]]
       names = temperature, external temperature, humidity, battery, pulsecount
       datacodes = h,h,h,h,L
       scales = 0.1,0.1,0.1,0.1,1
       units = C,C,%,V,p

[[25]]
    nodename = emonth7
    [[[rx]]]
       names = temperature, external temperature, humidity, battery, pulsecount
       datacodes = h,h,h,h,L
       scales = 0.1,0.1,0.1,0.1,1
       units = C,C,%,V,p

[[26]]
    nodename = emonth8
    [[[rx]]]
       names = temperature, external temperature, humidity, battery, pulsecount
       datacodes = h,h,h,h,L
       scales = 0.1,0.1,0.1,0.1,1
       units = C,C,%,V,p

borpin · 2022-12-22T18:10:45.588Z

mnbf9rca:

I’d like the data to show up on emoncms org. I can see the data from emonTx4 (albeit without labels), but the Glow data never appears:

You see the data on emoncms.org from the TX4 as the data is passing into emonhub and then out to the MQTT Broker on your base. The data from the IHD is going directly to your emonBase (? or emonPi). There currently isn’t an easy way to pass this to emoncms.org unless you have something like Node-RED to do the work.

@TrystanLea, could the MQTT Interfacer be modified such that it subscribes to a different base topic and then uses an HTTP interfacer to send the data on? Can the HTTP Interfacer send on JSON, I can’t remember

mnbf9rca · 2022-12-22T19:09:10.374Z

borpin:

You see the data on emoncms.org from the TX4 as the data is passing into emonhub and then out to the MQTT Broker on your base. The data from the IHD is going directly to your emonBase (? or emonPi)

Gotcha. And if i understand you right, the interfacer which is publishing to emoncms.org is subscribed to something in emonhub, not the MQTT broker.

I thought i saw a system architecture diagram somewhere but i’ve searched and cant find it any more - can you point me to it? I think it’d help me figure this out! Is there a way to push data directly in to emonhub?

BTW I’m not averse to using Node-RED - i’ve never used it before, but as you’ve recommended i, and there’s a guide on the site, t i’ll give it a go.

borpin · 2022-12-22T20:56:22.067Z

mnbf9rca:

And if i understand you right, the interfacer which is publishing to emoncms.org is subscribed to something in emonhub, not the MQTT broker.

emonhub is a multi source/destination tool. It will take data from various sources and then publish it back out by various means.

Often OEM use it to receive data by RFM or Serial and send it back out by MQTT / HTTP. It can send the same received data out to mutliple destinations using multiple protocols. I don’t think the ability to receive from an MQTT Broker, to then send back out is currently supported.

mnbf9rca · 2022-12-22T21:20:45.923Z

ok so let me see if i have this straight…

i should consider the combination of emontx + emonpi/emonbase to be “one thing” i.e. a publisher which sends data to emoncms.org. And if i want other data on emoncms.org, i should just push the data directly there?

r

borpin · 2022-12-22T21:49:25.223Z

The data on your local install of emoncms (be that an emonbase, emonpi or other install) is in no way linked/synced to the data on emoncms.org. You feed the data to each independently.

emonhub can send the same received data to both targets simultaneously.

mnbf9rca · 2022-12-23T22:18:52.796Z

i modified my script to send glow/hildebrand data direct to emoncms.org.