(SMETS2 + GLOW IHD + MQTT + emonSD) x (Me) = #NULL

[rashdown_online (Roland Ashdown)]

Hello all.

I apologise in advance for previous, this and future posts where I ask for some newbie guidance. But I hope you can help.

I’ve searched the forums (not exhaustively, but for some time) to see if there is any guidance. And there’s a lot of talk which is way above my head. NodeRED-this/JSON-that/config file here there and everywhere

By any chance, has a guide been drafted by those with bigger brains than me which clearly guides through the setup of an MQTT feed from a GLOW IHD to emonCMS (in my case on a Pi)?

I have connected the IHD via MQTT and the Inputs picked it up (or so I thought). NULL values came through. The forum posts I came across start to go into levels of coding, etc. which are far beyond my pay grade, hence the ask here.

Thanks in advance.

Roland.

[TrystanLea]

Hello @rashdown_online Im not aware of such a guide, have you been able to subscribe to the IHD MQTT message in any other way? Just to see what the IHD is providing?

Im guessing that the reason you are seeing NULL values is due to a need to translate the messages send by the IHD into a format that emoncms can read…

[rashdown_online (Roland Ashdown)]

Hi @TrystanLea . Thanks for getting back to me.

I’ve got it successfully connected to my HA instance. Here’s the unformatted JSON:

{“electricitymeter”:{“timestamp”:“2023-01-09T18:05:05Z”,“energy”:{“export”:{“cumulative”:0.000,“units”:“kWh”},“import”:{“cumulative”:4089.213,“day”:11.326,“week”:11.326,“month”:154.376,“units”:“kWh”,“mpan”:“2000006888638”,“supplier”:“EONNext”,“price”:{“unitrate”:0.28455,“standingcharge”:0.43390}}},“power”:{“value”:0.993,“units”:“kW”}}}

Can I ask, is the “/” vitally important in the mqtt topic?

My mqtt interface config is (out of the box):

[[MQTT]]
    Type = EmonHubMqttInterfacer
    [[[init_settings]]]
        mqtt_host = 127.0.0.1
        mqtt_port = 1883
        mqtt_user = emonpi
        mqtt_passwd = emonpimqtt2016
    
    [[[runtimesettings]]]
        pubchannels = ToRFM12,
        subchannels = ToEmonCMS,
        
        # emonhub/rx/10/values format
        # Use with emoncms Nodes module
        node_format_enable = 0
        node_format_basetopic = emonhub/
        
        # emon/emontx/power1 format - use with Emoncms MQTT input
        # http://github.com/emoncms/emoncms/blob/master/docs/RaspberryPi/MQTT.md
        nodevar_format_enable = 1
        nodevar_format_basetopic = emon/
        
        # Single JSON payload published  - use with Emoncms MQTT
        node_JSON_enable = 0
        node_JSON_basetopic = emon/

Thanks again for getting back to me.

Cheers.

[Mike_Henderson]

Yes, I’ve had to do translation when trying to get MQTT messages into EmonCMS from Evogateway. Shunted them through HA and then used Node-red to convert text to numeric, and sometimes TRUE/FALSE to 0/1. Perhaps the same could be done using a template in HA.

Perhaps you need to do the same with the IHD. I’m looking at purchasing that same IHD soon.

I use an Application called MQTT Explorer to help see what messages are available from a device and to get an idea of the format. http://mqtt-explorer.com/

[rashdown_online (Roland Ashdown)]

Hi @Mike_Henderson

Thanks for the note. I have MQTT explore too. However, the real purpose of it is lost on me

@Mike_Henderson

Shunted them through HA and then used Node-red to convert text to numeric, and sometimes TRUE/FALSE to 0/1

The bit that you mention above is where I really do come unstuck.

I’m new on my HA journey, too. I’m aware of Node-RED but never used it as I don’t have any automations at all (yet). I’ve literally got one “smart” device (Shelly Plus1 PM) that I’ve been testing on a light! I’ve been using it really to build up a whole-house energy energy consumption profile in readiness for SolarPV and battery installation.

Are you saying that is should follow this logic: GLOW (mqtt) → HA (sensor in configuration.yaml) → Node-RED → ???. It’s that last bit I’ve no idea about.

What I was hoping to do was use emon as the main “hub” for all things energy monitoring, and then have an integration in HA for emon (which I’m lead to believe creates sensors for all the emon devices/inputs within HA.)

It might all be a moot point as I’m hoping to have an SDM230 Modbus meter installed, just after the SMETS2, which will feed emon. I just was hoping to test emon out a bit before diving in with the expense of SDMs through out my configuration. And leveraging the power of the feeds in emon to really see what’s going down.

Here’s an extract from MQTT Explorer:

{“electricitymeter”:{“timestamp”:“2023-01-09T19:01:15Z”,“energy”:{“export”:{“cumulative”:0.000,“units”:“kWh”},“import”:{“cumulative”:4090.098,“day”:12.212,“week”:12.212,“month”:155.262,“units”:“kWh”,“mpan”:“2000006888638”,“supplier”:“EONNext”,“price”:{“unitrate”:0.28455,“standingcharge”:0.43390}}},“power”:{“value”:0.996,“units”:“kW”}}}

Thanks again everyone.

Roland.

[Mike_Henderson]

I just use MQTT Explorer to see what’s available and maybe the format of the data. I like it because of the drill-down view it gives me.

I’ll add a binary sensor to EmonPI from Evogateway into HA and pass it on to EmonPI right now.

So, I use explorer to find it.

It’s the room window open status and its value isn’t numeric, and when I get it into HA it’ll be the text “true” or “false” and if I pass that to EmonPI it’ll be ‘nan’ or some problem. I’m sure there’s another way of doing this but this is what I do.

Evogateway already has an MQTT connection HA but I still have to create my Entity in the HA /config/configuration.yaml MQTT section.

So I now have an Entity for the sensor in HA.

I now create a flow in node-red to convert and pass this on to EmonPI

The trigger block will pass on the status of the entity only when it changes.
The function block changes “true” to 1 and “false” to 0.
The MQTT send block passes it on to EmonPI.

The MQTT node needs to have the details of your EmonPI added as a server.
The function block can contain any code you need. It could be coded to rescale an analog value. If its a calculation that I struggle to do within the EmonPI I sometimes pass the input from EmonPI to HA and then pass the result back to EmonPI.

And here is the input that appeared in Emon PI in a format I could use on say a dashboard.

[rashdown_online (Roland Ashdown)]

@Mike_Henderson

This is exceptionally helpful! Thank you. I’m now a lot clearer on the logic. I’ve never used NodeRED before so this has helped clear things up a lot for this specific scenario.

I’ve already got a bunch of MQTT sensors in my configuration.yaml for the GLOW meter/integration. And this works really quite well. So I’ll leverage those.

I noticed that you’re about to get a GLOW IHD. If it helps, here’s the info I’ve got in my yaml (I don’t have gas, so have excluded the gas meter elements. If you have gas, too, then add a lot of the same for gas but change the “state_topic” from “electricity meter” to “gasmeter”:

mqtt:
  sensor:  
    - name: "SMETS2: Export"
      unique_id: "smart_meter_electricity_export"
      state_topic: "glow/[yourdeviceidhere]/SENSOR/electricitymeter"
      device_class: "energy"
      unit_of_measurement: "kWh"
      state_class: "total_increasing"
      value_template: "{{ value_json['electricitymeter']['energy']['export']['cumulative'] }}"
      icon: "mdi:flash"
    - name: "SMETS2: Import"
      unique_id: "smart_meter_electricity_import"
      state_topic: "glow/[yourdeviceidhere]/SENSOR/electricitymeter"
      device_class: "energy"
      unit_of_measurement: "kWh"
      state_class: "total_increasing"
      value_template: >
        {% if value_json['electricitymeter']['energy']['import']['cumulative'] == 0 %}
          {{ states('sensor.smart_meter_electricity_import') }}
        {% else %}
          {{ value_json['electricitymeter']['energy']['import']['cumulative'] }}
        {% endif %}
      icon: "mdi:flash"
    - name: "SMETS2: Import (Today)"
      unique_id: "smart_meter_electricity_import_today"
      state_topic: "glow/[yourdeviceidhere]/SENSOR/electricitymeter"
      device_class: "energy"
      unit_of_measurement: "kWh"
      state_class: "measurement"
      value_template: >
        {% if value_json['electricitymeter']['energy']['import']['day'] == 0 
          and now() > now().replace(hour=0).replace(minute=1).replace(second=0).replace(microsecond=0) %}
          {{ states('sensor.smart_meter_electricity_import_today') }}
        {% else %}
          {{ value_json['electricitymeter']['energy']['import']['day'] }}
        {% endif %}
      icon: "mdi:flash"
    - name: "SMETS2: Import (This week)"
      unique_id: "smart_meter_electricity_import_week"
      state_topic: "glow/[yourdeviceidhere]/SENSOR/electricitymeter"
      device_class: "energy"
      unit_of_measurement: "kWh"
      state_class: "measurement"
      value_template: "{{ value_json['electricitymeter']['energy']['import']['week'] }}"
      icon: "mdi:flash"
    - name: "SMETS2: Import (This month)"
      unique_id: "smart_meter_electricity_import_month"
      state_topic: "glow/[yourdeviceidhere]/SENSOR/electricitymeter"
      device_class: "energy"
      unit_of_measurement: "kWh"
      state_class: "measurement"
      value_template: "{{ value_json['electricitymeter']['energy']['import']['month'] }}"
      icon: "mdi:flash"
    - name: "SMETS2: Import Unit Rate"
      unique_id: "smart_meter_electricity_import_unit_rate"
      state_topic: "glow/[yourdeviceidhere]/SENSOR/electricitymeter"
      device_class: "monetary"
      unit_of_measurement: "GBP/kWh"
      state_class: "measurement"
      value_template: "{{ value_json['electricitymeter']['energy']['import']['price']['unitrate'] }}"
      icon: "mdi:cash"
    - name: "SMETS2: Import Standing Charge"
      unique_id: "smart_meter_electricity_import_standing_charge"
      state_topic: "glow/[yourdeviceidhere]/SENSOR/electricitymeter"
      device_class: "monetary"
      unit_of_measurement: "GBP"
      state_class: "measurement"
      value_template: "{{ value_json['electricitymeter']['energy']['import']['price']['standingcharge'] }}"
      icon: "mdi:cash"
    - name: "SMETS2: Power"
      unique_id: "smart_meter_electricity_power"
      state_topic: "glow/[yourdeviceidhere]/SENSOR/electricitymeter"
      device_class: "power"
      unit_of_measurement: "kW"
      state_class: "measurement"
      value_template: "{{ value_json['electricitymeter']['power']['value'] }}"
      icon: "mdi:flash"

template:
  sensor:
    - name: "SMETS2: Cost (Today)"
      unique_id: smart_meter_electricity_cost_today
      device_class: monetary
      unit_of_measurement: "GBP"
      state_class: "total_increasing"
      icon: mdi:cash
      state: "{{ (
          states('sensor.smart_meter_electricity_import_today') | float 
          * states('sensor.smart_meter_electricity_import_unit_rate') | float 
          + states('sensor.smart_meter_electricity_import_standing_charge') | float
        ) | round(2) }}"

I’ll let you know how I get on, and if it’s OK I might need to ask some more questions specifically around the pulling together of teh appropriate sensor data and preparing it to be published to emon.

Thanks again.

Roland.

[Mike_Henderson]

Wouldn’t it be easier just to use the EmonPI with a CT to collect the usage figures, they’re more or less in real-time.

I only wanted the new IHD to gain access to gas figures because I’m having to carry out some cunning calculations in the EmonPI to estimate it at the moment.

[rashdown_online (Roland Ashdown)]

@Mike_Henderson

Wouldn’t it be easier just to use the EmonPI with a CT to collect the usage figures, they’re more or less in real-time.

You’re absolutely right

But I’m using a re-purposed RPi-3 and have no CTs.

And besides, I’m punishing myself for making the decision all those years ago to go down the “tin & wires” route instead of software…

Thanks again.

[borpin (Brian Orpin)]

@rashdown_online

I’ve already got a bunch of MQTT sensors in my configuration.yaml for the GLOW meter/integration. And this works really quite well. So I’ll leverage those.

You can either use the MQTT Statestream - Home Assistant integration or an automation to update on change.

[rashdown_online (Roland Ashdown)]

Thanks @borpin

Every day is a skool day!

More things to look at

Thanks again.