All DCC users sign up to very strict privacy rules and are heavily audited annually including all companies that manage CAD data.
To which you can add CAD/IHD → 3rd party such as Hildebrand who are licensed by DCC to collect data which they can then provide to their users via the Bright App, API and mqtt.
You must have a Glow CAD (in the case of Hildebrand) which has a wifi connection to provide the data path to their servers.
Also interested.
Do CADs act as ZigBee repeaters to extend the range? At the moment I’m quite limited where I can put my IHD in the house due to distance to the meter.
How well do multiple Zigbee networks operate? I already have one for lighting. Is that likely to be part of my range issues?
Hi all,
This thread seems to have been a little quiet … i wondered if the suggested API/MQTT integration had been moved forward by anyone?
I’ve just bought a glowstick and got it set up (and pending the final config of my smart meter) … so happy to help in testing
1 Like
Do you have a citation for that? It seems pretty fundamental to the debate on this thread.
Looking at the Electricity Supply Regulations (section 49.4, parts (d) and (e)), the supplier has an obligation to allow the consumer access to their data:
(d) on request of the Customer at the relevant premises, it both establishes and thereafter maintains a connection through the HAN Interfaces between the Smart Metering System and each Relevant Consumer Device that is located within a part of the premises to which the HAN extends and is the subject of the request; and
(e) the connection established in accordance with paragraph (d) enables that Customer to access (at any time and, in the case of the Domestic Customer, free of charge) by means of each
Relevant Consumer Device, the Customer Information that:
(i) is capable of being stored in or held by the Smart Metering System (or any part of it); and
(ii) the Smart Metering System (or any part of it) is capable of sending to the Relevant Consumer
Device.
From that it appears that the supplier has to grant free of charge access to any data the meter can produce and the CAD can receive. If the CAD receives real-time data, the supplier has to make it available. It doesn’t mention anything about cloud services for this purpose and in-fact such a provision could discriminate against users who have no internet access (or have shared provision to which they are not allowed to connect new devices) so there has to be a non-cloud of providing access to your own data.
I’ve just stumbled across yr post dated 12 Aug and by now you may have a solution re API/MQTT.
If not take a look at a post I made on 7 Sep …
My script has been running for weeks now. The Hildebrand GlowStick has replaced an emonTx, RPi, AC/AC adapter, pulse counter & CT’s I previously had installed.
That is really interesting, especially,
Not sure even Octopus have implemented that level of access. “Any information that is capable of being stored…”
Hi John
Yes, in the end I used nodered running on a pi and a simple nodered setup - image below
Once mqtt enabled on hildebrand glowstick and you have credentials from them, this just works.
Nodered setup on pi is simple as well.
giving the following type of granular data
Happy to share the nodered setup/code if you wish
Julian
Just to confirm, is the data directly from the stick or are you retrieving it after it has been posted to the cloud?
What are you able to access; is it the current usage or can you see history? Is it only electricity or gas you see gas (if you have it). Is it only import or can you see export (even if your export is zero)?
From my reading of the regulations, you should be able to see everything but acknowledge that Hildebrand are facilitating access but aren’t the licensee so don’t themselves hold the responsibility to make the data available.
Julian
I’d be interested in some samples: I need an input into my emonpi/ emoncms from my SMETS2: This seems to be an interesting way to get rates and useage into the system using glowmarkt, your nodered should give me some useful hints (just being a bit lazy, guess I could work it out with lots of trial and error!).
Unless someone has done all the work to do what I want - to get a feed into the emonpi!
Many thanks!
1 Like
My node red flow looks like this … you’ll need to sort out your IP addresses, glowstick mac address etc … cut and paste between the ‘[]’ into your nodered
GUI
[
{
"id": "a31e89f5.a102e8",
"type": "tab",
"label": "glowstickMQTTEmoncms",
"disabled": false,
"info": ""
},
{
"id": "fbe2b82f.8d1248",
"type": "mqtt in",
"z": "a31e89f5.a102e8",
"name": "",
"topic": "SMART/HILD/xxxxxxxxxxxx", ### the unique id / mac for your glowstick ###
"qos": "2",
"datatype": "auto",
"broker": "87923771.022e08",
"x": 170,
"y": 280,
"wires": [
[
"4729f673.a2afd8",
"21dcf206.96247e"
]
]
},
{
"id": "a0638792.bb68f8",
"type": "debug",
"z": "a31e89f5.a102e8",
"name": "",
"active": false,
"tosidebar": true,
"console": false,
"tostatus": false,
"complete": "true",
"targetType": "full",
"statusVal": "",
"statusType": "auto",
"x": 490,
"y": 280,
"wires": []
},
{
"id": "4729f673.a2afd8",
"type": "json",
"z": "a31e89f5.a102e8",
"name": "",
"property": "payload",
"action": "",
"pretty": false,
"x": 190,
"y": 360,
"wires": [
[
"93d4ac29.88172"
]
]
},
{
"id": "93d4ac29.88172",
"type": "function",
"z": "a31e89f5.a102e8",
"name": "extract elements",
"func": "msg.meterSerial = msg.payload.elecMtr[\"0702\"][\"03\"][\"08\"]\nmsg.timeStamp = msg.payload.gmtime\nmsg.todayConsumptDelivered0401 = parseInt(msg.payload.elecMtr[\"0702\"][\"04\"][\"01\"],16)\n//msg.instantDemand0400 = parseInt(msg.payload.elecMtr[\"0702\"][\"04\"][\"00\"],16) - value is hex representation of an unsigned INT 24 bits\nmsg.currSummReceived0001 = parseInt(msg.payload.elecMtr[\"0702\"][\"00\"][\"01\"],16)\nmsg.currSummDelivered0000 = parseInt(msg.payload.elecMtr[\"0702\"][\"00\"][\"00\"],16)\n\nmsg.instantDemand0400 = parseInt(msg.payload.elecMtr[\"0702\"][\"04\"][\"00\"],16)\nif ((msg.instantDemand0400 & 0x800000) > 0) {\n msg.instantDemand0400 = msg.instantDemand0400 - 0x1000000\n}\n\nreturn msg;",
"outputs": 1,
"noerr": 0,
"initialize": "",
"finalize": "",
"x": 330,
"y": 420,
"wires": [
[
"a0638792.bb68f8",
"9ddc9a28.91f898"
]
]
},
{
"id": "9ddc9a28.91f898",
"type": "function",
"z": "a31e89f5.a102e8",
"name": "Send data as emonCMS Input",
"func": "var arrayobj = msg.payload.results\nvar newmsg = {}\nvar apikey =\"xxxxxxxxxxxxxxxxxxxxxxxx\"\n\n// for host if you get a 404 error, this might require 'http://x.x.x.x/emoncms/input/post?'\n// should also work for emoncms.org\n//var host = \"http://192.168.1.xxx/emoncms/input/post?\"\n//http://192.168.1.xxx/input/post?time=1598996829&node=1&csv=100,200,300\n\nvar host = \"http://192.168.1.xxx/emoncms/input/post?\"\nvar node_name = \"glowstick\"\n\n newmsg.url = host\n newmsg.url += \"time=\" + msg.timeStamp\n newmsg.url += \"\\&node=\" + node_name\n newmsg.url += \"\\&csv=\" + msg.instantDemand0400\n newmsg.url += \"\\&apikey=\" + apikey\n \n// newmsg.url += \"\\&fulljson={ \\\"time\\\": \\\"\" + escape(arr[index].interval_end) + \"\\\", \"\n// newmsg.url += \"\\\"consumption\\\": \" + arr[index].consumption + \"}\"\n// newmsg.url += \"\\&apikey=\" + apikey\n// bulk?data=[[-2,%22octopusGo%22,1000],[0,%22octopusGo%22,1000]]&time=1598596010\n //newmsg.url += \"&data=[[0,\\\"\" + node_name + \"\\\",\" + arr[index].consumption + \"]]\"\n //msg.dateconv = new Date(arr[index].interval_end);\n //msg.millisec = Date.parse(arr[index].interval_end);\n //newmsg.url += \"\\&time=\" + msg.millisec;\n ////newmsg.url += \"\\&csv=\" + arr[index].consumption;\n //newmsg.url += \"\\&fulljson={ \\\"time\\\": \\\"\" + escape(arr[index].interval_end) + \"\\\", \"\n //newmsg.url += \"\\\"consumption\\\": \" + arr[index].consumption + \"}\"\n\nreturn newmsg;\n",
"outputs": 1,
"noerr": 0,
"initialize": "",
"finalize": "",
"x": 630,
"y": 420,
"wires": [
[
"b07cc45e.0b4938",
"bf9d361f.ca5a08"
]
]
},
{
"id": "b07cc45e.0b4938",
"type": "http request",
"z": "a31e89f5.a102e8",
"name": "",
"method": "GET",
"ret": "txt",
"paytoqs": "query",
"url": "",
"tls": "",
"persist": false,
"proxy": "",
"authType": "",
"x": 710,
"y": 520,
"wires": [
[]
]
},
{
"id": "bf9d361f.ca5a08",
"type": "debug",
"z": "a31e89f5.a102e8",
"name": "",
"active": false,
"tosidebar": true,
"console": false,
"tostatus": false,
"complete": "true",
"targetType": "full",
"statusVal": "",
"statusType": "auto",
"x": 790,
"y": 340,
"wires": []
},
{
"id": "21dcf206.96247e",
"type": "debug",
"z": "a31e89f5.a102e8",
"name": "",
"active": false,
"tosidebar": true,
"console": false,
"tostatus": false,
"complete": "false",
"statusVal": "",
"statusType": "auto",
"x": 420,
"y": 200,
"wires": []
},
{
"id": "87923771.022e08",
"type": "mqtt-broker",
"z": "",
"name": "glowstick hildebrand",
"broker": "glowmqtt.energyhive.com",
"port": "8883",
"tls": "",
"clientid": "",
"usetls": true,
"compatmode": false,
"keepalive": "60",
"cleansession": true,
"birthTopic": "",
"birthQos": "0",
"birthPayload": "",
"closeTopic": "",
"closeQos": "0",
"closePayload": "",
"willTopic": "",
"willQos": "0",
"willPayload": ""
}
]
1 Like
Here’s the Python script that I’m using:
# PURPOSE: To input Smart Meter data into emoncms (data obtained via a Hildebrand Glow Stick https://www.hildebrand.co.uk/our-products/glow-stick-wifi-cad/ )
# With due acknowledgement to ndfred - a contributor to the Glowmarkt forum
# https://gist.github.com/ndfred/b373eeafc4f5b0870c1b8857041289a9
# Developed and tested on a Raspberry Pi running the Oct 2019 emon image updated to ver 10.2.6
# Electricity only - not Gas
# HOW TO ...
# IMPORTANT - Install mosquitto-clients with: sudo apt-get install mosquitto-clients XXXXXXXXXXXXX !!!!!!!!!
# The script will create an INPUT node to receive the data
# Copy this script file to /home/pi and make it executable with: chmod +x /home/pi/JB-glowmqtt.py # using the correct script name
# Run the script with: /usr/bin/python3 /home/pi/JB-glowmqtt.py # using the correct script name
# All being well, Smart meter data will appear in emoncms webpage Inputs refreshing every 10 secs - Power Now(W) and Daily & CUM Energy(kWh)
# Create FEEDS using Log to Feed and add UNITS (pencil drop-down) to each
# IMPORTANT NOTE ...
# Data from this script is input to emoncms via an http API.
# The EmonHubEmoncmsHTTPinterfacer is not required - it cannot send this script data to watchman
# FINALLY ONCE THE SCRIPT RUNS OK: Create the glow.service and enable it so the script runs on boot up as follows:
# Do: CTRL-C to stop the script then - Do: sudo nano /etc/systemd/system/glow.service and copy & paste in the following BUT adjust the script name ...
"""
[Unit]
Description=Glow Stick service
After=network.target
After=mosquitto.service
StartLimitIntervalSec=0
[Service]
Type=simple
Restart=always
RestartSec=1
User=pi
ExecStart=/usr/bin/python3 /home/pi/JB-glowmqtt.py
[Install]
WantedBy=multi-user.target
"""
# Then save & exit and to ensure the glow.service runs on boot up - Do: sudo systemctl enable glow.service
# AS A VERY LAST CHECK - Do: sudo reboot then SSH in again and check the service status with: systemctl status glow.service
# Finally close the SSH terminal and the script/service will continue to run surviving any future reboots
# ===============================================================================
import datetime
import logging
import json
import paho.mqtt.client as mqtt # paho-mqtt is already installed in emon
import requests
# Glow Stick configuration
GLOW_LOGIN = "xxxxxxxxxxxxxxxxxx"
GLOW_PASSWORD = "xxxxxxxxxxxxxxxxx"
GLOW_DEVICE_ID = "xxxxxxxxxxxxxxxxx"
# Emoncms server configuration
emoncms_apikey = "xxxxxxxxxxxxxxxxxxxxxxxx" # Change as appropriate
emoncms_server = "http://127.0.0.1"
node = "Glow Stick" # Name of the node created to receive the INPUT data
# Name each of the data inputs associated with the newly created node
di1 = "Power Now" # ref E_NOW below
di2 = "Daily Energy" # ref E_DAY below
di3 = "CUM Energy" # ref E_METER below
def on_connect(client, _userdata, _flags, result_code):
if result_code != mqtt.MQTT_ERR_SUCCESS:
logging.error("Error connecting: %d", result_code)
return
result_code, _message_id = client.subscribe("SMART/HILD/" + GLOW_DEVICE_ID)
if result_code != mqtt.MQTT_ERR_SUCCESS:
logging.error("Couldn't subscribe: %d", result_code)
return
logging.info("Connected and subscribed")
def on_message(_client, _userdata, message):
payload = json.loads(message.payload)
current_time = datetime.datetime.now().strftime("%H:%M:%S")
electricity_consumption = int(payload["elecMtr"]["0702"]["04"]["00"], 16)
if electricity_consumption > 10000000: electricity_consumption = electricity_consumption - 16777216 # Added JB - hex FFFFFF is 16777215 in unsigned 24 bit but -1 in signed 24 bit
E_NOW = electricity_consumption # Added JB
electricity_daily_consumption = int(payload["elecMtr"]["0702"]["04"]["01"], 16)
# electricity_weekly_consumption = int(payload["elecMtr"]["0702"]["04"]["30"], 16) # Data not provided by GLOW
# electricity_monthly_consumption = int(payload["elecMtr"]["0702"]["04"]["40"], 16) # Data not provided by GLOW
electricity_multiplier = int(payload["elecMtr"]["0702"]["03"]["01"], 16)
electricity_divisor = int(payload["elecMtr"]["0702"]["03"]["02"], 16)
electricity_meter = int(payload["elecMtr"]["0702"]["00"]["00"], 16)
electricity_daily_consumption = electricity_daily_consumption * electricity_multiplier / electricity_divisor
E_DAY = electricity_daily_consumption # Added JB
# electricity_weekly_consumption = electricity_weekly_consumption * electricity_multiplier / electricity_divisor
# electricity_monthly_consumption = electricity_monthly_consumption * electricity_multiplier / electricity_divisor
electricity_meter = electricity_meter * electricity_multiplier / electricity_divisor
E_METER = electricity_meter # Added JB
assert(int(payload["elecMtr"]["0702"]["03"]["00"], 16) == 0) # kWh
logging.info("Reading at %s", current_time)
logging.info("electricity consumption: %dW", electricity_consumption)
logging.info("daily electricity consumption: %.3fkWh", electricity_daily_consumption)
# logging.info("* weekly electricity consumption: %.3fkWh", electricity_weekly_consumption)
# logging.info("* monthly electricity consumption: %.3fkWh", electricity_monthly_consumption)
logging.info("electricity meter: %.3fkWh", electricity_meter)
# logging.info("Full payload: %s", json.dumps(payload, indent=2)) # Don't need this info printed
data1 = E_NOW
data2 = E_DAY
data3 = E_METER
# Send data to emoncms
dev_data = {di1: data1, di2: data2, di3: data3}
data = {
'node': node,
'data': json.dumps (dev_data),
'apikey': emoncms_apikey
}
response = requests.post(emoncms_server+"/input/post", data=data)
def loop():
logging.basicConfig(level=logging.DEBUG, format='%(message)s')
client = mqtt.Client()
client.username_pw_set(GLOW_LOGIN, GLOW_PASSWORD)
client.on_connect = on_connect
client.on_message = on_message
client.connect("glowmqtt.energyhive.com")
client.loop_forever()
if __name__ == "__main__":
loop()
This shows the INPUTS to emoncms:
Here are my FEEDS:
I’m a big fan of GlowStick.
Hope this helps
1 Like
Is that a SMETS 1 stick?
@borpin
Mid 2020, Octopus Energy fitted a SMETS1 SECURE Sprint 211 because SMETS2 3 phase meters are not yet available.
When I made my first enquiry to Hildebrand/GlowStick, I got the following response:
That is good news as we do have the ability to remotely do the Open/Join HAN instruction with Octopus.
We are about to start testing with SMETS2 polyphase meters and have never connected to a SMETS1 variant (as they are so rare!). What I’d like to propose, if that is OK with you, is that we send you one of our GlowSticks which works with a single phase SMETS1 Secure meter and see what happens with the polyphase - so no risk for you.
1 Like
@johnbanks for your script, did you ever create an extended version that included gas data as well as electric? I’ve just found out about this project, have a Hildebrand CAD and wanted to try monitoring gas as well as electric
@Townsmcp
The installation I’m involved with is rural with no gas supply and so I have no plans to extend my script.
A couple of comments tho’ …
ndfred’s script does include gas
https://gist.github.com/ndfred/b373eeafc4f5b0870c1b8857041289a9
So it should be possible for you to add gas to my script.
Also, I do not have a Hildebrand CAD but their GlowStick and so I don’t know whether my script works with a CAD. You could ask Hildebrand?
Regards
Ah ok. Thanks for clarifying that about the script
Going back to the original topic, has anyone got a finished method to get the above data into emoncms, noting that it delivers a day’s worth of half-hourly data a day late?
I suspect I need to do something with input/bulk but I don’t understand the help on time offsets and how time=1664194418 makes -6 mean 1387730121 seconds since 0 second and not 166194412.
It looks as if Trystan’s tried to be far too clever - he’s picked up the Time Now and put it into time=1664194418, whereas I think “1387730121 seconds since 0 second” is when he wrote the help page - December 22, 2013
@TrystanLea An elephant-sized oopsie here!
1 Like
1 Like