Continuing the discussion from Electricity meters with HAN interface:
and A taste of things to come? - UK smart meter data access
Glow have released an update so their IHDs have local MQTT 
Blog for more details
A new device without a display is close, but the price will be similar to the units with a display.
Probably a better place for this.
I bought one of the Glow CAD’s last week and this is the information it gives out from the electricity meter roughly every 6 seconds.
{“electricitymeter”:{“timestamp”:“2022-06-24T13:02:29Z”,“energy”:{“export”:{“cumulative”:0,“units”:“kWh”},“import”:{“cumulative”:4595.563,“day”:10.464,“week”:85.675,“month”:512.937,“units”:“kWh”,“mpan”:"***********",“supplier”:“Octopus Energy”,“price”:{“unitrate”:0.3,“standingcharge”:0.4457}}},“power”:{“value”:0.623,“units”:“kW”}}}
Gas meter data is transmitted at the same intervals from the CAD but it is only updated from the meter itself every 30 minutes
{“gasmeter”:{“timestamp”:“2022-06-24T13:13:30Z”,“energy”:{“export”:{“cumulative”:0,“units”:“kWh”},“import”:{“cumulative”:6728.026,“day”:0,“week”:9.781,“month”:87.295,“units”:“kWh”,“mprn”:“EMPTY”,“supplier”:"—",“price”:{“unitrate”:0.07276,“standingcharge”:0.2611}}},“power”:{“value”:0,“units”:“kW”}}}
I had asked Hildebrand whether the CAD can send data detailing the night and day usages as we have Economy 7, I had an initial reply from Jane to say she will check with the software people but thinks this is a limitation of the Zigbee protocol.
Interesting to know where you got that JSON from.
Because it uses the in/out quotes (not sure the UTF code for them) the Json Validation fails. I had to copy it and replace them to get it to validate. If that really is how it is coming out of the CAD, it’ll need to be changed by Glow @JaneatGlow.
{
"electricitymeter": {
"timestamp": "2022-06-24T13:02:29Z",
"energy": {
"export": {
"cumulative": 0,
"units": "kWh"
},
"import": {
"cumulative": 4595.563,
"day": 10.464,
"week": 85.675,
"month": 512.937,
"units": "kWh",
"mpan": "***********",
"supplier": "Octopus Energy",
"price": {
"unitrate": 0.3,
"standingcharge": 0.4457
}
}
},
"power": {
"value": 0.623,
"units": "kW"
}
}
}
Does the day/night info appear on the CAD/IHD itself? If so, it is their decoding, if not, probably a limitation. Can you see this on the Bright App?
Looking at the pasted JSON output, that ‘strings’ seems to be enclosed in Unicode codes U+201C & U+201D (ASCII and Unicode quotation marks).
I’m not enough of a guru to know the difference between Unicode and UTF-8 but this could be the issue.
It could also simply be the way that the example JSON was sourced and pasted.
I think JSON strings should be delimited by U+0022 i.e. a single byte UTF-8 code.
I’ve commented on their forum about the use of Z (Zulu) in the timestamp - really should be avoided and +00:00 used instead. Not all languages convert the Z correctly to a UNIX timestamp.
Well spotted @borpin re JSON.
It was taken from a debug output, i had not noticed it was after a JSON decode block so was the output of an array., raw JSON as follows:
{"gasmeter":{"timestamp":"2022-06-24T18:18:58Z","energy":{"export":{"cumulative":0.000,"units":"kWh"},"import":{"cumulative":6728.026,"day":0.000,"week":9.781,"month":87.295,"units":"kWh","mprn":"EMPTY","supplier":"---","price":{"unitrate":0.07276,"standingcharge":0.26110}}},"power":{"value":0.000,"units":"kW"}}}
{"electricitymeter":{"timestamp":"2022-06-24T18:19:16Z","energy":{"export":{"cumulative":0.000,"units":"kWh"},"import":{"cumulative":4599.216,"day":14.117,"week":89.328,"month":516.590,"units":"kWh","mpan":"********","supplier":"Octopus Energy","price":{"unitrate":0.30000,"standingcharge":0.44570}}},"power":{"value":0.570,"units":"kW"}}}I’ve trawled through the smart meter Zigbee document provided via a link from Jane at Glow and the TOU (Time Of Use) registers look available over the protocol if the user wishes to use them which in the case of my particular meter is 48x2 registers, up to one for each half hour of the day for use with the likes of Octopus Agile tarrif, one register for export one for import.
The CAD/IHD only shows accumulated total. The Bright app shows half hourly accumulations which i believe are transmitted over DCC when the meter would change between each TOU register if enabled.
I expect therefore the costs are calculated by the billing software then.
Hi. This discussion dropped into my email thanks to @borpin tagging me - as a reminder, we don’t have time to keep up with all the forums but I thought I’d add a few clarifications.
Just picking up on a couple of points that I checked with my technical colleagues - the quote issue is due to the fact that the email we send out with the guidance formats the quotes.
Timezone - might be worth checking with ISO 8601 - Wikipedia as apparently Z is mandated for UTC.
Costs - suppliers all use their billing systems to calculate the bills - not what is on the meter.
@spike3312 - Bright retrieves the consumption from the meter and separately the rates on the meter (including timeslots) and then calculates the cost of consumption dependent on those two figures and time of day. The IHD retrieves the calculated cost from the meter directly. Bright does not so that we can separate out tariff and provide cost of consumption predictions if you think of changing to a new tariff; our smart tariff prototype developed last year (dummy tariffs) is an example of how that can work - https://smarttariffsmartcomparison.org/home. Note that you can login with your Bright credentials.
Hence the tag 
No, it isn’t mandated, it is an option @JaneatGlow - Wikipedia is not authoratitive!
The problem is this → Treating of time in Emoncms - UTC or BST - #10 by borpin
TL;DR - Python doesn’t parse the Z correctly (it assumes local time). Therefore the ‘safe’ way is to use +00:00.
Am I following this correctly - your language of choice, using some library or other, does not parse a perfectly valid representation of a datetime?
Surely the answer is - use something that actually works?
The ISO for Date Time has so many differing parts to it, I’m sure I could build a vaild ISO DTG that would break most parsers!
Simplest and safest route is to use +00:00 which is valid.
Well all of the Smart Meter infrastructure in the UK uses Z, all our APIs use Z so it will not be changing.
Well when folk complain it is an hour out, we’ll know why.
They don’t complain anywhere else, on the APIs or on MQTT server side or local.
So I think it will just be you.
Nope anyone using Python who doesn’t know will experience it.
I have a Hildebrand Glow CAD, and two Secure SMETS1 meters for electric and gas provided by Ecotricity. I don’t have local MQTT but have remote MQTT which can be access using:
$ mosquitto_sub -h glowmqtt.energyhive.com -p 8883 -u '[email protected]' -P 'VerySecret' -t 'SMART/HILD/AABBCCDDEEFF'
The JSON looks broadly similar but has mostly numeric keys like the following:
{
"elecMtr":
{
"0702":
{
"03":
{
"01":"000001",
"05":"43",
[...]
(I have modified this to make it more human readable. The original is on a single long line with no spaces).
Is there an integration to read either the local or remote MQTT?
Not directly in emoncms.
Do you have Node-RED running or Home Assistant? If so, either will do it reasonable easily.
I have just installed Node-RED but I have not used it before …
I’m fairly new to OpenEnergyMonitor. My assumption was that integrating data from MQTT sources might use the emonhub MQTT Interfacer? If the MQTT source publishes suitable JSON I assumed it should be fairly easy.
However, the JSON from glowmqtt.energyhive.com is deeply nested and, in my case, contains data from two meters. Something would be required to split the data for each meter, translate the numeric keys to something more useful and flatten the nested data.
I’m also interested in integrating MQTT data from a Raspberry Pi Pico W where there is full control of the JSON generated.
On the same Pi as emoncms? If so that is generally a bad idea as the emon image is optimised for emoncms.
It is very simple in Node-RED.
Use a subscribe MQTT node to read the data
Use a function node to extract the consumption data and add a timestamp (such as)
var newmsg = {};
newmsg.payload = {};
const time = Date.parse(msg.payload.electricitymeter.timestamp)/1000;
const electric = msg.payload.electricitymeter.energy.import.cumulative * 1000;
newmsg.payload = JSON.stringify({time, electric});
return newmsg;
I wanted Wh and seconds (hence the 1000s)
Then an MQTT publish node to send it to the emonPi MQTT Broker with the base topic of emon/
Assumptions are the mother of all Foul ups!
No, send the data to the same MQTT Broker the emonPi is subscribed to (usually the local one for a standard install) with the base topic of emon/ and it is actually a PHP script that picks data up from MQTT - nothin to do with emonhub.
Again just publish to the emonPI broker with the right base topic.
https://docs.openenergymonitor.org/emoncms/mqtt.html#mqtt-publishers