A taste of things to come? - UK smart meter data access

I haven’t been through the process yet, but my impression is once the meters are installed you just ask Hildebrand to connect their IHD/CAD. They then have to verify that you’re actually authorised to request it ( i.e. it’s your HAN they’re connecting to ) and that’s about it. Once authorised, they handle it directly.

not sure if I’m just being dense, but with the Hildebrand CAD, is the realtime usage data via MQTT messages available directly from the CAD on the local wifi network, or is MQTT reliant on subscribing to a Hildebrand server on the internet?

There was a discussion about the communication channels with Smart Meters here Octopus energy : Smart Meter Feed - #29 by clive

[edit]
Re-reading all this, the bit I think I had missed was that you can buy your own CAD/IHD (from Glow for instance) and connect it to your PAN (so you will have more than one CAD connected) as an alternative means to access the meter/tariff data.

The bit that I still see as missing, is the ability to connect something like this stick to a RaspberryPi and then being able to locally use the instantaneous consumption data in your own way. @JaneatGlow care to comment?

My impression is that the only data stream from the Hildebrand CAD directly is to their servers, hence the MQTT will also require subscription to their server. The main reason for this seems to be to keep the IHD/CAD hardware and software as simple as possible, though IMHO it’s not THAT complicated to also have a mechanism to supply locally. Since this data path is directly from the CAD to their servers on the Internet ( i.e. not via the DCC ) we can’t really infer anything about the security of the link other than I’d imagine it has to pass certain standards for the device to be approved and for Hildebrand to connect to the DCC ( which they do to enable pairing etc. )

Again, this is all just my impression and I don’t know with absolute certainty ( yet ) this is how it all fits together. I’ll know more once I actually have one to play with.

1 Like

Would it be useful to wait for @JaneatGlow or @clive to notice these questions and answer them authoritatively?

Yes that other link and the comment from @JaneatGlow suggest that the data must be encrypted.

In the other thread these sorts of data transmissions were described as being to a “Head End”.

I’ll also repeat my comment in that thread (and update it a bit) - this is my understanding of the topology;

Terms

  • CAD - Consumer Access Device
  • IHD - In Home Display (a type of CAD)
  • Head End - SMSO or DCC
  • WAN - Wide Area Network - The method used to transfer Meter data to supplier (eventually) via Head End.
  • PAN - Personal Area Network - A zigbee network connecting the Meters, CAD/IHD and up to 5 remote load control switches.
  • Comms Hub - A device that creates / controls the PAN and access the WAN collecting the

Data flows

Meter → PAN → Comms Hub → WAN → Head End
Head End → WAN → Comms Hub
Meter → PAN → Comms Hub → PAN → CAD/IHD
Meter → PAN → Comms Hub → PAN → CAD/IHD → WAN → 3rd Party (Glow device)

I suspect the Comms Hub may well be built into the CAD/IHD in many cases.

That particular stick is just a headless CAD and the fact it has a USB connector is a red herring - it’s only used for power, not comms. I’ve been bugging @JaneatGlow via email and she’s said they don’t provide data locally nor do they have any plans to. Providing their cloud servers ( or “Head End” which is just a generic term for a server in the cloud supporting many clients ) remains active, and providing you have a stable Internet connection, practically there’s not THAT much difference technically between local access and getting data from the cloud. You can still use their API or MQTT server to get your data onto your PI. ( Obviously there are LOTS of arguments about reliability, security, reliance on others for support, etc etc. )

Head End in this context refers specifically to the SMSO or DCC as Brian noted, I believe. I agree it can be used more generally but here it isn’t. Specifically it distinguishes between the ‘official’ infrastructure and any servers Hildebrand or others may have.

I still think they need to be an approved company (DCC Other User?), but I could be wrong. I modified my data flows to refer to ‘3rd Party’ as that may be more accurate.

Most solutions currently available in UK don’t have local data access. It’s not compatible with their business models. I’m sure someone will come along and do it soon.

1 Like

We have been testing the Rainforest Automation CAD from Canada but its only SMETS1 not SMETS2 compatible atm. It has local data accesss.

1 Like

Half hour TOU tariff data (or traditional block tariff data) is stored in SMETS2 meter registers for the 24 hour period. There is a DUIS function (Read Tariff ) which ‘Other User’ DCC users can access which allows them to read it out. If you used this function as Other User every 12 hours you would be able to capture all the tariff information for a given user.

(SMETS1 is different)

1 Like

This might be of interest to others on this thread.

The comms hub is supplied by the DCC for Smets 2 and for Elec or Dual fuel plugs into a slot on the Electricity meter and for Gas only is a standalone unit.

For smet1 they are supplied with the elecetricty meter by the meter manufacturer.

That is best case - a lot of the time for whatever reason the tariff does not make it to the meter.

Also the DCC network is not reliable that you can do this every 12 hours and expect to get 14 answers a week. - e.g. it was down for some hours last night for ‘system upgrades’.

Our (glow) IHD/CAD is running on an embedded device running an rtos to keep costs down.
There is no scope for running ‘servers’ of any sort on it.
It will respond to ICMP but that, as far as open ports, is it.

I think that is a cop out personally but other opinions are available :slightly_smiling_face:.

Mine is based on my knowledge of our product - what is your based on?

I am back after several months to see what progress there has been - sadly not a lot although this is directed more at the Energy and IHD/CAD industries and not generally people here.

Firstly something I have not seen mentioned in this thread that might be of interest. Bulb and Samsung Smartthings did a deal. This allows Smartthings to access, display and utilise your energy information from Bulb. See the following links.

I have not seen a definitive answer as to whether Smartthings accesses data locally or via Bulb servers but suspect the later.

Pretty much all the CAD devices I have seen officially available in the UK are still limited to only providing data via cloud servers and not locally. This seems to include Chameleon, Geotogether, Hildebrand and Glow. :frowning:

@beaylott
Any news on if/when a UK version of the Eagle200 with SMETS2 support will be available? Also since you seem to have good contacts, is there any chance of RainForest or other makers including Apple HomeKit support? This is now a lot easier to get as no special chip is needed anymore and large portions of HomeKit are now open source to make it easier to develop with and for. Perhaps related would RainForest also add MQTT support?

Note: Since there are a number of smart plugs with energy monitoring capabilities that are HomeKit compatible I would expect that at least a subset of energy reporting capabilities from a CAD should be possible with HomeKit.

The link provided earlier by @borpin to in-home-displays is interesting. It is the first and only case I have seen an actually available Zigbee HAN booster. Normal Zigbee boosters will not work for a HAN. It’s a shame their CAD is yet another one that only provides data access via their cloud and not locally.

I am with OVO and after contacting @JaneatGlow last week about the possibility of real time monitoring I bought one of these:

The IHD/CAD has a very intuitive interface and is a breeze to set up and use.

Since last Saturday (09.May 2020) I also have MQTT access to my data.

I have written a node-red function node to extract relevant meter data, if anyone is interested I can post it here.

Data I have extracted
Wh Now (Instantaneous usage)
Wh today
Wh this week
Wh this month
Wh meter total

The MQTT data is refreshed every 12 seconds!

2 Likes