Yes this is why I think we need to produce a CAD sub-system (which itself would be largely open source but subject to certification) which can then be used more easily with open source platforms.
I think the primary aim should be to provide a gateway device to extract the data - a hat - that can be used however the user wants. If someone then wants to use that to provide a full platform they can.
I’m guessing that effectively this has been done by the devices that show the current consumption inside the house. Shame they did not feel the urge to provide the means to tap into that data from there. A nice little API to connect to the consumer display and problem solved.
[edit]
went back to the beginning of the thread - full circle 
I can’t speak to the reason it’s done in the UK, but in the US,
Zigbee is used to take advantage of its mesh networking.
Yes IHDs are in some respects CADs+screen. They have a large overlap in terms of their function (although CADs are not fully defined atm). The problem atm is that the people making the devices and the suppliers installing them have little incentive to open them up.
Yes one of the many IHD manufacturers (there are about 10million IHDs in the UK now). But no local interface or even remote access (at least for consumer).
Ok I’ve caught up now…
Back to your musing again
From my limited knowledge point of view, a ‘hat’ solution (would it need certification?) that simply converts the zigbee data to a serial output. It can then be integrated with any platform (ESP, Pi, Arduino). A bit like the RFM boards (RF to Serial) - or have I oversimplified the problem?
That could then be extended to provide a full CAD & IHD but the basic building block would be there for all to use.
tbh i am not completely clear on the requirements for certification, specifically if a module could be certified on its own or if it would need to be part of a complete product. It effectively has to pass both Zigbee certification and the UK smart metering certification (as well as the usual electrical/radio compliance stuff and other product standards deemed relevant). But it could still be developed as a module in the way you describe but done in such a way that it is relatively easy to incorporate the module in another product based on a different platform. This wouldnt be plug and play but it would significantly reduce the costs associated with getting a CAD approved.
I’ve just fired up Digi’s XCTU tool on a USB xBee-PRO and there is a firmware for a ZigBee SE End Device. There is a field in this firmware for supplying a “Link Key” (field is labeled “Encryption Key”) into which you can enter an AES hashed “Installation Code”.
I just need to find out how to generate an installation code from the ZigBee specs and then I have a ZigBee certified device, with MAC Address, serial number and Installation Code that can ask SME-Plc to attach to my HAN.
Seems too easy but I’ll give it a go; just need to get the smart meter installed now.
Their SmartHub description talks about remote access.
Have you asked them about the possibility of accessing data from that or from their SmartViewPro?
I’ve just fired up Digi’s XCTU tool on a USB xBee-PRO and there is a firmware for a ZigBee SE End Device. There is a field in this firmware for supplying a “Link Key” (field is labeled “Encryption Key”) into which you can enter an AES hashed “Installation Code”.
The xbee may not support the larger keys required for SMETS2. It may only support SEP 1.1. Also I think you would need to present it as a coordinator instead of an end device (but I think it can be these simultaneously).
Good luck to you though!
I just need to find out how to generate an installation code from the ZigBee specs and then I have a ZigBee certified device, with MAC Address, serial number and Installation Code that can ask SME-Plc to attach to my HAN.
The Zigbee certification is for different things. They certify modules separately to final products. Also, technically the supplier shouldnt allow it to connect unless it has passed approval but I think they are quite ignorant about these requirements atm so maybe you will get lucky!
We have been offered remote access by various people in the past. All for a high price and a lot of upselling on ‘cloud analytics’ and ‘intelligence’ etc. The goal is local access over a standard interface. People shouldnt have to tie themselves to a specific web service in order to get the data being produced by a meter in their own home which they have paid for through their energy bills. This is more consistent with the philosophy of open energy monitor which is enabling and opening access to energy and environmental data.
Also worth saying is whilst these companies like to pretend they can offer universal access, they usually have few inroads with the suppliers who currently are the only people who can whitelist the devices (this should change under SMETS2 though).
I wonder though, if the chip itself is certified, and all you are doing is giving that chip a specific set of setup instructions (and you happen to have soldered it to a PCB) does it need certification again? TBH most home hackers wouldn’t care . So something like this NXP JN1569 chip could just be used.
Sadly I think it needs a product certification to go through the UK smart meter approval. I could be wrong but this is my understanding.
From what I can see in the “Great Britain Companion Specification”, SMETS2 uses ZigBee SEP v1.2. The xBee firmware I have installed is XBP24BSE 3332 (ZigBee SE Router API) which mentions support for SEP 1.x in the previous release.
The xbee may not support the larger keys required for SMETS2
Haven’t seen anything to suggest this is the case, do you have any references for this?
I think I’ve figured out how to set the “Install Code” correctly using Digi’s In-Premise Display Simulator which derives the AES hased “Link Code” from the supplied “Install Code” (128 bit random code + 16 bit CRC eg. 83FED3407A939723A5C639B26916D505C3B5).
https://www.digi.com/resources/documentation/digidocs/90001931/getting_started/inpremise.html
Fingers crossed it works.
Depends on the specific hardware. What series is it?
On further reading I think this is unlikely to work, the critical note on the “In-Premise Display/Meter Simulator” is
“XBee modules are not Smart Energy certified for standalone use”
and following the link:
“A certificate must be issued by the certificate authority (CA). Each certificate is tied to the 64-bit extended address of the device”
Even if I could convince SES to try and install a non Zigbee certified device, the Co-Ordinator would probably reject the xBee for not having a valid cert.
https://www.digi.com/resources/documentation/digidocs/90001931/getting_started/inpremise.html
https://www.digi.com/resources/documentation/digidocs/90001931/certificate/certificate.html#certificate
yes you get that certificate after product certification.