John, thanks for that summary. In conclusion, we are some way off from getting an open source zigbee connection to the SMETS2 meters?
It seems that someone way back decided to put zigbee into the spec, but the manufacturers have done nothing to make this accessible. I can plug in a £20 smart plug and start reading power flow instantly over a zigbee link, but for a £420 smart meter that isn’t the case.
In July 2019, on this thread, Ben Aylott estimated £150K to develop software to access SMETS2, which suggests there is something very wrong with SMET2.
If a home owner were to use software to access their smart meter, would they need some certification?
As with a lot of IoT, there seems to be a trend to force users to go via cloud links to get their own data. This can be access via “n3rgy”. Does this provide semi “Real-Time” data?
Meanwhile, the OEM EmonTX is quick to install and reads all this data without hassle. Are we stuck with that for the next decade?
SMETS meters use standard Zigbee wireless frequencies (currently) but use a different Zigbee ‘profile’ to that normally used by smart home products. I believe the specs should be available but in order to be allowed to be connected to the SMETS HAN i.e. the Zigbee network used by the meter, IHD and optionally CAD the device has to be officially tested and certified. Chameleon as an example will have done this but RainForest have not.
The legitimate justification for requiring devices to be tested and certified is security and probably safety, again there does also seem to be some use of this as an obstacle.
This testing and certification has a cost and designing and building a CAD has a cost. There probably is some case of these costs being inflated to keep people out but to be fair there will be some legitimate costs.
My belief is that n3rgy is using data uploaded by all SMETS meters via the DCC network and this is far less frequently updated than data obtainable via a CAD.
In some European countries smart meters have a special serial port called a P1 port and it is possible to get an appropriate USB cable and plug that in to a computer and read the data and this approach does not require approval or certification. I don’t yet myself have a UK smart meter so I don’t know if any UK smart meters have this same port. https://www.amazon.co.uk/Kaifa-Dutch-Smart-Meter-ft232r-ISKRA-ME-382-EN-MT3/dp/B07DFM2H6S
Thing is, I don’t mind the HAN being private and the devices needing paired. I do object to that data not being made available out of the CAD/IHD. Even a simple USB serial interface would suffice (and cost would be minimal). But making the data totally inaccessible is plain wrong. This of course is a result of the regulations being written to favour the supplier and not the customer.
I totally agree with you, the fact that only one manufacturer will even consider selling you hardware shows that the rest are not interested in end users and are operating a cartel.
It’s also down to economics. The number of people that would actually make use of being able to locally access the data is small, very small. Add in the cost of supporting the small number of people using it, it just doesn’t make sense to do.
Today n3rgy collects data from the meter once a day. i.e. it downloads your data in the early hours of the morning and makes it available to those who have been granted permission to access it. We’re also piloting increasing this collection frequency to every 2 hours (and potentially more frequently in the future) . With regards to CAD pairing, we would be happy to also offer this service when and if a market of CADs becomes available but as previously mentioned, there’s very few CADs available on the market today which aren’t already part of a closed system.
Out of interested, what do you guys believe the imperative for real time data is? If you have the tariffs (which always need to be published the day before) you already have the mechanism to automatically minimise the cost of power hungry appliances!?
Whilst I agree it might be small, you could build a smaller device, no (pretty useless) display, and allow devices to connect via WiFi (like a massive number of smart switches do) to get the info. You could even then do updates to fix bugs and allowing API access to the data is easy.
Octopus provide API access to probably a small part of their user base. Are you suggesting they shouldn’t?
My point is, it should have been built into the specification to provide local access. Commercially, no one is going to do it unless they have to generally as they have too much self interest.
The main interest is to measure exports/imports for renewables generators.
Admittedly that is going to be a bigger market in other countries with better solar resource or subsidies, and it seems each country is implementing something different for its smart meters, with the only common interfaces being the LED display and the pulsing LED.
I would tend to agree that the near-real-time data isnt needed for a lot of things. But I think one clear use case for a CAD is if you want to adjust power usage of a smart appliance (battery/EV charger/diverter) according to actual generation output (or some other control signal) then you need near real-time data from the grid feed. A lot of current energy smart appliances come with these grid feed monitors which use CT clips for this purpose (and in some cases G98 compliance purposes). But most are not interoperable and we are increasingly seeing situations in our work now where our members have a battery from one manufacturer, diverter from another, EV charger from yet another and they all have a CT clip watching the grid feed so they can follow PV output etc. but then racing each other to grab any excess with no coordination or prioritisation possible. Seems like an interoperable CAD+energy management system would be better to sit in middle here dealing with the control priority issues as well as maybe reducing costs by not having multiple of these devices clipped on (and one would hope a basic Zigbee SEP CAD would end up being a bit cheaper than a CT monitor).
No it’s the harshness of commercial reality, the more features you add, the bigger impact you have on the profit margin for all involved.
There’s also the security aspect, once you provide the ability to do firmware updates you open up devices to potential abuse / intrusion.
They key here is Octopus built systems from the ground up, without having to bring with them years work of legacy. The API that we as consumers are exposed to is just a small rework on what they’ll be using internally. It’s great that they are the leading disruptor to the sector, and long may it continue.
I asked this question because I’ve been working with an advanced system provider that does exactly this. Based on ToU import and export tariffs it decides what to do with the energy. The system takes the real time information from the hybrid inverter (not smart meter) because it needs information of domestic load, solar generation and battery status. The ‘intelligence’ combines this with information with the ToU tariffs, weather forecasts and predicted domestic usage patterns then chooses to run from battery/solar or import/export/neutral the grid. It will also export the remaining capacity of the battery before the max £ export ToU (SEG) runs out.
None of this needs real time data from the smart meter but it does take historic consumption information form the meter (to check it’s working correctly) and export information (to calculation SEG revenue). It also takes the tariffs from the meter which are available the day before (when it’s calculating solar generation based on weather forecasts)
It also does a bunch of other things but, from an energy perspective, it’s very impressive.
EDIT. My basic argument is the smart meter only shows import and export from the grid which isn’t that useful for effective energy management. You need solar generation, domestic load, battery status and a bunch of other things first. Smart meter data helps you tune the system, not operate it.
EDIT2: Yes it has an integration with Octopus Agile tariffs (both import and export)
It is a cop out. If a £15 smart switch can provide an API, a £60 IHD should be able to do the same especially as every user is actually paying for these things (green levy). As I said, it should have been written into the spec. The IHDs are as much use as a chocolate fireguard. I bet most end up in a kitchen drawer. If we actually had to pay for one, no one would bother. This means the ‘commercial reality’ is they can design something useless, charge a fortune for it (probably) because the energy companies have to use them and make a fat profit off us poor souls. Can I send mine back for a refund?
You’re forgetting the onerous initial and on going security requirements authorised devices have, plus the customisations to international standards the UK have performed.
I have solar panels on a feed in tariff - I understand that smart meters should record energy feeding back into the grid from the panels
Does your API have a field for this energy feed?
Does anyone here have any insight into the DCC network roll out? I’m very keen to use a variable price tarrif - I have a SMETS2 meter installed but no DCC coverage in my area!
Supposedly it’s meant to have 99% coverage by 2020 but I’m guessing it’s nowhere near that.
I’m in the Southwest so looks like I should be covered by Telefonica and their cellular network. Given the lack of mobile coverage where I am, I don’t think it bodes that well.
Thanks for the link though, really interesting read.
