Sharing heat pump data, An open heat pump dataset?

Aha, great, well done Fiona @frh, look forward to hearing more about how these different initiatives progress! We have a few too many different things on in parallel to take part in a large grant funded project as a main partner at least, but great to see these initiatives happening!

1 Like

Thanks Mick. This level of heat pump monitoring is definitely a niche pursuit at the moment due to the issues you’ve noted. How to go about addressing your final point is a whole separate conversation! I wonder if @BetaTeach has thoughts on Mick’s point:

  • There needs to be a drive for three things in heat pump monitoring that I see
    1. Mandated Inbuilt heat meter and temp probes in heat pumps
    2. Combined with a simple method of interrogating that data (username/password, API etc)
    3. All in a common and agreed format/fields; as mentioned above in ‘meta-data’.

Maybe manufacturers and installers would be motivated if there was demonstrated consumer demand for this? I think the more data is publicly available, the more there will be the expectation that consumers should be able to understand the performance of their system. Imagine if there was no way to tell how much power your PV was producing?!

1 Like

My company is involved in the Greenwich TIME (Thermal Infrastructure Motivating Electrification) project under Stream 1, although I’m not personally involved. I likely would have been except I’ve just started maternity leave. (So when I suddenly stop responding to messages you’ll know why!)

According to the website, “Successful Stream 2 projects will be published later in summer 2022.”

Hopefully not too long then. It will be very interesting to hear whether/how OEM data can be integrated into the Stream 2 project.

1 Like

Hi Trystan, I would participate with my data from a Vaillant AroTherm Plus 13kw unit but I might need help if you need specific formats or charts. Peter

1 Like

Just to put some bones on the ‘cost’ of full bananas class 2 heat and electricity monitoring.

My 22mm heat meter and all associated OpenEnergy gear and sundry cabling/boxes etc will come in at about £600.

I’ve priced something similar for Damon Blakemore for a larger 40mm system and it’s closer to £700 total. The larger the pipework (and heat pump) the more expensive the heat meters get.

So what manufacturer is going to outlay the £300/£400 cost of a proper heat meter in their units?
Will the vast majority of customers want to pay the extra for monitoring if manufacturers pass the cost onto the user?

1 Like

Thanks Fiona, Mick. Yes agreed the cost is quite prohibitive for all the kit, especially once the larger heat meters are involved and I think there is quite a bit of potential to have something lower cost on our end too. E.g An ESP32 based MBUS reader or something similar, Pi Pico.

A lot of manufacturers do seem to be including Sika VFS flow meters as standard to do some level of consumption and production monitoring (usually always seperated like that and not reported as COP or SCOP). This is the case for most Mitsubishi Ecodan installations, Glyn’s Samsung installation, Vaillant Arotherm and perhaps Daikin units as well (there’s one on my farther in-laws system). There is also movement towards more connectivity e.g Ecodan Melcloud.

Glyn bought the Samsung Modbus interface for his heat pump for ~£150 and unfortunately it doesn’t provide any info on flow rates, electricity consumption or heat outputs. There is a CN105 connector on the Mitsi Ecodan that should provide local access to a lot of this information but it’s protocol is not documented officially. It would be great to see manufacturers provide better local or/and remote access to the data they are already collecting.

It would be good to do some work on validating the in-built monitoring that these heat pumps are doing, using the more expensive class 1 and 2 electric and heat meters.

There’s also a potential to piggy-back on the output of the Sika flow meter that comes with some of the heat pumps, @Rachel has already done this, e.g: Heat flow sensor for ASHP - #16 by MyForest, this seems like a really promising approach to a lower cost solution. Again it would be good to compare class 2 heat metering with reading the analog voltage from these. The EmonTx v4 with it’s 12bit ADC should be able to help do this more accurately as well.

2 Likes

Thanks @PjH, how are you monitoring the AroTherm? Are you collecting data using emoncms or your own setup?

Hi, I am using emoncms - a Sontex Superstatic 531/Supercal 441 for thermal data and an optical feed from a dedicated ASHP electricity meter as data sources, feeding into an EmonPi running emoncms, with a DS18B20 for external air temperature. There must be a lot of useful data in the Vaillant controller interface but I have not seen anyone suggesting a way of accessing it. Peter

1 Like

Thanks Peter, I’ve created an account for you on emoncms.org and sent you a DM with details if you are happy to share the data that way.

I think there may be a modbus communication module for the Vaillant but I’m not sure how much data is accessible using it. Looks like there are some having luck with Vaillant/MultiMATIC over at home assistant Vaillant/MultiMATIC integration - #28 by Peto - Development - Home Assistant Community

For general interest of others here, it’s always worth having a look at Home Assistant first for integrations and searching for code that mentions a particular hardware model’s on github:

Not sure how much is directly applicable to what we need but may be a good starting point for wider integrations.

1 Like

Just to give us a concrete example of the privacy problem, I’m sure you could all work out when you would chose to burgle my house if this “My Solar” data was public:

Unfortunately it’s not just a question of deferring publishing the data. For example, if you can see that historically I’m always away on a Sunday evening then you can take a pretty good guess when to try and burgle my house.

It’s tempting to use a pseudonym so that people can’t find the physical property, but that has the glaring flaw that once the pseudonym has been compromised you can’t recover.

At the moment I don’t have answers about how to safely publish the full dataset without causing privacy issues. I am investigating solutions, but I don’t know if there will be any.

1 Like

@PjH vaillant ASHP data can be collected via ebus. There is big ebus community out there that’s developed software and hardware to read from this. Google ebusd on GitHub.
Here is some of the data I read. I’m able to calculate the CoP using a combination of emon power consumption monitoring (emontx3) and yield data via ebus. It’s all logged into emon via MQTT.

3 Likes

Nice one @Zarch Pretty good experience here (in a Passivhaus) of a 3.5kW Arotherm+ system, other than the controls and reporting. Waiting on emontx reappearing in the shop to try and get better monitoring for next heating season. I just wish Vaillant would open up their API to users, as much of the information is captured and could be made available without additional CT clamps, flow temp monitors etc. There are modbus/ebus possibilities (thanks @modeller ), but it is hard work for non-techs like me. At least you were sensible and got a heat meter included in the specification!

2 Likes

Thanks for the example @MyForest and for raising this with a bit more prominence.

Yes a good point.

Definitely a complex question!

1 Like

That’s my concern with Smart Meters. OK, so all the data securely encrypted. Unfortunately, a lot of organisations have believed that to be the case with their customers’ data, and found to their and their customers’ cost that they were wrong.

1 Like

That’s great @modeller, what kind of resolution is the data via ebus? can you poll every 10s and receive new data?

Thanks for raising the privacy issue @MyForest, it’s a tricky one. Do you think the privacy concerns are at all reduced for heat pump data relative to solar PV/electricity consumption, or pretty much exactly the same? Occupancy is (or can be) less directly obvious if heating is on fairly continuously, although it’s still clear when you’re having a two week holiday.

Different people will have different comfort levels of course but a platform would need to either offer different options or have some (probably relatively conservative) policy.

@TrystanLea’s profile explorer potentially gets around this, but that may also indicate if you are always out at a certain time of day!

1 Like

On the bright side, heating is pretty coarse.

I know from friends who do electricity consumption analysis that they can tell if you are using a toaster or a hoover from the power profile.

For heating it’s pretty going to be just “the occupants are there”, or in the worst case “we can tell the kids are back from Uni because the heating is on more at night”.

I’m torn between the drive to share as much as possible to allow others to do their own analysis and the inevitable problem of sharing so much that random people can tell what you are doing (or worse yet, they simply run a bot against all shared data and ask “who isn’t at home tonight?”)

Setting permissions on the data does seem to make sense, but then I have to try and guess if I can trust you and that is hard with the looseness of this forum for example.

1 Like

Without the context the data is almost useless. With a gas boiler everything is effectively just blasted with power and the only real variable is is it on or off. but a heat pump has to be used in a more subtle way if it is to be beneficial. Context is vital but security is also important. It must be possible to de-localise a system description to keep security intact while keeping an amount of localisation to enable local weather effects to be considered. probably a role for this forum here in enabling checks for authenticity. eg is this data from a trusted source. once out in the known world there will be a queue of people wanting to submit bad data to suit their own agenda or just because they can.

Note this is not a reason to can the project. its a good useful idea which I will be happy to contribute to. (circa 1935 solid wall house with air source heat pump solar PV and Tesla power wall in the midlands)

2 Likes

I feel that as long as the data is sufficiently anonymised, then most people will be happy for their data to be shared. By this I mean included in a comparitive set of “similar heating systems near to your location”, or a broader analysis of heat pump performance.

We need to consider why we’d collect data about heat pumps. If we’re simplying sharing our own experiences, then curated / annotated screenshots with accompanying context are probably more useful. There are many examples of these on this forum, which have been great talking points.

IMHO, the real benefit of collecting data from many systems is to provide an overall picture of heat pumps: how well do they perform in general? how about certain types of property? underfloor vs. radiators? Seeing individual systems by themselves (without context) doesn’t seem nearly as useful.

I remember listening to a @BetaTeach podcast titled “How Well do Heat Pumps Perform - looking at the OFGEM data” discussing the results from this report [PDF] ← this episode / report seems somewhat relevant to this discussion.

What really struck me is that the researchers had “a dataset containing anonymised information from over 2,200 domestic heat pump installations was obtained from Ofgem”, yet they had to discard about 3/4 of that dataset, analysing the performance just 598 ASHP installs.

So, one question to ask is: what would an ‘open heat pump dataset’ driven by the community be able to show that other, official datasets (e.g. MMSP) haven’t? What data analysis / pruning would be needed to be done in order to extract something useful from it? How can this be done automatically?

1 Like

Just getting to this, apologies - thank you @TrystanLea for initiating and for the work on public dashboards. Timely for me as a few villagers will be showcasing heatpump installs at a local energy fair in Buckinghamshire at the end of the month, so this will serve as a nice real-time demonstration.

I would support @Timbones latest comments - and I would be very happy to contribute to a detailed public dataset assuming location/identity is sufficiently fuzzed (had wanted to contribute to the MMSP, but our installer declined to participate sadly). That said, would be keen to explore how/what/why this could be contributed effectively and what aspects we would want to build in to maintain a reasonable level of data quality. Weather Underground springs to mind.

I’m very happy with the OEM data coming off our Midea system and Sontex meter installed in January - which has proven essential in optimising and diagnosing some early teething issues. I’m not particularly confident in my COP calculations but will continue to work on this!

1 Like