Sharing heat pump data, An open heat pump dataset?

Emoncms
1

A few folk have expressed interest in sharing heat pump data publicly to enable learning and familiarisation with the technology. @Zarch is in the process of setting up a public dashboard for his soon to be installed Vaillant Arotherm air source heat pump and Fiona Hughes (@frh) of https://twitter.com/greenheatcoop has also been in touch with the interest of developing an open heat pump dataset.

It’s been possible to provide public access to Emoncms dashboards since the early days but the same functionality has not been possible with emoncms apps such as the My Heatpump app. This has now been implemented along with some other changes to improve emoncms public data accessibility: the ability to use the emoncms graph interface to explore feeds made public in more detail.

Here’s an example public heat pump app working, using my own data. Try navigating to the public feed list and graph interface as well:
https://emoncms.org/midterrace

image
image1391×938 51.2 KB

Im not uploading data to this account in realtime (hence the not live note). Im logging the data locally and then just doing a periodic upload using the emoncms sync module. The idea being that this provides at least some level of privacy. I think ideally there should be an option to specify time restrictions on data that is made public, e.g allow access to data with a one week delay.

To date I’ve usually only shared my heat pump data using screenshots of graphs and monthly averages and totals, this provides a useful opportunity to contextualise what I’m sharing. I have however had the benefit of looking at detailed data from other heat pump systems for years through work with @johncantor and others. Seeing John’s data is what gave me confidence to go ahead and install a heat pump in the first place.

Would others be interested in developing an open heat pump dataset? Is a fully open public dataset the right way to go about this? or would something which requires some kind of sign up process to access shared heat pump data be better? Or is it better to share specific analysed outputs or contextualised screenshots?

Copying a few other people into this who may be interested @MyForest @Timbones @mjr @Jimbo1 @thedawnbefore @glyn.hudson

5 Likes
2

I’d be happy to contribute. I’ve got an 8kW Ecodan, installed in April this year.
I’m not 100% confident in my heat measurement at present, but I need a bit more actual heating to test the set-up better. I’ve basically piggy-backed and ESP8266 onto the Sika vortex meter and have a couple of temp sensors strapped to the pipework. The temperature differential is low and I’m not entirely convinced that the 0.5-3.5V output scaling is the same on the Mitsubishi supplied unit as on the Sika standard unit, so still a bit of work to do there. I’m also logging ‘degree days’, which might allow more useful comparison between different UK locations.

Rachel

image
image1296×443 99.3 KB

3 Likes
3

Thanks @Rachel that’s great to hear! Glyn and I are also keen to explore piggy backing on the Sika vortex flow sensors for flow measurement so it would be great to compare notes on that. Glyn has one on his Samsung and they seem to be on a large number of systems now. Glyn has a Sontex 749 as well so we will be able to compare accuracy.

1 Like
4

I’ve been thinking about how to publish heat pump data for a while, since this topic about public heatpump dashboards. I did set up a public instance of My Heatpump app on my own instance, thought it’s only accessible to a whitelist of IPs (more to protect against bots than for privacy).

I had wanted to build something from scratch along similar lines as pvoutput.org, but only got as far as designing the database before getting stuck on user management. There is something to be said for leveraging the technology the emoncms already provides, amirite?

What pvoutput is really good at, and I was hoping to emulate, is aggregating daily data into longer periods, i.e. weeks, months and years. This helps smooth out the day-to-day variances, and show typical usage patterns. I have my own graphs set up for these, but it would be nice to have some standardized views that everyone uses.

Being able to cumulate running costs while taking into account TOU tariffs would be useful (again, like pvoutput does), along with RHI calulations, comparisons to typical oil and gas boilers, etc.

It would also be nice to see some high-level analyses of heat pump performance, such as average power consumption vs. hour of day [example], or flow temp vs. outside temp, or monthly running times [example].

Another nice-to-have that would be useful in conversations is to be able to link to specific day and time period of the heat-pump chart, so show off specific situations, though maybe edited/ annotated screenshots are more useful here.

I’m not too worried about the privacy of my own ASHP data, as long as the identity and location is sufficiently fuzzy.

The “live” view of heatpump data (i.e. throughout the day) is perhaps only really useful to the owner, because of how contextual it is. pvoutput has this, but I don’t use it myself and I only upload daily data. I had planned to focus on daily data first, mainly to minimise the data storage.
On the other hand, solar PV is a static system that is wholey dependent on the sunshine, unlike a heatpump. Collecting detailed power and temperature data allows for much more sophisticated analysis.

And another thing; it’s worth remembering that this data isn’t so much about the performance of the heatpump itself, but of the whole heating system: radiators, underfloor, pipes, tanks, buffers, insulation, etc. It might be worth capturing that somewhere if this data is to be used for any wider research…

4 Likes
5

Thanks @Timbones!

Great, I missed that post and glad that the changes above address quite a few of the limitations that you outlined with sharing access to the heat pump apps using apikeys.

Great, yes happy for emoncms to form a component of this. I did wonder if it might be a combination of another web page hosted seperatly providing a list of systems and contextual information, that could then either link directly to dashboards hosted on emoncms (emoncms.org or other servers) or host it’s own dashboards that might pull data from emoncms.org or other servers… it could even be platform independent and pull data from other services and time series databases…

That kind of insight will Im sure be crucial to making this a useful initiative.

Yes agreed re fuzzy location in particular and identity as much as possible. I guess Im already linking my identity to the dashboard above, key for me is limiting access to current data with some kind of delay I think.

Exactly. Most useful research and learning outcomes will be on the historical dataset.

Yes agree completely, there needs to be some kind of front-end website to the data that provides all of this context. Id be happy to work with you and others on this.

2 Likes
6

It would be great to see more data. It would be nice to compare a Panasonic, and also to look at size… i.e. whilst we have seen some very poor performance from ‘oversized’ systems (oversized in my view), it would be nice to see some data showing good performance from such systems. Good work, all involved

1 Like
7

Thanks John, yes would be great to see more on that. It would also be really interesting to see data on systems with and without low loss headers, buffers in different configurations and perhaps different control approaches that make better use of these different system configurations. E.g I think David @MyForest has managed to compensate for his large Ecodan with improved controls…

1 Like
8

Yes, David has done a good job improving his performance. I need to focus in on my 6kW

image
image691×471 44.7 KB

Find out why the COP plummets in certain conditions. I might need to join David and make a control to stop it running when it looks like the graph after 15:14

1 Like
9

Hi all, thanks @TrystanLea for starting the discussion, and thanks @Timbones @Rachel and @johncantor for sharing your thoughts already. There are (obviously) a lot of directions this could go in over time. My initial thought with this was similar to @Timbones’s point on aggregating daily data to understand usage patterns across time and across different users, but I think the leaderboard and linking to the heating system ‘meta data’ are also important aspects.

Right now there are a few studies which have used measurements to develop diurnal profiles for heat pumps, but there’s not much/any information behind these about the installation or controls. And these tend to look “peakier” and more “boiler-like” than what I’ve seen from OEM users and others posting about their high-performance installations.

@TrystanLea or @johncantor do you have thoughts on the pictures below based the data you’ve seen through the years? My theory is that high-performance, weather compensated installations might look somewhat different to these.

CLNR profiles

Aggregated winter weekday
Aggregated winter weekday602×507 61.2 KB

Screenshots taken from these pdfs:
http://www.networkrevolution.co.uk/wp-content/uploads/2015/01/CLNR-L091-Insight-Report-Domestic-Heat-Pumps.pdf

I can put together a few more thoughts on this later in the week and post for comment. Then maybe the next step is more discussion (or even a brainstorming zoom) on what are the most important aspects to start with / what would be a reasonable “minimum viable product.” Then could plan what makes sense to be within emoncms.org vs potentially on another platform. Happy to hear thoughts on process as well as on the dashboard idea!

4 Likes
10

I would be interesting at others thoughts on you graphs. I find it takes time to digest things, and feel that others are better at bigger-picture analysis… I tend to like zooming in on actual temperatures, power-input and heat-output over short periods. e.g. if it were car MPG tests, I would be looking at tyre pressures and filters and things. Between us, we should be able to draw some useful conclusions.

2 Likes
11

Thanks @frh really interesting to see those profiles, got me intrigued to see how mine looks, so started coding something up to do this. Here’s last winter Dec 21-> Apr 22:

image
image978×452 124 KB

It does look a lot smoother I guess. The spikes at 2-3am and 1-3pm is hot water heating. No morning heat up ramp and slightly more overnight except around 11pm…

12

Here’s my hourly consumption* for heating only during February and March, showing ramp up in the morning then keeping level until it’s set back at 11pm.

image
image714×441 30 KB

*The original chart was showing kWh of produced heat - I don’t think the shape will change much between them.

This is certainly the kind of chart I’d like to see produced automatically, both for my own system and for an ‘average’ system for comparison.

2 Likes
13

Interesting to see @Timbones, quite a different shape again!

Ah I missed that, mine shows kW electric used as well. Seems like quite low amounts of heat producted then in the original graph…

14

I would like to do all that I can to help make a public dataset like this possible and don’t want the billing/account costs of emoncms.org to be a barrier. Id be happy at least for the first 20 users who wish to use emoncms.org to share ~8 feeds publicly for this dataset, for these feeds to be free. I think we’d be lucky to get to 20 users but if it does build into something larger this could be reviewed again in future, perhaps with free feeds if resolution is lower and data retention time shorter.

At the same time it would be good to avoid emoncms.org lock in. If we did put together a front end website that holds a leader board and contextual information about the systems it should be possible to implement in such a way that data could be pulled in from a variety of remote servers.

15

Nice plots @TrystanLea and @Timbones. The first plot looks like very low amounts of heat because it is averaged across lots of households which will have max demands at different times. Both of yours look generally flatter than the originals, but it may be that if 10-20 profiles were averaged together it would start to look like those pictures anyway. It’s December and January that seem flattest in Trystan’s plots, start to see a bit more ups and downs in Feb-April, I wonder if yours has the same Tim?

Your offer on billing costs is very kind Trystan! I wonder if we could get a small pilot version implemented whether it would be possible to get some funding support that might cover part of these costs. Something to consider as we go.

16

Hi all, I have just joined the platform as well as this conversation. I will know some of you on hear and some have been guests on the podcast (e.g. @johncantor)

I’ve mentioned to Trystan there is something being built which will do what you all want. From my stand point I’ve always though it a good idea where monitored systems can help engineers learn good practice and processes. I’ll hopefully be able to discuss more shortly and I’ll keep Trystan informed too

5 Likes
17

Yep. As @TrystanLea says, as part of my energy-stats umbrella of (hopefully) useful stuff i’ve created a cloud install of emoncms and will be pushing my sharky heat meter based data to it.

Plan is to make the emoncms heatpump app fully accessible to anyone that wants to play with the data. Custom URL, tweaked apache config, ssl etc, then the emoncms app embedded.

As my heat engineer and I have almost ticked every retro-fit requirement, we are hoping that the system will perform well when it gets installed in the next month or so.

Hopefully can dispel some myths about retrofit in a 1930s semi. Will be awful if it doesn’t perform like we hope though!! :face_vomiting:

It’s gonna be a 5kW Vaillant Arotherm Plus with a 250L Mixergy cylinder and Eddi… i’ve also got Solar, Zappi and Battery too.

I will also obviously contribute the data to this new project too.

I’ve been part of the PVouput solar website for almost 4 years now and all my generation/import/export data is up there. So I love the idea of aggregated data from lots of disparate system configurations.

1 Like
18

One of the features of PVOutput is the ability to compare two systems directly. For example, weekly generation of Zarch vs mine - both are about the same size, similar location, but with different orientation.

It would nice to be able to do the same for heat pumps, though I suspect the variation will be much larger. Something else to add to our growing wishlist :wink:

2 Likes
19

Thanks @BetaTeach and thanks for signing up here!

@Zarch very much looking forward to seeing your data, I know it’s a lot of pressure for it all to work out… :slight_smile: the combination of the heat pump and battery should be great for those profiles, allowing you to shift most of it offpeak I imagine! Great to see your PV output, it does highlight how good some of the PV output features are!

@Timbones really interesting to see that, yes very effective!

20

Thanks Nathan @BetaTeach for the info. It’s encouraging to hear others are already thinking about this. Do you have an idea of when you’ll be able to share more? How soon is “soon”? :slight_smile:

I’ve written up a google doc trying to capture and organise the various ideas above. It should be editable. @TrystanLea and others please do add in your thoughts! Nathan it will be interesting to hear how similar the project you are involved with is, whether objectives are similar (I think so!) and whether there is potential for OEM data to be incorporated.

1 Like