Scripts to drive an Ecodan heatpump via MELCloud

Hardware Heatpump
1

I know I’ve been chatting about my “algorithm” for a couple of years. Something happened that delayed me actually sharing it.

I thought it was about time I got on with it.

So now you can see I’ve started sharing it here:

It comes with a number of caveats.

Obviously don’t try and run it :slight_smile:

It uses the state from MELCloud so you’d need that cached. It only expects about ten hours so it’s not too onerous. In fact I don’t think it ever uses anything like that depth of history. Of course I fetch the latest state before running this script so it has a vaguely up to date view of what the heatpump is doing.

It can absorb pywws raw input files too if they are available.

I am stripping out the really personal bits from the code, but it still has much of my personal stuff in, such as my hard-coded schedule which won’t suit any one.

I’m trying to move to newer structures for things, whilst also keeping it working and also sharing it. You can all guess how that’s going.

I’ve only included a few of the simpler Action things so far in actionProviders. It’ll take me more time to port all of the actions over to something shareable. The blockers list is complete.

I’ve got some useful unit tests but they aren’t ported to this new bubble yet.

You should start at act if you want to see what it’s up to.

To give you a flavour, the output looks a bit like this (but it’s likely to change quite a bit):

image
image2125×294 177 KB

I suspect this’ll be most interesting to @mjr and @Timbones but please be gentle with me when you’re spotting the innumerable flaws.

2

I love pywws, but with a second Fine Offeset weather station turning up its toes, and no other means of pushing the data to pywws, I have had to drop it. A real shame as the various data it could produce were incomparable. There is a certain beauty to the design, I just wish there was a means of interfacing it to other stations.

I’m going to move to an Ecowitt station although looking at the website for Fine Offset, they look like there is a close association.

3

That’s a shame to hear @borpin - these things always break down over time, it’s why I’ve held back on using much IoT in my house.

Arguably in my case I should just be pushing the data to EmonCMS from lots of places and having my algorithm pull from there. I had been avoiding doing that because I know the filesystem is always going work but other things might break and I want something as important as the heating to keep working.To be honest EmonCMS has been pretty good so I should re-consider using it as a hub.

Another thing is that my current setup would work for someone who didn’t have EmonCMS, but arguably that’s just because it’s leaning on MELCloud-formatted data instead of taking feeds from EmonCMS which could possibly work for other heatpumps.

Hmm.

4

Yep, it is really good as it is a simple flat file store as well.

Wherever possible I always install in parallel with the existing system and thus have a backup. So for heating, the old thermostat is still there, just set on anti-frost. To use it, just set the temperature.

For lights, I generally have wifi plugs, with the lamps off that. Simple button to switch on. I have a few Zigbee devices, but these are really unreliable, so are going to get ditched.

Avoid cloud base solutions whenever possible and never for critical tasks.

I think things to consider when doing this stuff are, “what if I am not here”, “what if there is no external internet”, “what if I sell the house”. Risk-based design.

Emoncms being file-based is also pretty robust and portable. I know others have gone the InfluxDB route, but I do like the Emoncms architecture.

5

Yes, I agree with you @borpin.

We also have the old, simpler stuff in place which can be used if required. If the extra stuff vanishes the simpler stuff will take over, it’ll just be less efficient.

The extra things we add need to be clever enough to work alongside the simpler stuff (including humans :slight_smile: ). In fact that’s exactly what this huge chunk of code is doing. It’s trying to gracefully take back control if the management has been interfered with. Of course that’s fraught too, especially if you have multiple agents trying to operate on a thing.

1 Like
6

According to wxforum Ecowitt is an (or the) official reseller for Fine Offset, the manufacturer.

You might try to look there for reliable information about weather stations.
https://www.wxforum.net/index.php?PHPSESSID=tkhn2hci4quqdbbu4blbjp4la8&board=111.0

1 Like
7

@johncantor You may (or may not!) find this interesting.

I like using software to drive my systems because I can write tests like this:

I can very well see that this would be an anathema to someone who is happier looking after the hardware.

Of course the distractions in setting up the test and the rigmarole of caring for them and trying to run them is an annoyance to me. Even if I automated the running of them in a good environment, I still need to run that somehow. Even if I automated the running of that on every push and made it block the release of the software (as I do in my day job) then I still have to manage all that stuff.

The saving grace for me is that it’s protecting me from my own stupidity. For example it’s ensuring the startup temp can’t be higher than the maximum flow temp so at least I’m not going to break things. Even that is flawed of course because I could easily have mixed up the names of things.

Anyway, maybe that was interesting.

PS Here’s how the software chatters to me if I set it up in an IDE:

image
image881×529 82 KB

8

Have you published your MELCloud scraping scripts anywhere? I’ve written my own script based on information I gleaned from your posts, but others may appreciate something they can download.

While I’m here, and it’s vaguely on-topic, how frequently can you pull data from MELCloud servers? I’ve been able to get data every minute from them, but every now and then their servers will send stale data until I back off.

For instance, today I switched from every 2 minutes to every 1 minute, and the flow/return and energy values stayed the same until I went back to every 2 minutes. It’s as if there’s some over-eager caching engine on the server or something.

image
image1308×740 116 KB

Here, I’ve switched the yellow ‘Solar’ feed to show grid import, which is more-or-less what the EcoDan is drawing. You can see there’s 3 cycles of the HP, yet the energy and temps have flatlined. Looking at the Hourly Temperatures report on the official website shows what’s really happening (from 16:00):

image
image933×362 34 KB

What is peculiar, though, some of the fields from MELCloud, like OperationMode, OutdoorTemperature and TargetHCTemperatureZone1 (Desired in 1st chart above) will continue to show real-time values.

This tends to happen for a few days, and then sorts itself out so that I can request data on a per-minute basis, until happens again some weeks later. Anyone else had this issue?

10

Similar? I get values which change only every 5 minutes ever since I started collecting back in January.

I may lengthen the interval in my collection script (based on the one by @TrystanLea ) to 2 minutes to see if that gets more frequent updates.

11

Yes, I’m seeing these problems @Timbones and @mjr since 2021-11. Until the firmware upgrade I was reliably getting per-minute data for all the fields.

I’m polling once a minute.

I’ve now altered my script to use ListDevices for what I can glean, but then also scrape GetHourlyTemperature. That’s quite ugly because the correct slot in GetHourlyTemperature keeps moving. I often get an incorrect reading exactly on the hour as the structure rotates. It’s very boring.

I merge them together into something like what I used to get from ListDevices but without the pump information which I can manage without. You can see in my algorithm that I was never using the pump state to guide things, just the OperationMode.

So far I haven’t been brave enough to publish fetch.py (320 SLoC) because I don’t want to upset Mitsubishi. I know it’s our data, but it is their servers. As a sysadmin I do try to be nice to other sysadmins. It does feel crazy that it’s taking me 320 lines of code to read a JSON body.

12

I’ve been able to poll ListDevices every minute, quite reliably, for most of this year. That is until it suddenly gets stuck dishing out the same values. When I notice this happening, I throttle it to 2 minute intervals (hello crontab), and I instantly get good data again. Just less frequently. After a couple days I go back to every minute, and it seems okay for a few more weeks.

It feels like the server is penalising me personally because I’ve used up my monthly quota or something (I haven’t noted when this happens). But it’s more likely to be some unintended glitch or incompetence than it is malice.

I could do with some way to detect when this happens and throttle back automatically, but I’ve not found a field that changes often enough. Maybe I’ll just poll less often overnight, or when I know it’s off.

13

Yeah, I’m using the heat pump frequency over three readings to see if it’s working again. It isn’t. Sigh

14

I’ve picked one of my feeds and counted how many null values appear for each day, which will be low for when I could poll every minute and high when I could only poll every other minute. Here’s how it looks:

image
image932×154 6.01 KB

It’s perhaps too early to draw any conclusion, but it looks like it tends to break towards the end of the month. Maybe that means we can return to minute polling next week?

I had assumed that this was just affecting me personally, but there’s a chance that it affects everyone at the same time. Like maybe they run out of processing quota on their AWS instance or something…

15

Ouch.

Unfortunately that’s an even more compelling reason to not invite more people to start polling every minute using our dodgy scripts.

Let’s see if it magically fixes itself on the 1st.

1 Like
16

Aren’t cloud services great :rofl:

Try 90s polling for a month?

17

Yep, it’s horrible.

In the UK we have a Renewable Heat Incentive scheme that’s essentially paying for your systems for us. We have to be sensible for seven years and that’s why I’m not even considering tinkering with the hardware.

Even if I could do all the monitoring by bolting things on, I’d still need to interfere to send it commands and that’s likely to look exciting to the assessors.

Actually, the government introduced the Metering and Monitoring Service Package (MMSP) program specifically to encourage owners to monitor their systems. It’s an extra thing on top and they paid for that too. Arguably our experience is showing that it’s not playing out as they might have hoped.

1 Like
18

Is it looking healthy now it’s a new month @Timbones ?

19

Nope. :frowning:
Will test again tomorrow. Back in March, I didn’t switch to the high rate until the 4th, possibly because I forgot about it.

I have worked out that when idle, CurrentEnergyConsumed shows a 60 W pulse every 3 minutes or so (averages out to about 20 W continuous consumption). This can be used as a “heartbeat” - if it sticks at the same value for several readings, then I’ll know to poll less often.

I also have a plan to poll less often when previous readings show the unit as idle (OperationMode = 0, water pumps off, etc). However, then I won’t be able to read the heatbeat to know if I can read at the higher rate when the pump is running. Needs more thought…

There may be a better way, which is to intercept the messages that the unit is sending to the cloud every minute (I’ve integrated my solar inverter this way, works very nicely). There’s this GitHub project that does exactly that, with 3 different methods. I tried this out before, without success - maybe I’ll give it another go…
Unfortunately, I think Mitsubishi have since added encryption to their messages, which make it significantly harder.

2 Likes
20

I’d forgotten about this for a few days, but yesterday (May 8th) I was able to switch back to polling every minute.

1 Like
21

@Timbones did that hold for you? I’ve been getting intermittent stale data ever since the firmware update (most recently with a five minute polling interval).