Using Emoncms alongside Pasivliving MMSP

I don’t know how many people signed up for the RHI MMSP package along with their ASHP install, but I did. This solution came with a Sontex heat-meter, electricity meter and a monitoring platform. There was a up-front cost for the equipment/platform, but the government pays a sum quarterly over 7 years which covers the full cost. This scheme closed along with RHI a couple of years ago.

Anyway, the monitoring solution provided (Pasivliving) is pretty poor and I’d like to start using Emoncms. I can’t however rip out the MMSP setup and replace it as, per the contract, it needs to remain installed and working for the full 7 years.

What I was wondering is if anyone was/is in the same situation and what approach you have taken?

Few approaches come to mind:

  1. Install additional heat-meter and electricity meter dedicdated to emonPi/encomCMS
  2. Try to find a way to “split” the Modbus connectios to allow two masters (Pasivliving hub + emonPi)
  3. Poll the Pasviliving hub for heat/electricty and provide data to Emoncms.
  4. Poll the Pasviliving monitoring site for heat/electricty and provide data to Emoncms.
  5. Use the existing meters but interface via alternative protocols.
    i) Use the Sontex m-bus interface (pasivliving uses modbus)
    i) Use pulse output of the SDM630 (pasivliving uses modbus)

Given I can’t find any way nice way to pull the information from the local hub or the monitoring platform, I’m thinking about option 4). Keen to understand if anyone else has already gone down one of these paths though…

Hi, http://passivhub.local/portal/ provides access to more information than is visible on the main portal, including flow, RWT, cumulative consumption and production, updated every 2 minutes. Someone on another forum pointed this out to me and it is possible to write code to extract the information from the local portal and store it in a CSV file or upload it elsewhere- in my case to a Google spreadsheet. Once in the Google sheet, formulae display different calculations including daily consumption, COP and cost.
I can share the python code if that would be helpful. I presume the data could be uploaded to the OEM site, but I didn’t know how to do this. I have no experience of Emoncms, but would be interested to learn more.

I looked here (http://passivhub.local/portal/) earlier, but I think I was on a different subnet at the time and it didn’t load!

You are right though, all data is available locally via the hub. So the preferred solution would be to pull the data from here and push it into emonCms. Let me do a bit of digging and see if this is available via a API that can be polled to avoid the need for screen-scraping.

If you want csv, isn’t the easiest option to download from passivliving.com ?

http://passivhub.local/v/1/cachedevents?eventId=*

pulls the data in JSON form.

So just need a way to poll, parse and insert data into ecomCms. Need to look at ecomCms to see if there is anything there that could help, or if I need to put together a seperate process to do this.

Now

Hi Daniel,

This was one of my initial attempts. Let me know how you get on integrating with EmonCMS

import requests
import csv
import time
url = ‘http://passivhub.local/v/1/cachedevents?eventId=*’
headers = {
‘value’: ‘application/json, text/javascript, /; q=0.01’,
‘accept’: ‘application/json, text/javascript, /; q=0.01’
}
page = requests.get(url, headers=headers)
file = open(‘passiv.csv’, ‘a’, newline = ‘’)
writer = csv.writer(file)
writer.writerow([‘Role’, ‘Value’, ‘TimeStamp’])
i = 1
while i < 6:
page = requests.get(url, headers=headers)
data = page.json()
for item in data:
role = item[“hubRole”]
value = item[“eventValue”]
ts = item[“eventTimeStamp”]
writer.writerow([
role,
value,
ts])
time.sleep(120)
i += 1
file.close()
print(‘CSV created’)

This is doc:

https://docs.openenergymonitor.org/emoncms/postingdata.html
https://emoncms.org/site/api#input

You can download CVS’s from here: PassivLiving

Inputs are:

Thanks for the links. Unfortunately I find that the OEM instructions assume a level of knowledge that is several levels above mine. If you can point me in the direction of a more idiot’s guide, that would be great.

Devil’s advocate:

“Or what?”

It’s not a very good system. Binning it in favour of OEM would be an improvement. If the “oops the modbus cable fell off” actually gets actioned you can always pop it back in.

1 Like

Not sure what the Pasivliving hub is implemented with, but I’ve found you can add change the polling frequency and add additional modbus registers via a ReST API (http://192.168.0.237/v/1/hans/config)! For example, by default, electrical power is not exposed (only accumulated kWh), but I’ve now added that. Now to find a way to post these to Emoncms!

@markocoheat With this finding, it think I’ll leave this solution in place, enhance the data it pulls from modbus meter and then pull from this. Rather than pull the modbus cable out and hook up to raspberry pi.

1 Like

Just look at emoncms docs, you can use the hosted version at https://emoncms.org/

  1. Create emoncms account
  2. Add heatpump level 3 device with will add inputs and feeds.
  3. Add heatpump app
  4. Send values to via ReST calls.

I managed to get this working yesterday. I poll Pasivliving hub, parse out the values and then post them to emconcms. I had chatGPT generate the code for me, but happy to share if required.

The other thing I did was change the pasivliving hub configuration to:

  1. Poll modbus more frequenty
  2. Expose electricty meter power from Modbus meter via pasivling api.

One thing to note though, because of the way Pasivliving works, the 2min polling frequency used by default is baked into their visualization platform. If you change the frequency to 30s, like I have, this means more frequent updates on emoncms, but it means that Pasivliving breaks and it can’t deal with the adjusted frequency and thinks power is now 1/4.

1 Like

Awesome!

Next step is to pull additional information from ebusd:

  • Target temp
  • Outside temp
  • Heating On

I’m wondering though, if there isn’t a argument for just doing ebusd->emoncms (all the information is available via ebus). The caveat is that I don’t think vaillant power consumption number includes internal valves/pumps/controls.

All up and running:

  • Existing Pasivliving hub (and MID-approved Sontex/Eastron meters).
  • Cloud-based Encomcs.
  • Short python script running on a raspberry Pi polls Pasivliving hub and pushes data to Encomcs every 30s.
  • Extended python script to also pull some data from ebusd (target flow temp, outside temp) every 30s.
  • Configure Encomcs feeds and heat pump app.
  • Register with heatpumpmonitor.org

(because pasivliving setup uses electricity meter L2 voltage to determine if DHW is active, I am also pushing this to emoncms so it can demarcate DHW blue. The yellow heating demarcation is based on desiredFlowTemp>0).

FYI @MikeJH

All the Pasivliving stuff is still connected and working, just (because I tweaked the hub polling frequency) the web-visualization is broken because it assumes 120s polling. That said, enoncms heat pump app is much better, so I don’t care about this!

3 Likes

Yes please, the code would be very helpful. I am not familiar with ReST calls. I am happy to poll every 2 minutes - in fact I only send values to Google sheets every 4 minutes and only if flow > 0. Is it necessary to get the power data from Passiv? I presumed Emoncms would display that from the cumulative energy data and timestamps, which is what I do in Sheets. If so, can you explain how you did it? Thanks

Something like this. I realized that you don’t actually need to use names already defined in emconcms as it will automatically create the inputs once you post data to it. So you need to

  1. Create emconcms cloud account.
  2. Replace api key and run script.
  3. Go to emconcms input, find the inputs and create feeds for relevant fields (converting/specifiying units if/as required)
  4. Create econcms MyHeatPump app using feeds.

Give me a shout if you need any help or further pointers. You can run this script headless by setting up a service, or if not with “nohup python3 passivlivingToEmconcms.py &”

import requests
import time
import json

# URL for fetching JSON document
passivHubUrl = "http://passivhub.local/v/1/cachedevents?eventId=*"

# URL for posting data to emoncms
emoncmsInputBaseUrl = "https://emoncms.org/input/post"

# emoncms api key
emoncmsApiKey = "[YOUR API KEY]"

# emconcms node name
emconcmsNode = "PassivLivingHP"

# polling frequency
pollingFreq = 120

while True:
    try:
        # Fetch data from passivliving hub on local network
        data = requests.get(passivHubUrl).json()
        print(data)
        # Construct JSON payload
        payload = {}
        for event in data:
            if 'serverEventName' in event and 'eventValue' in event:
                payload[event['serverEventName']] = event['eventValue']

        # Post data to emconcms
        if payload:
            fullJson = '{' + ','.join(f'"{key}":{value}' for key, value in payload.items()) + '}'
            emoncmsRequestUrl = f'{emoncmsInputBaseUrl}?node={emconcmsNode}&fulljson={fullJson}&apikey={emoncmsApiKey}'
            print(emoncmsRequestUrl)
            response = requests.get(emoncmsRequestUrl)
            if response.status_code == 200:
                print("Data posted successfully.")
            else:
                print(f"Failed to post data. Status code: {response.status_code}")
    except Exception as e:
        print(f"An error occurred: {e}")

    # Wait before next iteration
    time.sleep(pollingFreq)

If you need heat pump input power (W measured by electricity meter), I can tell you how to enable this on the pasivliving hub box. Let me know…

Thanks Daniel, that’s very helpful. I will give it a go and see how I get on.

I think you will need electricity power. To enable this:

  1. GET http://passivhub.local/v/1/hans/config
  2. PUT http://passivhub.local/v/1/hans/config

In the PUT operation put the response from GET, but add a transducer with unique Id. In my case I needed to add this, but I don’t want to give you full payload as yours may be slightly different. The modbus address 0012 is based on Easton SDM630 three-phase meter.

https://www.postman.com/ is a good tool for doing these GET/PUT operations. Can use curl command, but postman easier.

                                {
                                    "transducerNo": "4",
                                    "zone": "GLOBAL",
                                    "role": "HEATPUMP_METER",
                                    "transducerType": "POWER",
                                    "address": "0012",
                                    "count": "2",
                                    "scaleFactor": "1"
                                }

Your script ran first time. Many thanks. However, I am struggling with Emoncms. I can see the inputs updating every 2 minutes, but when I click on the spanner beside say zone 1 temperature, then I don’t see the latest value (just NaN) and nothing appears if I look at the graph. I cannot find any useful instructions anywhere. Sorry to be such a newbie!

  1. For each input that you want to record (or that is needed for heat pump app) you need to add a process to “log to feed”. By default it will use “create new” and use a default name. You can also add additional processes (before log to feed) to convert W → kW etc as required.
  2. Then go to the feed tab and you should see feeds for all of the input you logged to feed. Both the input and the feed values should update every 2mins.
  3. Once you have this working go to apps and create a new “MyHeatPump” app. The configuration of this will request feed names for the different parameters it needs. In some cases you might need to go back and convert units. Once these are all mapped you can save and lauch the app.