Using Emoncms alongside Pasivliving MMSP

markocoheat · 2024-02-06T21:54:34.040Z

dfeist:

I can’t however rip out the MMSP setup and replace it as, per the contract, it needs to remain installed and working for the full 7 years.

Devil’s advocate:

“Or what?”

It’s not a very good system. Binning it in favour of OEM would be an improvement. If the “oops the modbus cable fell off” actually gets actioned you can always pop it back in.

dfeist · 2024-02-07T19:18:29.143Z

Not sure what the Pasivliving hub is implemented with, but I’ve found you can add change the polling frequency and add additional modbus registers via a ReST API (http://192.168.0.237/v/1/hans/config)! For example, by default, electrical power is not exposed (only accumulated kWh), but I’ve now added that. Now to find a way to post these to Emoncms!

@markocoheat With this finding, it think I’ll leave this solution in place, enhance the data it pulls from modbus meter and then pull from this. Rather than pull the modbus cable out and hook up to raspberry pi.

dfeist · 2024-02-08T11:40:26.004Z

Just look at emoncms docs, you can use the hosted version at https://emoncms.org/

Create emoncms account
Add heatpump level 3 device with will add inputs and feeds.
Add heatpump app
Send values to via ReST calls.

I managed to get this working yesterday. I poll Pasivliving hub, parse out the values and then post them to emconcms. I had chatGPT generate the code for me, but happy to share if required.

The other thing I did was change the pasivliving hub configuration to:

Poll modbus more frequenty
Expose electricty meter power from Modbus meter via pasivling api.

One thing to note though, because of the way Pasivliving works, the 2min polling frequency used by default is baked into their visualization platform. If you change the frequency to 30s, like I have, this means more frequent updates on emoncms, but it means that Pasivliving breaks and it can’t deal with the adjusted frequency and thinks power is now 1/4.

marko · 2024-02-08T14:04:11.773Z

Awesome!

dfeist · 2024-02-08T16:35:47.969Z

Next step is to pull additional information from ebusd:

Target temp
Outside temp
Heating On

I’m wondering though, if there isn’t a argument for just doing ebusd->emoncms (all the information is available via ebus). The caveat is that I don’t think vaillant power consumption number includes internal valves/pumps/controls.

dfeist · 2024-02-09T13:41:33.834Z

All up and running:

Existing Pasivliving hub (and MID-approved Sontex/Eastron meters).
Cloud-based Encomcs.
Short python script running on a raspberry Pi polls Pasivliving hub and pushes data to Encomcs every 30s.
Extended python script to also pull some data from ebusd (target flow temp, outside temp) every 30s.
Configure Encomcs feeds and heat pump app.
Register with heatpumpmonitor.org

(because pasivliving setup uses electricity meter L2 voltage to determine if DHW is active, I am also pushing this to emoncms so it can demarcate DHW blue. The yellow heating demarcation is based on desiredFlowTemp>0).

FYI @MikeJH

All the Pasivliving stuff is still connected and working, just (because I tweaked the hub polling frequency) the web-visualization is broken because it assumes 120s polling. That said, enoncms heat pump app is much better, so I don’t care about this!

image1112×504 37.2 KB

MikeJH · 2024-02-12T17:02:37.883Z

dfeist:

I had chatGPT generate the code for me, but happy to share if required

Yes please, the code would be very helpful. I am not familiar with ReST calls. I am happy to poll every 2 minutes - in fact I only send values to Google sheets every 4 minutes and only if flow > 0. Is it necessary to get the power data from Passiv? I presumed Emoncms would display that from the cumulative energy data and timestamps, which is what I do in Sheets. If so, can you explain how you did it? Thanks

dfeist · 2024-02-13T14:57:18.013Z

Something like this. I realized that you don’t actually need to use names already defined in emconcms as it will automatically create the inputs once you post data to it. So you need to

Create emconcms cloud account.
Replace api key and run script.
Go to emconcms input, find the inputs and create feeds for relevant fields (converting/specifiying units if/as required)
Create econcms MyHeatPump app using feeds.

Give me a shout if you need any help or further pointers. You can run this script headless by setting up a service, or if not with “nohup python3 passivlivingToEmconcms.py &”

import requests
import time
import json

# URL for fetching JSON document
passivHubUrl = "http://passivhub.local/v/1/cachedevents?eventId=*"

# URL for posting data to emoncms
emoncmsInputBaseUrl = "https://emoncms.org/input/post"

# emoncms api key
emoncmsApiKey = "[YOUR API KEY]"

# emconcms node name
emconcmsNode = "PassivLivingHP"

# polling frequency
pollingFreq = 120

while True:
    try:
        # Fetch data from passivliving hub on local network
        data = requests.get(passivHubUrl).json()
        print(data)
        # Construct JSON payload
        payload = {}
        for event in data:
            if 'serverEventName' in event and 'eventValue' in event:
                payload[event['serverEventName']] = event['eventValue']

        # Post data to emconcms
        if payload:
            fullJson = '{' + ','.join(f'"{key}":{value}' for key, value in payload.items()) + '}'
            emoncmsRequestUrl = f'{emoncmsInputBaseUrl}?node={emconcmsNode}&fulljson={fullJson}&apikey={emoncmsApiKey}'
            print(emoncmsRequestUrl)
            response = requests.get(emoncmsRequestUrl)
            if response.status_code == 200:
                print("Data posted successfully.")
            else:
                print(f"Failed to post data. Status code: {response.status_code}")
    except Exception as e:
        print(f"An error occurred: {e}")

    # Wait before next iteration
    time.sleep(pollingFreq)

dfeist · 2024-02-13T15:05:13.508Z

If you need heat pump input power (W measured by electricity meter), I can tell you how to enable this on the pasivliving hub box. Let me know…

MikeJH · 2024-02-13T20:35:42.840Z

Thanks Daniel, that’s very helpful. I will give it a go and see how I get on.

dfeist · 2024-02-13T21:51:50.481Z

I think you will need electricity power. To enable this:

GET http://passivhub.local/v/1/hans/config
PUT http://passivhub.local/v/1/hans/config

In the PUT operation put the response from GET, but add a transducer with unique Id. In my case I needed to add this, but I don’t want to give you full payload as yours may be slightly different. The modbus address 0012 is based on Easton SDM630 three-phase meter.

https://www.postman.com/ is a good tool for doing these GET/PUT operations. Can use curl command, but postman easier.

                                {
                                    "transducerNo": "4",
                                    "zone": "GLOBAL",
                                    "role": "HEATPUMP_METER",
                                    "transducerType": "POWER",
                                    "address": "0012",
                                    "count": "2",
                                    "scaleFactor": "1"
                                }

MikeJH · 2024-02-13T22:09:38.317Z

dfeist:

Create emconcms cloud account.

Replace api key and run script.

Go to emconcms input, find the inputs and create feeds for relevant fields (converting/specifiying units if/as required)

Create econcms MyHeatPump app using feeds.

Your script ran first time. Many thanks. However, I am struggling with Emoncms. I can see the inputs updating every 2 minutes, but when I click on the spanner beside say zone 1 temperature, then I don’t see the latest value (just NaN) and nothing appears if I look at the graph. I cannot find any useful instructions anywhere. Sorry to be such a newbie!

Process list set up1488×424 61.6 KB

dfeist · 2024-02-13T22:47:50.852Z

For each input that you want to record (or that is needed for heat pump app) you need to add a process to “log to feed”. By default it will use “create new” and use a default name. You can also add additional processes (before log to feed) to convert W → kW etc as required.
Then go to the feed tab and you should see feeds for all of the input you logged to feed. Both the input and the feed values should update every 2mins.
Once you have this working go to apps and create a new “MyHeatPump” app. The configuration of this will request feed names for the different parameters it needs. In some cases you might need to go back and convert units. Once these are all mapped you can save and lauch the app.

Robert.Wall · 2024-02-13T23:02:10.823Z

Note a very important point: your Feed Interval (you choose and set it when creating the Feed, you cannot change it afterwards) must be equal to or slightly longer than the time between sets of data coming in. So if your data arrives every 2 minutes you should make your Feed Interval the same or ideally slightly longer than this, never shorter. Otherwise, you will record NULL values in the data if a value gets delayed in processing.

The default is 10 s, which would be very bad in your case.

The useful documentation for emonCMS starts here: emonCMS: Getting started emonPi/Base — OpenEnergyMonitor 0.0.1 documentation though it might be better to start with the 3rd page Emoncms Core Concepts — OpenEnergyMonitor 0.0.1 documentation

MikeJH · 2024-02-14T21:13:44.123Z

Thanks Daniel and Robert. I have watched a couple of youtube videos and reread the documentation. I have managed to get the feeds done and a new heat pump app. I haven’t got an electric power input or feed yet and I wonder if that explains the error message that keeps popping up when I launch the App. Have you any thoughts?

Apps error message1148×804 135 KB

dfeist · 2024-02-14T21:32:16.379Z

Not seen that error, but it’s likely due to this. I explained how to add power, given this a go?

MikeJH · 2024-02-14T21:53:12.660Z

I haven’t tried to add electric power. I looked at the output of GET http://passivhub.local/v/1/hans/config

None of the addresses seemed to match yours and the scalefactor for the “transducerType”: “POWER” is 0.01. I worry that I could mess it up big time due to my inexperience.

part of it looks like this:
“pollingRegisterMap”: [
{
“transducerNo”: “1”,
“zone”: “GLOBAL”,
“role”: “HEATMETER_FLOW_TEMPERATURE”,
“transducerType”: “TEMPERATURE”,
“address”: “822”,
“count”: “2”,
“scaleFactor”: “1”
},
{
“transducerNo”: “2”,
“zone”: “GLOBAL”,
“role”: “HEATMETER_RETURN_TEMPERATURE”,
“transducerType”: “TEMPERATURE”,
“address”: “824”,
“count”: “2”,
“scaleFactor”: “1”
},
{
“transducerNo”: “3”,
“zone”: “GLOBAL”,
“role”: “HEATMETER_OVERALL”,
“transducerType”: “ENERGY”,
“address”: “202”,
“count”: “2”,
“scaleFactor”: “1”
},
{
“transducerNo”: “4”,
“zone”: “GLOBAL”,
“role”: “HEATMETER_OVERALL”,
“transducerType”: “POWER”,
“address”: “802”,
“count”: “2”,
“scaleFactor”: “0.01”
},
{
“transducerNo”: “5”,
“zone”: “GLOBAL”,
“role”: “HEATMETER_OVERALL”,
“transducerType”: “FLOW”,
“address”: “812”,
“count”: “2”,
“scaleFactor”: “0.1”

dfeist · 2024-02-14T22:02:05.407Z

Thats the heatmeter section, there should be another section for the electricity meter.

MikeJH · 2024-02-15T12:28:20.510Z

Yes, of course! Sorry about that. Even so, I am reluctant to change anything in the local portal. I have added some code to my script to calculate average electric power over the previous 2 minutes and post that value. Hopefully that will get rid of the error message - so far so good. Thanks for all your help.

dfeist · 2024-02-15T15:49:52.196Z

The last (optional) step is to add your install to HeatpumpMonitor.org,

Regarding the hans config on the passivliving hub, while I haven’t explicitly sent an invalid request to see what happens, it is unlikely that you can break anything. It obviously makes sense to keep hold of the response you got from the GET method as a backup, but I can’t see you haveing any issues. That said, if the approach you are using achieves the same outcome that makes sense too. I edited hans config as i wanted 30s

Did you get the heat pump dashboard up and running? You should also be able to use the diverter status from passivliving to highlight DHW.