Script for feeding Honeywell Evohome data into Emoncms

As this might also be useful for others who have the Evohome system of Honeywell; below a Python script I use to send temperature values of my house to Emoncms.


#!/usr/bin/python
# Script to monitor and read temperatures from Honeywell EvoHome Web API

# Load required libraries
import requests
import json
import time

# Set your Evohome login details in the 2 fields below
USERNAME = 'LOGONNAME'
PASSWORD = 'PASSWORD'

# Initial JSON POST to the website to return your userdata
url = 'https://tccna.honeywell.com/WebAPI/api/Session'
postdata = {'Username': USERNAME, 'Password': PASSWORD, 'ApplicationId': '91db1612-73fd-4500-91b2-e63b069b185c'}
header = {'content-type': 'application/json'}
data = json.dumps(postdata)
response = requests.post(url, data=data, headers=header)
str_response = response.content.decode("utf-8")
# print(str_response)
userinfo = json.loads(str_response)

# Extract the sessionId and your userid from the response
userid = userinfo['userInfo']['userID']
sessionId = userinfo["sessionId"]

while True:
    params = ""
    # Next, using your userid, get all the data back about your site
    url = 'https://tccna.honeywell.com/WebAPI/api/locations?userId=%s&allData=True' % userid
    header['sessionId'] = sessionId
    response = requests.get(url, data=json.dumps(postdata), headers=header)
    str_response = response.content.decode("utf-8")
    # print(str_response)
    fullData = json.loads(str_response)[0]

    for device in fullData['devices']:
        if params != "":
            params += ","
        # Add every device with values to url json string
        params += device['name'] + "_temp:"
        params += str(device['thermostat']['indoorTemperature'])
        params += "," + device['name'] + "_thermostat:"
        params += str(device['thermostat']['changeableValues']['heatSetpoint']['value'])
    print(params)
   
    # Below I will generate the url I am using two URLS: 1 for legacy emoncms and 1 for Emoncms9
    url = "http://URLTOEMONCMS/input/post.json?node=2&apikey=APIKEY&json={"
    url += params + "}"
    print(url)
    url2 = "http://URLTOSECONDEMONSMSINSTALLATION/input/post.json?node=2&apikey=APIKEY&json={"
    url2 += params + "}"
    print(url2)

    # now open the url and send the values
    response = requests.get(url, headers=header)
    response2 = requests.get(url2, headers=header)


    # read the response and print it
    str_response = response.content.decode("utf-8")
    print(str_response)
    str_response2 = response2.content.decode("utf-8")
    print(str_response2)

    #clear the parameters for the next round   
    del params

    # sleep 60secs
    time.sleep(90)

1 Like

I wondered if you could help me out and explain how this script gets integrated into EmonCMS? It is just what I am looking for but not sure what feeds I need, how to integrate it etc - although experienced in many programming languages etc just not aware of how you go about this in emoncms.

Thanks

If you replace this

http://URLTOEMONCMS/

with the url to your emoncms instance, the data should appear in the ā€˜Inputsā€™ page.

You will need to run the script from the base system.

Thanks for your comment.

Yes I have started to look at it and I will need to run the python script I assume from a linux terminal/ as a startup process /crontab.

I will need to make specific the Honeywell and EMONCMS user/passwords/connections strings/api ids. I was wondering if there is an environment within EMONCMS that periodically runs a a script and not external as this suggests. I donā€™t really like looping scripts so I might remove that bit and use crontab instead.

I am also not sure why you print it - presumably to stdio - in which case is this just for debugging or is there a more fundamental reason for this - is it re-directedā€¦

Food for thought.

No there isnā€™t and crontab is not really ideal for something that needs to run this regularly. A looping python with a sleep works pretty well.

I suspect the printing is for degbug. Iā€™ve never used the script.

Something to be aware of with regards to the Python sleep() command.

Using 5 seconds for the purposes of illustrationā€¦

A simple sleep delay results in a 5 second delay between measurements.
If the measurement takes 100 ms, the time between measurements is 5.1 seconds.
Hereā€™s a simple way to ensure the measurements are taken at 5 second intervals:

import time
starttime=time.time()
while True:
  print "tick"
  time.sleep(5.0 - ((time.time() - starttime) % 5.0))

Hereā€™s another:

import time, traceback

def every(delay, task):
  next_time = time.time() + delay
  while True:
    time.sleep(max(0, next_time - time.time()))
    try:
      task()
    except Exception:
      traceback.print_exc()
      # in production code you might want to have this instead of course:
      # logger.exception("Problem while executing repetitive task.")
    # skip tasks if we are behind schedule:
    next_time += (time.time() - next_time) // delay * delay + delay

def foo():
  print("foo", time.time())

every(5, foo)

If you want to do this without blocking your remaining code, you can use this to let it run in its own thread:

import threading
threading.Thread(target=lambda: every(5, foo)).start()

This solution combines several features rarely found combined in the other solutions:

  • Exception handling: As far as possible on this level, exceptions are handled properly, i. e. get logged for debugging purposes without aborting the program.
  • No chaining: The common chain-like implementation (for scheduling the next event) you find in many answers is brittle in the aspect that if anything goes wrong within the scheduling mechanism ( threading.Timer or whatever), this will terminate the chain. No further executions will happen then, even if the reason of the problem is already fixed. A simple loop and waiting with a simple sleep() is much more robust in comparison.
  • No drift: This solution keeps an exact track of the times it is supposed to run at. There is no drift depending on the execution time (as in many other solutions).
  • Skipping: This solution will skip tasks if one execution took too much time (e. g. do X every five seconds, but X took 6 seconds). This is the standard cron behavior (and for a good reason). Many other solutions then simply execute the task several times in a row without any delay. For most cases (e. g. cleanup tasks) this is not wished. If it is wished, simply use next_time += delay instead.

Ref:

1 Like

So I have managed to implement this solution. My requirement was to display the house temperatures on an old tablet along with state of charge for my BMW i3; whether our solar is producing enough energy to start up appliances such as the washing machine and finally display the current costing for Octopusā€™ Agile tariff.

I have a home website that displays other useful information so added a page to this which displays the Octopus Agile App alongside a home built dashboard for the BMW i3,

I would rather lose the export feed in the Octopus App so now that I have this working I will look to have a feed directly from Octopus so I can remove the export feed. I would appreciate it if the export could be optional but who knows that might come up in a later version of it.

So you need your username and password for EvoHome ( and it needs to have a wifi connection). You need your write API key and local host/ip address for your emoncms installation.
You may need to use pip to download the python modules but on the emoncms it seems to have them. My pop!_os laptop I use for development didnā€™tā€¦

I created on the pi a directory - readtemp under /home/pi and put in a file readtemp.py ( python2)

 #!/usr/bin/python
 import requests
 import json
 import time
 import math

 USERNAME = '***'
 PASSWORD = '***'

 #### Initial JSON POST to the website to return your userdata
 url = 'https://tccna.honeywell.com/WebAPI/api/Session'
 postdata = {'Username': USERNAME, 'Password': PASSWORD, 'ApplicationId': '91db1612-73fd-4500-91b2-e63b069b185c'}
 header = {'content-type': 'application/json'}
 data = json.dumps(postdata)
 response = requests.post(url, data=data, headers=header)
 str_response = response.content.decode("utf-8")

#print(str_response)
userinfo = json.loads(str_response)

#### Extract the sessionId and your userid from the response
userid = userinfo['userInfo']['userID']
sessionId = userinfo["sessionId"]
while True:
    params = ""

    # Next, using your userid, get all the data back about your site
    url = 'https://tccna.honeywell.com/WebAPI/api/locations?userId=%s&allData=True' % userid
    header['sessionId'] = sessionId
    response = requests.get(url, data=json.dumps(postdata), headers=header)
    str_response = response.content.decode("utf-8")
    #print(str_response)
    fullData = json.loads(str_response)[0]

    for device in fullData['devices']:
        stemp = ""
        if params != "":
            params += ","
        # Add every device with values to url json string
        if device['thermostatModelType'] == "DOMESTIC_HOT_WATER": params += "hot_water_temp:"
        else:  params += device['name'] + "_temp:"
        stemp = str(math.ceil((device['thermostat']['indoorTemperature'])*10)/10)
        params += stemp.strip(".0")
        #params += "," + device['name'] + "_thermostat:"
        #params += str(device['thermostat']['changeableValues']['heatSetpoint']['value'])
    #print(params)

    # Below I will generate the url I am using two URLS: 1 for legacy emoncms and 1 for Emoncms9
    url = "http://***.***.***.***/emoncms/input/post.json?node=evohome&apikey=****&json={"
    url += params + "}"
    #print(url)


    # now open the url and send the values
    response = requests.get(url, headers=header)

    # read the response and print it
    str_response = response.content.decode("utf-8")
    #print(str_response)

    #clear the parameters for the next round
    del params

    # sleep 60secs
    time.sleep(120)

This file needs the details mention above and gets loaded in the directory as readtemp.py and chmod +x to make it executable.

It has a infinite loop with 2 min delay. I donā€™t like this approach and may move to a bash running a script every 2 minutes. The reason for this is that if there is an error that is temporary the shell script will re-run it. Otherwise the python will crash. I prefer to see it crash record the output and debug what is wrongā€¦

To get it to run at boot up edit the /etc/rc.local file and at the very bottom but before exit 0

run readtemp program

/usr/bin/python2 /home/pi/readtemp/readtemp.py &

use which python2 to get the path. I always put the full path in a startup script just in case PATH is not there.

So it works for meā€¦ ( ps -eaf | grep read - you should see it running.)

Its been an interesting few hours. I am not a python person more golang; C++; PHP etc etc so have learnt a lot.

The edit for the post seems to use the # for some reason and so spoils the code displayā€¦Not sure how to get rid of this.

Thanks to everyone for your help.

[Edited for presentation - Moderator (RW)]

For future reference, when posting code or bash output, put in 3 ā€˜backticksā€™ (found at the top left of the keyboard normally) on a line on their own, then the code, then 3 more backticks on a line following the code.

    ```
    code
    ```

If it is something like php you can add a language identifier that after the first 3 backticks so ```php

I really do not understand what you are saying here.

Thanks for info about code.

Re Octopus app. I do not use the export tariff but use a standard fee. Having the export line on the graph is confusing anyway so I would prefer to be able to remove that line and just have the rest. Hope that answers your question

Do you mean the Octopus Agile App? If so that line is the half hour cost on Octopus Agile - nothing to do with Export, and that is the point of the graph. If you donā€™t want it, donā€™t use that App, use the My Electric instead.

If you encapsulate your Pythoin routine in a try/except/else/finally construct, the script can continue running, i.e. not abort.

Brian, Yes I mean the Octopus Agile App. Yes I want to remove the optional outgoing line seen below ( in purple)

My Electric does not have the Agile cost shown.

Bill,
To be controversial and I have taught programming for some decades now and audited code. I have reservations around the try/except construct. It was first introduced to tidy up error handling and works very well in that fashion. However it does encourage lazy programming in that it encourages happy path programming and discourages the handling of exception events. I would prefer to handle the events and use the construct for those that are not normally seen simply due to the expense of the construct. Of course if you are disciplined and handle all normal errors and use this for an exception then that works well.

For this script EvoHome randomly decides not accept your request and returns a different json message. The code does not capture this an an exception is produced. So I placed in the script the following:

if 'userInfo' not in userinfo: 
    #print "not allowed"
    exit(1)

I also note the infinite loop does not work in this case as both requests need to be executed sequentially. You canā€™t request the first then loop around the second.

So you will realise I have gone to a shell script calling the python script. This is also lazy and in slow time will write the script properly.

I have also changed the feed time to 4 minutes. It is a house after all and I suspect the temp does not change that much - that might help EvoHome.

Finally I tried and failed at a rounding approach by stripping out ā€œ.ā€ and ā€œ0ā€ and using ceil. This did not work for temperatures such as 20.00 which ended up as 2. I should have realised this - there does not seem to be a rounding function that I can find in python so in slow time will work out a better function.

Thanks for all your help Bill, I can see the arguments for the construct and I hope you can see why I use it sparingly but in this case I suspect by handling the refusal to connect I should really put the construct in and will do in slow time.

Now back to my paid job :slight_smile:

1 Like

Ah, I see. There must be an assumption about the tariff. @TrystanLea?

So are you on the Agile Tariff?

Hi,
Yes I am on the Agile Tariff. However, I am not on the outgoing - being paid for what you send to the grid preferring the fixed export rate. The idea is for the household to see when the costs are low and use the high energy items then, keeping in mind the solar contribution. The outgoing line confuses as they all have different scales. So removing it would be great as the rest is exactly what I wanted.

1 Like

Itā€™s possible to remove the export tariff line and solar by not selecting a solar_kwh feed, but not currently possible to hide the tariff but show the solar on the graph, or configure a flat export tariff, which might be better?

I would prefer the export Tariff removed but keep the solar display. So an option would be best.I use SSE for the fixed export but it is of little consequence as it is fixed.
If you could do that it would be greatly appreciated.

1 Like

Iā€™ve created an enhancement request on the repository for this app to keep track of this Show/hide export tariff line option Ā· Issue #148 Ā· emoncms/app Ā· GitHub so that it can be considered as part of the next round of development on this app.

1 Like

thanks much appreciated