Fetch Tesla Power Wall State of charge using EmonHub

emonHub
1

Reflecting on @PeteF’s suggestion here My Solar PV Battery app - #32 by PeteF I wonder whether a simple EmonHub interfacer would be a good approach to fetch Tesla PowerWall state of charge? I know we have multiple different ways of doing this now and the NodeRED approach is a neat approach as well, but I wonder whether including this in EmonHub provides an easier way for users to bring this data in, without needing a separate Pi running NodeRed?

UPDATED: This feature has now been merged into the main emonhub branch, here’s the merged pull request for reference:
https://github.com/openenergymonitor/emonhub/pull/93:

Steps to use

Run emonPi update as normal… or:

cd /opt/openenergymonitor/emonhub
git pull origin emon-pi
sudo service emonhub restart

Then in emonhub.conf add the following interfacer configuration:

[[PowerWall]]
    Type = EmonHubTeslaPowerWallInterfacer
    [[[init_settings]]]
    [[[runtimesettings]]]
        pubchannels = ToEmonCMS,
        name = powerwall
        url = http://POWERWALL-IP/api/system_status/soe
        readinterval = 10

Example emonhub.log output:

2020-05-11 10:28:15,954 DEBUG    PowerWall  Request response: {"percentage":23.5}
2020-05-11 10:28:15,955 DEBUG    PowerWall  69 NEW FRAME : 
2020-05-11 10:28:15,956 DEBUG    PowerWall  69 Timestamp : 1589189295.96
2020-05-11 10:28:15,956 DEBUG    PowerWall  69 From Node : powerwall
2020-05-11 10:28:15,957 DEBUG    PowerWall  69    Values : [23.5]
2020-05-11 10:28:15,957 DEBUG    PowerWall  69 Sent to channel(start)' : ToEmonCMS
2020-05-11 10:28:15,958 DEBUG    PowerWall  69 Sent to channel(end)' : ToEmonCMS
2020-05-11 10:28:16,069 DEBUG    MQTT       Publishing: emon/powerwall/soc 23.5
2020-05-11 10:28:16,070 INFO     MQTT       Publishing: emonhub/rx/powerwall/values 23.5

The part Im not sure about is turning off PYTHONHTTPSVERIFY, ideally it would be possible to do that specifically for this request? Do all powerwalls require this?

2

This could allow a user to fetch the data by just adding the emonhub.conf entry in the emoncms editor e.g:

image
image1051×687 69.3 KB

3

It looks like the version of python currently running on emonSD (version 2.7.16 on mine) may not verify the certificate by default? but there are also solutions to more recent versions python ignore certificate validation urllib2 - Stack Overflow

We will be updating emonhub to python3 soon and it looks like there is a couple of solutions there in terms of either ignoring the certificate or allowing a user to specify the public certificate of the powerwall perhaps?

4

I followed your instructions but am getting nothing powerwall-related in emonhub.conf :frowning:
I have restarted emonhub.
Log level: debug.
Here is my emonhub.conf entry:

[[PowerWall]]
Type = EmonHubTeslaPowerWallInterfacer
[[[init_settings]]]
[[[runtimesettings]]]
pubchannels = ToEmonCMS,
name = powerwall
url = http://172.16.20.110/api/system_status/soe
readinterval = 10

Using emonSD-17Oct19
Version: low-write 10.2.2
Everything up to date AFAIK.

I’ve probably done something stupid.
Not sure where to start trying to diagnose…

5

Thanks @PeteF, what do you see in the emonhub log window? At the top you should see the interfacer being added e.g:

2020-05-11 12:01:45,096 INFO     MainThread Creating EmonHubTeslaPowerWallInterfacer 'PowerWall' 
2020-05-11 12:01:45,098 INFO     MainThread Setting PowerWall url: http://192.168.1.71/api/system_status/soe
2020-05-11 12:01:45,098 INFO     MainThread Setting PowerWall readinterval: 10
2020-05-11 12:01:45,099 INFO     MainThread Setting PowerWall name: powerwall
2020-05-11 12:01:45,099 DEBUG    MainThread Setting PowerWall pubchannels: ['ToEmonCMS']

If you run restart and then open the log immediately you should hopefully see that:

sudo service emonhub restart
tail -f /var/log/emonhub/emonhub.log

I assume you updated emonhub and switched to the powerwall branch above? What does it show when you type the following:

cd /opt/openenergymonitor/emonhub
git status
6 
    pi@emonpi:~ $ !sudo
    sudo service emonhub restart
    pi@emonpi:~ $ !t
    tail -f /var/log/emonhub/emonhub.log
    2020-05-11 12:14:08,465 DEBUG    MQTT       Publishing: emon/emontx3cm16/rssi -55
    2020-05-11 12:14:08,466 INFO     MQTT       Publishing: emonhub/rx/16/values 25275,248.39,47,8,511,-3,4000,867,10343,8417,300,300,300,1,-55
    2020-05-11 12:14:09,255 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content'? 18 32 32 0 51 14 64 0 34 0 128 120 70 32 32 0 86 112 0 32 32 (-100)
    2020-05-11 12:14:09,762 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content'? 17 0 32 0 32 0 32 20 136 0 34 192 61 32 2 0 0 0 0 32 0 (-98)
    2020-05-11 12:14:09,969 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content'? 0 16 32 20 32 0 128 0 160 0 0 192 61 32 2 0 0 0 0 32 0 (-98)
    2020-05-11 12:14:11,873 INFO     MainThread EmonHub emonHub emon-pi variant v2.1.2
    2020-05-11 12:14:11,874 INFO     MainThread Opening hub...
    2020-05-11 12:14:11,875 INFO     MainThread Logging level set to DEBUG
    2020-05-11 12:14:11,876 INFO     MainThread Creating EmonHubJeeInterfacer 'RFM2Pi' 
    2020-05-11 12:14:11,880 DEBUG    MainThread Opening serial port: /dev/ttyAMA0 @ 38400 bits/s
    2020-05-11 12:14:13,892 INFO     MainThread RFM2Pi device firmware version: OK
    2020-05-11 12:14:13,893 INFO     MainThread RFM2Pi device current settings:  5 248 249 140 9 132 3 238 93 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 (-0)
    2020-05-11 12:14:13,894 INFO     MainThread Setting RFM2Pi frequency: 433 (4b)
    2020-05-11 12:14:14,896 INFO     MainThread Setting RFM2Pi group: 210 (210g)
    2020-05-11 12:14:15,899 INFO     MainThread Setting RFM2Pi quiet: 0 (0q)
    2020-05-11 12:14:16,901 INFO     MainThread Setting RFM2Pi baseid: 5 (5i)
    2020-05-11 12:14:17,903 INFO     MainThread Setting RFM2Pi calibration: 230V (1p)
    2020-05-11 12:14:18,906 DEBUG    MainThread Setting RFM2Pi subchannels: ['ToRFM12']
    2020-05-11 12:14:18,907 DEBUG    MainThread Setting RFM2Pi pubchannels: ['ToEmonCMS']
    2020-05-11 12:14:18,909 INFO     MainThread Creating EmonHubMqttInterfacer 'MQTT' 
    2020-05-11 12:14:18,914 DEBUG    MainThread Setting MQTT subchannels: ['ToEmonCMS']
    2020-05-11 12:14:18,919 DEBUG    MainThread Setting MQTT pubchannels: ['ToRFM12']
    2020-05-11 12:14:18,920 INFO     MainThread Setting MQTT nodevar_format_enable: 1
    2020-05-11 12:14:18,921 INFO     MainThread Setting MQTT node_format_enable: 1
    2020-05-11 12:14:18,922 INFO     MainThread Setting MQTT nodevar_format_basetopic: emon/
    2020-05-11 12:14:18,918 DEBUG    RFM2Pi     device settings updated: E i5 g210 @ 433 MHz q0 USA 0
    2020-05-11 12:14:18,925 INFO     MainThread Creating EmonHubEmoncmsHTTPInterfacer 'emoncmsorg' 
    2020-05-11 12:14:18,928 DEBUG    MainThread Setting emoncmsorg subchannels: ['ToEmonCMS']
    2020-05-11 12:14:18,930 DEBUG    MainThread Setting emoncmsorg pubchannels: ['ToRFM12']
    2020-05-11 12:14:18,931 INFO     MainThread Setting emoncmsorg url: https://emoncms.org
    2020-05-11 12:14:18,932 INFO     MainThread Setting emoncmsorg senddata: 1
    2020-05-11 12:14:18,933 WARNING  MainThread Setting emoncmsorg apikey: obscured
    2020-05-11 12:14:18,934 INFO     MainThread Setting emoncmsorg sendstatus: 1
    2020-05-11 12:14:19,029 DEBUG    RFM2Pi     device settings updated: E i5 g210 @ 433 MHz q0 USA 0
    2020-05-11 12:14:19,141 DEBUG    RFM2Pi     1 NEW FRAME : OK 15 220 55 85 85 121 52 126 85 133 170 79 85 113 85 238 90 85 85 122 161 170 170 63 80 85 85 53 89 85 85 101 32 101 32 101 32 84 85 85 85 (-46)
    2020-05-11 12:14:19,145 DEBUG    RFM2Pi     1 Timestamp : 1589195659.14
    2020-05-11 12:14:19,146 DEBUG    RFM2Pi     1 From Node : 15
    2020-05-11 12:14:19,147 DEBUG    RFM2Pi     1    Values : [25225, 248.76000000000002, 43, -48, 26, 36, 4027, 4294964271L, 1386, 3168, 300, 300, 300, 1]
    2020-05-11 12:14:19,148 DEBUG    RFM2Pi     1      RSSI : -46
    2020-05-11 12:14:19,149 DEBUG    RFM2Pi     1 Sent to channel(start)' : ToEmonCMS
    2020-05-11 12:14:19,150 DEBUG    RFM2Pi     1 Sent to channel(end)' : ToEmonCMS
    2020-05-11 12:14:19,226 INFO     MQTT       Connecting to MQTT Server
    2020-05-11 12:14:19,253 DEBUG    RFM2Pi     device settings updated: E i5 g210 @ 433 MHz q0 USA 0
    2020-05-11 12:14:19,330 INFO     MQTT       connection status: Connection successful
    2020-05-11 12:14:19,331 DEBUG    MQTT       CONACK => Return code: 0
    2020-05-11 12:14:19,338 DEBUG    emoncmsorg Buffer size: 1
    2020-05-11 12:14:19,359 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content'? 1 30 192 153 99 160 128 0 8 160 9 16 16 146 4 0 30 0 0 0 4 (-102)
    2020-05-11 12:14:19,433 INFO     MQTT       on_subscribe
    2020-05-11 12:14:19,463 DEBUG    RFM2Pi     device settings updated: E i5 g210 @ 433 MHz q0 USA 0
    2020-05-11 12:14:19,570 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content'? 16 135 136 5 82 56 32 35 128 50 32 243 32 0 32 32 136 179 8 160 36 (-96)
    2020-05-11 12:14:19,676 DEBUG    RFM2Pi     2 NEW FRAME : OK 5 255 249 148 9 147 3 245 93 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 (-0)
    2020-05-11 12:14:19,680 DEBUG    RFM2Pi     2 Timestamp : 1589195659.68
    2020-05-11 12:14:19,681 DEBUG    RFM2Pi     2 From Node : 5
    2020-05-11 12:14:19,682 DEBUG    RFM2Pi     2    Values : [-1537, 2452, 915, 240.53, 0, 0, 0, 0, 0, 0, 0]
    2020-05-11 12:14:19,683 DEBUG    RFM2Pi     2 Sent to channel(start)' : ToEmonCMS
    2020-05-11 12:14:19,684 DEBUG    RFM2Pi     2 Sent to channel(end)' : ToEmonCMS
    2020-05-11 12:14:19,790 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content'? 0 0 0 32 27 96 0 96 33 0 130 25 32 4 32 0 44 128 32 64 128 (-99)
    2020-05-11 12:14:19,837 DEBUG    MQTT       Publishing: emon/emonpi/power1 -1537
    2020-05-11 12:14:19,839 DEBUG    MQTT       Publishing: emon/emonpi/power2 2452
    2020-05-11 12:14:19,841 DEBUG    MQTT       Publishing: emon/emonpi/power1pluspower2 915
    2020-05-11 12:14:19,842 DEBUG    MQTT       Publishing: emon/emonpi/vrms 240.53
    2020-05-11 12:14:19,844 DEBUG    MQTT       Publishing: emon/emonpi/t1 0
    2020-05-11 12:14:19,846 DEBUG    MQTT       Publishing: emon/emonpi/t2 0
    2020-05-11 12:14:19,848 DEBUG    MQTT       Publishing: emon/emonpi/t3 0
    2020-05-11 12:14:19,849 DEBUG    MQTT       Publishing: emon/emonpi/t4 0
    2020-05-11 12:14:19,851 DEBUG    MQTT       Publishing: emon/emonpi/t5 0
    2020-05-11 12:14:19,853 DEBUG    MQTT       Publishing: emon/emonpi/t6 0
    2020-05-11 12:14:19,854 DEBUG    MQTT       Publishing: emon/emonpi/pulsecount 0
    2020-05-11 12:14:19,856 INFO     MQTT       Publishing: emonhub/rx/5/values -1537,2452,915,240.53,0,0,0,0,0,0,0
    2020-05-11 12:14:19,894 DEBUG    RFM2Pi     device settings updated: E i5 g210 @ 433 MHz q0 USA 0
    2020-05-11 12:14:20,045 DEBUG    RFM2Pi     3 NEW FRAME : OK 16 233 55 85 85 80 52 123 85 93 85 85 87 168 170 245 90 85 85 54 86 85 85 60 125 85 85 180 117 85 85 101 32 101 32 101 32 84 85 85 85 (-56)
    2020-05-11 12:14:20,049 DEBUG    RFM2Pi     3 Timestamp : 1589195660.05
    2020-05-11 12:14:20,051 DEBUG    RFM2Pi     3 From Node : 16
    2020-05-11 12:14:20,052 DEBUG    RFM2Pi     3    Values : [25276, 248.37, 46, 8, 512, -3, 4000, 867, 10345, 8417, 300, 300, 300, 1]

And

    pi@emonpi : ~ $ cd /opt/openenergymonitor/emonhub
    pi@emonpi : /opt/openenergymonitor/emonhub $ git status

    On branch powerwall

    Your branch is up to date with 'origin/powerwall'.

    nothing to commit, working tree clean
7

Strange, I wonder if the emonhub.conf changes didnt save for some reason? are the settings still here if you open the editor again?

8

Yes, still there.

9

I put the [[PowerWall]] section at the bottom. Is that right?

10

that might be it, It needs to be in the interfacer section, just above the ### nodes ### comment, similar to my screenshot above

11

Bingo. Sorry about that. But now we have the predicted certificate issue…

2020-05-11 13:20:52,209 WARNING PowerWall Exception caught in PowerWall thread. Traceback (most recent call last):
File “/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py”, line 40, in wrapper
return f(*args)
File “/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py”, line 102, in run
rxc = self.read()
File “/opt/openenergymonitor/emonhub/src/interfacers/EmonHubTeslaPowerWallInterfacer.py”, line 43, in read
response = urllib2.urlopen(self._settings[‘url’])
File “/usr/lib/python2.7/urllib2.py”, line 154, in urlopen
return opener.open(url, data, timeout)
File “/usr/lib/python2.7/urllib2.py”, line 429, in open
response = self._open(req, data)
File “/usr/lib/python2.7/urllib2.py”, line 447, in _open
‘_open’, req)
File “/usr/lib/python2.7/urllib2.py”, line 407, in _call_chain
result = func(*args)
File “/usr/lib/python2.7/urllib2.py”, line 1241, in https_open
context=self._context)
File “/usr/lib/python2.7/urllib2.py”, line 1198, in do_open
raise URLError(err)
URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:727)>

I think python 3 is installed. It defaults to using 2 (2.7.16 same as you). I haven’t looked into using v3 though.

Is there any way to get PYTHONHTTPSVERIFY=0 into emonhub?
Otherwise I can look into a “proper” solution, although I don’t understand the whole certificate thing at all, other than that it’s a perpetual problem for many people! Not saying it’s not a good thing though…

12

Great, thanks @PeteF

I wonder if you could try with your original script the following:

import urllib2, json
# 1. Fetch data from API 1
response = urllib2.urlopen('http://URL_OF_YOUR_POWERWALL/api/system_status/soe')
jsonstr = response.read()
data = json.loads(jsonstr)
soc = data['percentage']
# 2. Post data onwards to API 2
response = urllib2.urlopen('http://URL_OF_YOUR_EMONPI/input/post?node=powerwall&data=soc:'+str(soc)+'&apikey=YOUR_API_KEY', context=ssl._create_unverified_context())

Note addition of context=ssl._create_unverified_context()

13

Could you also try:

import urllib2, json, ssl
# 1. Fetch data from API 1
response = urllib2.urlopen('http://URL_OF_YOUR_POWERWALL/api/system_status/soe')
jsonstr = response.read()
data = json.loads(jsonstr)
soc = data['percentage']
# 2. Post data onwards to API 2
ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
response = urllib2.urlopen('https://URL_OF_YOUR_EMONPI/input/post?node=powerwall&data=soc:'+str(soc)+'&apikey=YOUR_API_KEY', context=ctx)

You may need to specify https in the above request url as well?

It probably needs to specific powerwall certificate doesnt it?

14

I think this is a great idea @TrystanLea very elegant and sort of fits in with the original idea of emonhub.

However, please, please, please let us move this to Python3 ASAP and ensure any new code is Python3 compliant.

1 Like
15

@TrystanLea did you check for a pip package?

16

Thanks @borpin that looks really useful, would be great to be able to get the other parameters out of the powerwall as well

17

Note - Python3.

1 Like
18

You meant to add this to the Powerwall request, didn’t you?

19

OK both of those test worked :slight_smile:

For @borpin, I am now trying the script with Python3. First requirement was to add () in print statements :slight_smile:
Then urllib2 → urllib.
The saga will continue.

20

Great, can you also try in the 2nd example above:

ctx = ssl.SSLContext(ssl.PROTOCOL_TLS)

thanks for testing!!