Powerwall State of Charge Interfacer stopped working - Tesla updated their software

The Powerwall SOC interfacer stopped working for me overnight, which obviously makes the My Solar Battery app rather less than useful. Errors in emonhub log:

WARNING PowerWall PowerWall couldn't send to server: 403 Client Error: Forbidden for url: https://1xx.xxx.xxx.xxx/api/system_status/soe

Having checked various things, it seems Tesla have done another of their famous soft-shoe shuffles and changed the local TEG (gateway) interface in their latest software update(20.49.0) and authentication is now required.

This thread has more information, skip to post #42:
Tesla Owners API v3 enabled

I am going to try the required authentication manually, but itā€™s a bit beyond my pay grade to figure out how to fix it within the interfacerā€¦ @TrystanLea?

@PeteF
I too am in the UK and ā€˜manageā€™ a system at my sonā€™s place with 3 x PowerWallā€™s. We do not use the Interfacer but a script instead:

# PURPOSE: To input key PowerWall data into emoncms

# HOW TO ...

# The script will create an INPUT node to receive the data

# Copy this script file to  /home/pi  and make it executable with:  chmod +x /home/pi/JB.powerwall.py  # using the correct script name

# Run the script with: /usr/bin/python3 /home/pi/JB.powerwall.py  # using the correct script name

# Create FEEDS using Log to Feed and add UNITS (pencil drop-down) to each

# IMPORTANT NOTE ...
# Data from this script is input to emoncms via an http API.  

# IMPORTANT NOTE ...
# Reserve a LAN IP address in the ROUTER to fix it for the WIRED connection to PowerWall

import sys, requests, json, time
from time import sleep  

# Emoncms server configuration
emoncms_apikey = "18f63xxxxxxxxxxxxaabe048da"   # For Node 15 - use appropriate key
emoncms_server = "http://127.0.0.1"
node = "Power_Wall"  # Name of the node to be created to receive the INPUT data from PowerWall
gateway = "http://192.168.1.119"  # IP address of the Tesla Gateway on local network - IMPORTANT: Reserve a LAN IP address in the ROUTER to fix it
node = node
# Give a name to each of the data inputs associated with the newly created node ...
di1 = "Charge_percent"
di2 = "Power_Now"
di3 = "Total_Export_Energy"
di4 = "Total_Import_Energy"

# Repetitive Looping starts here ...

starttime = time.time()
while True: 
	
	# First enquiry
	reply = requests.get(gateway+"/api/system_status/soe", timeout=10, verify=False)
	jsonstr = reply.text.rstrip()
	data_p1 = json.loads(jsonstr)
	
	# Second enquiry
	reply = requests.get(gateway+"/api/meters/aggregates", timeout=10, verify=False)
	jsonstr = reply.text.rstrip()
	data_p2 = json.loads(jsonstr)

	# Extract relevant data from the json's

	data1 = data_p1['percentage']                 
	data2 = data_p2['battery']['instant_power']
	data3 = data_p2['battery']['energy_exported']
	data4 = data_p2['battery']['energy_imported']

	# Send data to emoncms

	dev_data = {di1: data1, di2: data2, di3: data3, di4: data4} 

	# print (dev_data)   # a progress marker - can be commented out

	data = {
	  'node': node,
	  'data': json.dumps (dev_data),
	  'apikey': emoncms_apikey
	}

	response = requests.post(emoncms_server+"/input/post", data=data)
	
	

# End of Loop
	time.sleep(10.0 - ((time.time() - starttime) % 10.0))      # Repeat every 10 secs


# FINALLY ONCE THE SCRIPT RUNS OK: Create the powerwall.service and enable it so the script runs on boot up as follows:
# Do: CTRL-C to stop the script then - Do: sudo nano /etc/systemd/system/powerwall.service  and copy & paste in the following (using the correct script name) ...

"""

[Unit]
Description=PowerWall Status
After=network.target
After=mosquitto.service
StartLimitIntervalSec=0

[Service]
Type=simple
Restart=always
RestartSec=1
User=pi
ExecStart=/usr/bin/python3 /home/pi/JB.powerwall.py

[Install]
WantedBy=multi-user.target

"""
# Then save & exit and to ensure the powerwall.service runs on boot up - Do ...  
#    sudo systemctl start powerwall.service    and then    sudo systemctl enable powerwall.service

# AS A VERY LAST CHECK - Do: sudo reboot then SSH in again and check the service is active with:  systemctl status powerwall.service

# Finally close the SSH terminal. The script/service will continue to run surviving any future reboots

# Unkowns: what happens to the script/service when emoncms is updated as it will not be included in any Export ... so - caveat emptor

The script is still functioning. It makes two API calls:
/api/system_status/soe and /api/meters/aggregates
The firmware version is 20.40.3 so it looks like we will be similarly afflicted in the near future.

In addition I have two scripts to control PowerWall charging behaviour ā€“ so that it charges at overnight cheap rates. Hereā€™s one:

"""
PURPOSE ...
To set the PowerWall Reserve charge to 100 percent

"""

import requests, json
from requests.structures import CaseInsensitiveDict

# Enter energy_site identifier ...
en_site = "15xxxxxxxxxxxxx1858"

url = "https://owner-api.teslamotors.com/api/1/energy_sites/"+en_site+"/backup"

# Get latest Access Token from file
f = open("/home/pi/ev-charging-scripts/access_token.txt", "r")
acc_tok = f.read()

headers = CaseInsensitiveDict()
headers["Accept"] = "application/json"
headers["Authorization"] = "Bearer "+acc_tok
headers["Content-Type"] = "application/json"

data = {"backup_reserve_percent": 100}

resp = requests.post(url, headers=headers, data=json.dumps(data))

print(resp.status_code)	

But these scripts stopped working a couple of days ago because my Authority Token script stopped running a couple of days ago returning 400. Clearly this is where the authorisation body is buried:

"""
PURPOSE ...
Get a new Access Token and save it to a file
Access Tokens expire after 45 days
So run this script monthly via crontab -e

Other scripts can read the current Access Token (acc_tok) from the file by including the following lines ...

f = open("/home/pi/ev-charging-scripts/access_token.txt", "r")
acc_tok = f.read()

"""
import requests, json, datetime
from requests.structures import CaseInsensitiveDict

url = "https://owner-api.teslamotors.com/oauth/token"

headers = CaseInsensitiveDict()
headers["Content-Type"] = "application/json"

data = '{"grant_type": "password", "client_id": "81527cff06843c8634fdc09e8ac0abefb46ac849f38fe1e431c2ef2106796384", "client_secret": "c7257eb71a564034f9419ee651c7d0e5f7aa6bfbd18bafb5c5c033b093bb2fa3", "email": "xxxxxxxxxxxxxxxxxx.com", "password": "xxxxxxxxxxxxx"}'

response = requests.post(url, headers=headers, data=data)

if response.status_code in [200]:
    token_dict = json.loads(response.text)
    for x, y in token_dict.items():
        print(x, y)
else:
    print(response.status_code)

x = token_dict["created_at"] + token_dict["expires_in"]
date = datetime.datetime.fromtimestamp(x)
print("Valid UNTIL: " + date.strftime('%Y-%m-%d %H:%M:%S'))

# Over-write the access token in a file (access_token.txt) in the SAME directory as this script
f = open("/home/pi/ev-charging-scripts/access_token.txt", "w")
f.write(token_dict["access_token"])
f.close()

Thx for the link. This also looks informative:

Clearly written by an expert for expert readers ā€“ which I am not.

If you do come up with a fix (albeit temporary) then please share it.

Thx

It looks like the HomeAssistant component will be fixed on the next release More methods on Powerwall local API require authentication on firmware 20.49.0 Ā· Issue #45955 Ā· home-assistant/core Ā· GitHub

Later messages in this thread may help. Add support for Tesla Powerwall - Feature Requests - Home Assistant Community

My suspicion will be that this will be an ongoing issue (API changes) so finding a way of using the HA integration may be a better long term path. If they canā€™t fix it, probably no one can.

Thanks Brian @borpin. Sadly, I fear that you are correct in that this will be an ongoing issue.

Apologies if this has been covered elsewhere, butā€¦
Assuming that HA manage to fix it (in their mainstream release), and since I personally have grown extremely fond of the My Solar Battery app, what would be your recommended mechanism to get the HA data into an emoncms feed? Personally I didnā€™t get on with Node-red last time I tried it. Or perhaps even a choice depending on oneā€™s level technical expertise?

Alternatively, do you think the best approach might be to use HA as the ā€œmaster display deviceā€ (replacing My Solar Battery app etc) and bring the emoncms data into HA, which I know is possible?

Sorry, I donā€™t know enough about either to be able to advise on that front.

If HA can extract the same data, then it is possible to then send it to emoncms as an alternative input source.

Another My Solar Battery app user lamenting the loss of Tesla SOC.

For info - they seem to have it working for HA though there is a query over SOC I think.

@borpin @TrystanLea

I found this script ā€¦

ā€¦ on a Tesla Users Forum ā€¦

The script uses argparse which totally bemuses me and I think it is meanā€™t to be run from the command line. Notwithstanding, Iā€™ve cobbled together script bits enough to get a new style Access Token and have saved that to a .txt file in /home/pi. The token is valid for 45 days.

Over the next few days Iā€™ll tidy up the cobble and post the result here. Itā€™s a simple concept - a script that runs every month to update the Access Token saved to a .txt file. emon modules and userā€™s bespoke scripts then just need to access that .txt file.

Hope this may help.

1 Like

If you could, that would be hugely appreciated; I enjoy my Powerwall feed within HA, but TBH much of what has been said so far in this thread leaves me befuddled :face_with_head_bandage:

TIA

Clearly there are several ways to skin this cat i.e. to get Powerwall state of charge into emoncms, given that the interfacer no longer works, and my guess is that it probably wonā€™t ever again given that the development team here has much better things to doā€¦

I have today managed to achieve it using a new installation of the latest HA, together with the following integrations:

  • Tesla
  • Tesla Powerwall
  • Emoncms History

It means the My Solar Battery app works again :slight_smile:

1 Like

@TrystanLea

Iā€™ve had a script running on an RPi (also running the emon image) which grabs SOC and power data from a PowerWall battery on the same local network. The data is then published to emoncms - basically doing the same job as the emon Tesla Interfacer. Hereā€™s a screen shot ā€¦

My script stopped working a few days ago. Tesla updated their software to ver 20.49.0 which added another layer of security. Cookie information is now passed to the RPi when it first connects with PowerWall. Subsequently the RPi must send this cookie information with every enquiry it makes. If the connection remains unbroken (is not re-established) then the cookie information remains unchanged. It certainly does not change in a period of 24hrs.

With time spent and great assistance from Google, Iā€™ve modified my script and it is now up & running again. The script is intended to be run as a service. It is well documented/commented.

# PURPOSE: To input key PowerWall data into emoncms

# BACKGROUND - In Jan/Feb 2021, Tesla added a layer of security (with ver 20.49.0) - many 'things' stopped working incl the previous version of this script

# HOW TO ...

# This script is intended to be run on a Raspberry Pi on the local network with PowerWall

# The script will create an INPUT node to receive the data

# Copy this script file to  /home/pi  and make it executable with:  chmod +x /home/pi/JB.powerwall.py  # but using the chosen script name

# Run the script with: /usr/bin/python3 /home/pi/JB.powerwall.py  # but using the chosen script name

# In emon create FEEDS using Log to Feed and add UNITS (pencil drop-down) to each

# IMPORTANT NOTE ...
# Data from this script is input to emoncms via an http API.  
# The EmonHubEmoncmsHTTPinterfacer is not used/required

# IMPORTANT NOTE ...
# Reserve a LAN IP address in the ROUTER to fix it for the connection to PowerWall

# VERY IMPORTANT NOTE ...
# In order to provide continuous functionality, the script must be run as a service - see HOW TO below 

# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

import sys, requests, json, time, urllib3, os
from time import sleep  
from requests.structures import CaseInsensitiveDict     

# User Data
t_email = "xxxxxxx@xxxxxxx"   # email address for Tesla account
t_pwd = "XXXXX"  # The last 5 digits/characters of the password on the sticker behind the cover of the Gateway 2 unit

# Script Run data
loop_freq = "30.0"         # The frequency in secs that the script will grab PowerWall data for INPUT to emon
restart_hrs = "48.0"       # Belt & braces assuming Tesla will periodically not recognise the current cookie info - so refresh it every restart_hrs
# Belt & braces because, if at any time, Tesla does not recognise current cookie info (ie: the script loses connection), the script will restart via the powerwall.service 

# Emoncms server configuration
emoncms_apikey = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"   # For RPi concerned - use appropriate key
emoncms_server = "http://127.0.0.1"
node = "Power_Wall"  # Name of the NODE to be created to receive the INPUT data from PowerWall
gateway = "https://192.168.1.119"  # IP address of the Tesla Gateway on local network - IMPORTANT: Reserve a LAN IP address in the ROUTER to FIX it

# Name for each of the data INPUTS to emon associated with the newly created emon NODE
di1 = "Charge_percent"
di2 = "Power_Now"
di3 = "Total_Export_Energy"
di4 = "Total_Import_Energy"

# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

# House-keeping ...
loop_freq = float(loop_freq)
restart_secs = float(restart_hrs)*3600 

# Initial Basic login to get the cookie info ...

requests.packages.urllib3.disable_warnings()    # To disable SSL certificate warnings 

url = gateway+"/api/login/Basic"    

headers = CaseInsensitiveDict()
headers["Accept"] = "application/json"
headers["Content-Type"] = "application/json"

data = {"username": "customer", "email": t_email, "password": t_pwd}     # ref User Data above ...

a_session = requests.Session()      # ref:  https://www.kite.com/python/answers/how-to-get-session-cookies-from-a-website-in-python
a_session.post(url, headers=headers, data=json.dumps(data), verify=False)

session_cookies = a_session.cookies
cookies_dictionary = session_cookies.get_dict()
# print("Cookies Dictionary is:")
# print(cookies_dictionary)

# print (" ")
# print ("AuthCookie is:")
# print (cookies_dictionary['AuthCookie'])
ac = cookies_dictionary['AuthCookie']

# print (" ")
# print ("UserRecord is:")
# print (cookies_dictionary['UserRecord'])
ur = cookies_dictionary['UserRecord']
# print ("AuthCookie="+ac+";UserRecord="+ur)


# Repetitive Looping starts here ...

# Intro ...
headers = CaseInsensitiveDict()
headers["Cookie"] = "AuthCookie="+ac+";UserRecord="+ur            # A KEY LINE - so Tesla recognises 'legitimacy'
headers["Accept"] = "application/json"
headers["Content-Type"] = "application/json"

# Loop ...
starttime = time.time()
while True:
	
	# First enquiry
	reply = requests.get(gateway+"/api/system_status/soe", headers=headers, verify=False) # XXXXXXX soe not sof
	print (reply.status_code)
	if reply.status_code!=200:
		os.system("sudo systemctl restart powerwall.service")   # To cover the situation where Tesla no longer recognises the cookie info - so it will be refreshed
		time.sleep(10)                  # A pre-caution
	jsonstr = reply.text.rstrip()
	data_p1 = json.loads(jsonstr)
	
	
	# Second enquiry
	reply = requests.get(gateway+"/api/meters/aggregates", headers=headers, verify=False)
	jsonstr = reply.text.rstrip()
	data_p2 = json.loads(jsonstr)
	# print (reply.status_code)
	
	# Extract relevant data from the json's

	data1 = data_p1['percentage']                 
	data2 = data_p2['battery']['instant_power']
	data3 = data_p2['battery']['energy_exported']
	data4 = data_p2['battery']['energy_imported']

	# Send data to emoncms

	dev_data = {di1: data1, di2: data2, di3: data3, di4: data4} 

	# print (dev_data)   # a progress marker - can be commented out

	data = {
	  'node': node,
	  'data': json.dumps (dev_data),
	  'apikey': emoncms_apikey
	}

	response = requests.post(emoncms_server+"/input/post", data=data)
	

	elapsed = time.time() - starttime
	# print (elapsed)
	if elapsed > restart_secs:
		os.system("sudo systemctl restart powerwall.service")   # Belt & braces assuming Tesla will eventually not recognise the current cookie info - so refresh it
		time.sleep(10)                  # A pre-caution
		
# End of Loop
	time.sleep(loop_freq - ((time.time() - starttime) % loop_freq))      # Repeat loop every loop_freq secs
	

# FINALLY ONCE THE SCRIPT RUNS OK: Create the powerwall.service and enable it so the script runs on boot up as follows:
# Do: CTRL-C to stop the script running in the terminal then - Do: sudo nano /etc/systemd/system/powerwall.service  and copy & paste in the following (using the correct script name) ...

"""

[Unit]
Description=PowerWall Status
After=network.target
After=mosquitto.service
StartLimitIntervalSec=0

[Service]
Type=simple
Restart=always
RestartSec=1
User=pi
ExecStart=/usr/bin/python3 /home/pi/JB.powerwall.py

[Install]
WantedBy=multi-user.target

"""
# Then save & exit and to ensure the powerwall.service runs on boot up - Do ...  
#    sudo systemctl start powerwall.service    and then    sudo systemctl enable powerwall.service

# AS A VERY LAST CHECK - Do: sudo reboot then SSH in again and check the service is active with:  sudo systemctl status powerwall.service

# Finally close the SSH terminal. The script/service will continue to run surviving any future reboots

# Unkowns: what happens to the script/service when emoncms is updated as it will not be included in any Export ... so - caveat emptor

This is a solution for accessing: PowerWall/api/system_status/soe and PowerWall/api/meters/aggregates over the local network.

Separately an Access Token has been introduced by Tesla. This is used when emulating the Tesla mobile App to change charge/discharge behaviour. But that is a separate project.

Hope this helps.

1 Like

johnbanks,
Thank you for posting this fix.
I have the script running, but not yet as a service, the feeds are there and showing data, now to get the My Solar Battery app to work. As the app has run flawlessly since I installed it I am having to re-learn the setting up of feeds for the new node.

Again, thank you for the script.

This is probably showing as a new Input. When processing the input point it to the Feed you used to use and the App should just work.

Well, I have johnbanksā€™ script running as a service, battery_soc setup as PHPTIMESERIES percentage and my dashboard shows Tesla SOC on the allocated dial.
The My Solar Battery app however displays ā€œERROR: timeseries.value datastore[battery_soc].data[1470] is undefined, data length: 1470ā€ on startup and afterwards displays a frozen view of the My Solar Battery display.

I cannot figure out what else needs defining.

Also, I installed the My Solar Battery app a second time as My Solar Battery App 2 (after I had re-defined battery_soc as a %) to see if that fixed the error - it didnā€™t. How do I delete the second instance of an app ?

Got My Solar Battery app going again.
Changed the battery_soc from PHPTIMESERIES to PHPFINA.

Steps involved -

  1. Get johnbanks script going according to his instructions. (I altered the loop_freq to 10 and added another debug command in the ā€œInitial basic login ā€¦ā€ section - # print url - as I was getting errors due to my router needing a reboot.)
  2. Edit Emonhub.conf to delete the no longer working Tesla interfacer.
  3. Deleted the old battery_soc feed then in Inputs in the Power_Wall node under Charge_percent added a second Log to feed of battery_soc using Engine PHPFINA.
  4. In feeds, edited the Power_Wall feed battery_soc to Feed Unit Percent (%)

Again thanks to johnbanks for the script to get TrystanLeaā€™s excellent app going again.

Hi all,

Thanks for your script @johnbanks. Just getting around to updating my emonpi, Iā€™m now on python3 and so can now try and fix the broken Tesla link.

Iā€™ve followed the instructions provided - excellent documentation. However, I get a Failed to establish a new connection... No route to host error. Iā€™ve triple checked the gateway address, username and password - all correct.

Iā€™m also unable to access the gateway in a web browser - unless I connect to the gateway wifi. Iā€™d assumed this change was due to the new authentication protocol, but perhaps itā€™s linked?

Any suggestions very welcome!

@ miker

Yr report ā€“ No route to host ā€“ suggests that the Tesla Gateway IP address you are using is incorrect.

Do you have means to scan yr local network to list all the connected devices?

I use nmap installed on the RPi running the emon image.

This is what I see ā€¦

Nmap scan report for 192.168.1.119

Host is up (-0.099s latency).

MAC Address: CC:88:26:32:15:DE (Unknown)

The Tesla Gateway does not exactly advertise itself but the MAC Address is the give-away.

Look for a device with MAC Address starting CC:88:

Note the full MAC Address for future ref and the IP address that yr router has currently assigned ā€“ use that in the script.

I had a lot of problems with the Gateway IP address changing. My recommendation ā€“ assign a fixed IP address to the MAC address in yr router - I had to consult the router User Manual on how to.

Hope this helps

PS ā€“ For completeness and not to confuse ā€“ I actually see two devices on the local network with MAC Address CC:88: etc because the Tesla Gateway is connected to the local network both by LAN cable and by WiFi.

Thanks John - you pointed me in the right direction. Everything was in place already. A case of turning it off and on again. I had to disconnect the gateway from the router and reconnect, then hitting it with an nmap -Pn scan forced it to wake up.

via GIPHY

@ miker

I checked my system this morning and by the strangest of coincidences noticed that PowerWall had stopped communicating with emon at 2:15am.

I also noticed that Tesla seemed to have done a sotware update to ver 21.13.2 (as shown at the bottom of the mobile App page). The Tesla Motors Club forum has some ā€˜rumblingsā€™ about this:

I restarted my powerwall.service with:

sudo systemctl restart powerwall.service

ā€¦ and it started working again ā€¦ so two happy bunnies!

I donā€™t know whether you have set up a service. You too may suffer ver 21.31.2 problems shortly.

v20.49.0 for me. Looks like I have that joy to come!