Following @TrystanLea and @Thelmike in ClimaCell Weather Forecast API
I decided to try and integrate the solcast solar forecasting into emonCMS.
I’ve written a simple python script that firstly uses the request module to pull down the 30 min forecasts for my site. These are 7 day forecasts (i.e. 336 half-hr forecasts), but I only want the next 12 hrs so I only keep the first 48 arrays (they consist of the 10% and 90% probability bound, and a median forecast value).
Each array consists of a timestamp (in UTC zone), the median, 10 and 90% values.
I use pandas to hold the data and so it’s easy to remove unwanted rows and to do the conversion to local time, and add a column for unix time.
A loop through each row then constructs a string which becomes the url to post the data to local emonCMS. Basically I want to graph the 30 min forecasts for the next 12 hrs, and overlay the actual PV output measured from the Fronius inverter.
I struggled a bit to get the post format to work, but settled on the
http://192.168.xx.xxx/input/post?time=1590715800&node=138&csv=4.6412,3.3056,4.6412&apikey=myAPIkey
format which allowed me to import the correct time and values.
While still in the loop, the request then sends the url string to local emonCMS. I wait a few seconds between each post as my rPi is already a bit overloaded.
Once inside the feed, I can graph the data:
The forecasts are quite good in the short term, but my system consists of 2 strings, one facing east and the other facing north. In the free version of Solcast I can only specify one string, so I use the east facing one. Consequently as the day progresses the forecasts are less reliable.
I use cron to run the script every hour between 06:00 and 14:00, so only use 16 requests per day.
Note also that the free version allows up to 20 requests a day, so my script is under that limit.
I’m sure there is an easier or more refined way, but so far it works…
The script has a few now unnecessary lines (used in debugging etc) but it is still a work in progress!
# setup unchanging parts of url to build string:
url1 = 'http://192.168.xx.xxx/input/post?time='
url2 = '&apikey=myAPIkey'
import requests
from time import sleep
import json
import dateutil.parser
from datetime import datetime
import pytz
import csv
import pandas as pd
solcastData = []
# uncomment next line to save data to file
# fname = 'solcastD3.csv'
# blockCount = 0 # not currently used
# the solcast urls - use your API keys etc
# url = "https://api.solcast.com.au/rooftop_sites/mysite/forecasts?format=json&api_key=myAPIkey"
r = requests.get("https://api.solcast.com.au/rooftop_sites/mysite/forecasts?format=json&api_key=myAPIkey")
#print(r)
data = r.text
datajson = r.json() # get the data in json
# df = pd.DataFrame(datajson) # put it into a dataframe
df1 = pd.DataFrame.from_dict(datajson)
# this will split the string into 5 cols
df2 = df1['forecasts'].apply(pd.Series)
#drop last column which never changes
df2.drop(df2.columns[4],axis=1,inplace=True)
# make sure the timestamp is in datetime format
df2['period_end']= pd.to_datetime(df2.period_end,infer_datetime_format=True)
# now convert to local time and add epoch time
df2['local_time']= df2['period_end'].dt.tz_localize("UTC").dt.tz_convert("Australia/Sydney").astype('int64')//1e9
# truncate to 24hrs - if 7 day forecasts not needed
df3 = df2[0:48]
# write to file - useful to check
df3.to_csv('solcastcs324v.csv')
# check the dataframe has the right time etc
print(df3)
# create url string - first loop through and pull out values
# then complete url string and send to server
for index, row in df3.iterrows():
#print(row['pv_estimate'], row['local_time'])
url_string = url1+str('{0:.0f}'.format(row['local_time']))+'&'+'node=138'+'&'+'csv='+str(row['pv_estimate'])+','+str(row['pv_estimate10'])+','+str(row['pv_estimate90'])+url2
print(url_string)
r=requests.get(url_string)
print(r.status_code)
print('indexrow =', index)
sleep(5)