Community
OpenEnergyMonitor

Community

Integrating solar forecasting

Tags: #<Tag:0x00007f6e088d2818>

Following @TrystanLea and @Thelmike in ClimaCell Weather Forecast API
I decided to try and integrate the solcast solar forecasting into emonCMS.
I’ve written a simple python script that firstly uses the request module to pull down the 30 min forecasts for my site. These are 7 day forecasts (i.e. 336 half-hr forecasts), but I only want the next 12 hrs so I only keep the first 48 arrays (they consist of the 10% and 90% probability bound, and a median forecast value).

Each array consists of a timestamp (in UTC zone), the median, 10 and 90% values.
I use pandas to hold the data and so it’s easy to remove unwanted rows and to do the conversion to local time, and add a column for unix time.
A loop through each row then constructs a string which becomes the url to post the data to local emonCMS. Basically I want to graph the 30 min forecasts for the next 12 hrs, and overlay the actual PV output measured from the Fronius inverter.
I struggled a bit to get the post format to work, but settled on the
http://192.168.xx.xxx/input/post?time=1590715800&node=138&csv=4.6412,3.3056,4.6412&apikey=myAPIkey
format which allowed me to import the correct time and values.
While still in the loop, the request then sends the url string to local emonCMS. I wait a few seconds between each post as my rPi is already a bit overloaded.
Once inside the feed, I can graph the data:

The forecasts are quite good in the short term, but my system consists of 2 strings, one facing east and the other facing north. In the free version of Solcast I can only specify one string, so I use the east facing one. Consequently as the day progresses the forecasts are less reliable.
I use cron to run the script every hour between 06:00 and 14:00, so only use 16 requests per day.
Note also that the free version allows up to 20 requests a day, so my script is under that limit.
I’m sure there is an easier or more refined way, but so far it works…
The script has a few now unnecessary lines (used in debugging etc) but it is still a work in progress!

# setup unchanging parts of url to build string:
url1 = 'http://192.168.xx.xxx/input/post?time='
url2 = '&apikey=myAPIkey'
import requests
from time import sleep
import json
import dateutil.parser
from datetime import datetime
import pytz
import csv
import pandas as pd
solcastData = []
# uncomment next line to save data to file
# fname = 'solcastD3.csv'
# blockCount = 0 # not currently used
# the solcast urls - use your API keys etc
# url = "https://api.solcast.com.au/rooftop_sites/mysite/forecasts?format=json&api_key=myAPIkey"
r = requests.get("https://api.solcast.com.au/rooftop_sites/mysite/forecasts?format=json&api_key=myAPIkey")
#print(r) 
data = r.text
datajson = r.json() # get the data in json
# df = pd.DataFrame(datajson) # put it into a dataframe
df1 = pd.DataFrame.from_dict(datajson)
# this will split the string into 5 cols
df2 = df1['forecasts'].apply(pd.Series)
#drop last column which never changes
df2.drop(df2.columns[4],axis=1,inplace=True)
# make sure the timestamp is in datetime format
df2['period_end']= pd.to_datetime(df2.period_end,infer_datetime_format=True)
# now convert to local time and add epoch time
df2['local_time']= df2['period_end'].dt.tz_localize("UTC").dt.tz_convert("Australia/Sydney").astype('int64')//1e9
# truncate to 24hrs - if 7 day forecasts not needed
df3 = df2[0:48]
# write to file - useful to check
df3.to_csv('solcastcs324v.csv')
# check the dataframe has the right time etc
print(df3)
# create url string - first loop through and pull out values
# then complete url string and send to server
for index, row in df3.iterrows():
    #print(row['pv_estimate'], row['local_time'])
    url_string = url1+str('{0:.0f}'.format(row['local_time']))+'&'+'node=138'+'&'+'csv='+str(row['pv_estimate'])+','+str(row['pv_estimate10'])+','+str(row['pv_estimate90'])+url2
    print(url_string)
    r=requests.get(url_string)
    print(r.status_code)
    print('indexrow =', index)
    sleep(5)
1 Like

I’m doing something similar in order to work out when to charge the battery system (and soon the EV).

I’ve got East, West & South systems so I put it all combined in as South and tweaked the efficiency factor.

I’m uploading my actual PV data every five minutes to Solcast and the accuracy correlation is now 0.97 after several weeks of their PV Tuning system assimilating the data i.e. the forecast is very close now to what output I then get.

2 Likes