Script for predictive solar output

I’ve been wondering how best to tackle forecasting solar radiation for quite a while, so coming across this thread has reinvigorated my effort at managing my battery storage. Thanks guys.

for someone who might be curious… I have being comparing to solcast… for me solcast is not very accurate. it generally always over and by alot even accounting for the differing panel angle I have … here an output for last few days … a couple weather APIs are generally within couple 100 watts and track my actual solar production perfectly if it is stable cloud or clear day . Solcast always tracks weird and loop sided and off considerably.
day1


day2

Day3

day4

how I trigger an event, is set an alarm in kapaciptor I combine sum of three values and set the range ( 3 time the normal if I used 4 sample it would be 4 times) then the handler is exec - which sends an mqtt that set my charging protocol for the day, I set different ranges and send different charge protocols via mqtt depending on the daily predicted


I use 3 values as aeris is always pretty accurate but sometimes it not so I like to use 3 of my most accurate APIs so i get an average that generally pretty close
simple easy reasonable accurate for my set up

This look brilliant thanks Stephen - i will implement this now and then try to create a docker image - are you OK with that - if i get it going do you want to host it on your github ?

Also BTW - Solcast are discontinuuing their tuning feedback posting of data and instead are providing the ability to define two rooftop arrays

Craig

feel free to use as you like. and sure I will post it to github for you

Stephen,

JUst working through the install now.

The plan is to initially create a virtual machine and validate anything that causes issues during the installation process, once done i will then go back and add each of those elements to a Docker image.

I intend to install this on Ubuntu 18.04 LTS as this will probably give me the most compatiblity with th various libraries etc

I will feedback into this thread initially any updates for getting it running in this environment

Craig

OK so the first question - looking at the Pysolar link in your first post - they note issues with Python 3 - have you based all of your stuff on Python 2 or 3 ?

Craig

I use python 3 - I believe the incompatibility is a date time issue for a leap second or something like that. so when it close to sunrise or sunset near the spring and fall solstice I get an infinity error . but my script just ignores those errors so it not a big deal

Stephen,

I have been playing around trying to get this to work (and to streamline an install routine) and would like clarification if possible on the following

  1. What user should this be run as ? I have been running as a normal system user (craig) and installing everything using su - however when i then run START from within my home directory $HOME/Prediction - there are a heap of permission errors thrown

  2. Regardless of which user i run it as i am getting issues with jq - it does not appear to be perfoming any inline processing of the downloaded data - and just throws up the jq help screen when invoked in the script

  3. I have started off with a basic config just enabling AERIS to start with - just confirming if i have to enter the lat/longtitude and the API details in both the START file and also the solar-aeris files as there are entries in both files ?

Craig

  1. that’s weird no I am just running in my home user in my case stephen. but also other users as well on other computers – what is the permission errors are you getting?

  2. that is also weird – perhaps it’s the version of jq problem . i have it working on several machines with out issue. but they all happen to be kubuntu 20.04 .

  3. no you just use the api in START file. BUT if you want, you can run as stand alone file instead of running ./START you just run ./solar-aeries if you put the key in there as well and change ENABLE=1 (if you want it download a new json from their server) . I made it that way so you can debug a script easier

but for full disclosure - both wbit and areis API do not currently work for me as they both have invalidated the key for forecast data after months of working fine…
so even though the API key is invalidate… all it throws up is this as an error

if I use in debug mode api key in aeris-script and enabled= 1

using in script
using in script
using in script
awk: cannot open /dev/shm/Aeris_EOD.csv (No such file or directory)
Runtime error (func=(main), adr=9): Divide by zero
(standard_in) 1: syntax error
(standard_in) 1: syntax error
Runtime error (func=(main), adr=9): Divide by zero

and if I check /dev/shm/JSON-A

{
  "success": false,
  "error": {
    "code": "insufficient_scope",
    "description": "The client provided is not authorized to access the request."
  },
  "response": []
}

so as a suggestion try a another one of the other APIs incase they changed the json formats for aeris

Stephen,

Thanks for the quick response.

First thing I will do is build a new VM using Ubuntu 20.04 to make sure this is not the issue and will then revert back

Craig

OK Stephen,

New VM build running Ubuntu 20.04 - all updates run

Much cleaner run through this time - so some of the stuff must have had older versions etc for 18.04 Ubuntu

After the install and setup of pysolar, lynx, bc i then cloned the repository into a directory /home/craig/Prediction

Installed Mosquitto and Mosquitto-clients

Edited the panelconf and added in 3 arrays
Edited START and disabled wind gathering
Added API for OpenWeathermap

Then chmod +x START and ran it

Got an error in line 184 Solar-Open - permission denied.

chmod +x Solar-Open

Then got a permission error on panelconf

chmod +x panelconf

And Boom it all works now. !!!

Now the next thing is to understand

  1. The format of the panelconf definitions
  2. What it is actually spitting out for me when it runs

Thanks for the help on this

I will post a PR on Github with updates for the doco for others new to this as i go

Craig

as to panel config… I have 3 orientations to my panels. as I wanted a long production arc. currently it is +16hrs and I believe peaks at 18 hrs on my longest day…

Example

G1A=-4        #closest to angle / 7.5degrees   -negative for east  positive for west facing -e 35degrees /7.5 = 4.666  

G1w=950 # panel wattage rated 


G2A=1          #  panel facing due south or north depend what side of the equator 
G2w=950


G3A=5
G3w=950

##### HARDWARE and other configurations 

albedo=7    #albedo effect % of panel output 
inverter=.90  #inverter  average efficency

the panel config is not too hard
G1A ( Group 1 Angle )
due south would be 0 , so in my case I am using -4,1 and 5
so the angles of my panel is the east panel is -30 degree offset from due south. and other set is 1 so ~+7degree and the west solar panel is setting of 5 ~+37 offset

G1W ( group 1 watts ) the rated watts of the panel group

albedo is just the % of panel output they produce when light does not hit it directly . if you live say in bright reflective conditions that number can be quite high

inverter — is just the inverter rated average efficiency

I had to use an array with degree cos because for the life of me i could not figure out how to do cos in degrees and only in rad in linux and that why it done in blocks of 7.5 degrees as I was lazy to produce an huge array

Aah good one and thanks for the explanation - it had me stumped for a while why you did the divide by 7.5 !!

I actually have 4 sets of panels (6 strings in total) with 4 different orientations.

I will use what you have and then see if i can work out how to add an additional string

thanks for all the help

Craig

to add another group is easy-
add variable near the top
G4w=
G4A=
Group4=0
then copy and paste group one
then rename all references to;
N1a → N4b
N1b -->N4b
G1A → G4A
G1w → G4w
Group1 → Group4
then in Total add this to the beginning of the equation
(($Group4+($G4w * $albedo/100)) +

Good one thanks Stephen,

WIll work this in

If i had multiple arrays in the same orientation but one gets badly shaded during the winter period could i just define it as a seperate array with a very low output during this time - or would there be a smarter way to handle it ?

Craig

for more accurate you know roughly the solar array starts to shade just build a 365 point array or smaller say 73 point roughly every 5days so if it shades 100% to 2 months just use date +%j … the solar minimum is ~dec 20 day ~356 so middle would be dec 19 355 (71) 1 month prior day 325 ( 65) and 1 month after would be day 20 (4) and then X time period that a percentage of shading
so add something like this to it panelconf

DAY=$( date  +%j ) 
DAY=$(echo " $DAY/5 "| bc)
if [  $DAY < 1 ]; then DAY=73;  fi
shading=( 0 0 0 0 .15 .30 .45 .60 .75 .9 1 1 1 1 1 1   ....... 1 1 1 1 .9 .75 .60 .45 .30 .15 0 0 0 0 0 0 0 0) ## 73 point array 
SHADING=$(echo ${shading[$DAY]}  )
 
((($Group4 * $SHADING )+($G4w * $albedo/100)) +

and if need be you can do the same for hourly shading during time of day probably alot more complex but do able

I was thinking about your shading question this morning . there is another way you could do it. using pysolar azimuth and altitude calculator … it be a alot more accurate and you could add in for multiple objects that shade your solar panels
what you would do there . you add in object information of it angle of location to your solar panels, the height of the object , width of the object and distance of the object from the solar panels (then you would also have to provide height top of the solar panels and bottom - and width so solar structure) .

the easiest method for this : ( with out writing a whole bunch complex math plotting equations )

here a rough example of what to do…

first make copy of radiation.py and rename altitude.py ( edit within it and comment out the last line and then uncomment #print(altitude_deg)
in your API scripts modify variable potential
ie; potential=$(./panelconf $numberA $LON $LAT $Time )

add this to panelconf
altitude=$(python3 -W ignore altitude.py -lon $2 -lat $3 -t $4)
Build 3 arrays—

  1. build 24 hr array ( 48 point – 1/2 hour interval) call it “upper” in this array you place in the time slot the angle at which the the sun start shade your panel
    2.build a second 24 hr array ( 48 point) call it “lower” in this array you place in the time slot the angle at which the the sun fully shade your panel
  2. build a third 24 hr array (48 point ) call it “percent” the percentage the of shading the object causes ie a single tree cast a shadow that shades 10% of your solar panels
  3. to calculate your the sun altitude using pysolar
print(get_altitude(latitude, longitude, date))

now it just a easy math equation-
givens;
upper – 45
lower – 35
percent – 10

VAR=$(echo " $1 * 2 "| bc -l) #  convert 1/2 hour interval to whole number
upper=( 0 0 0 0 ......0 0  45 45 45 0 0.... 0 0 0 0 0 0 0 0) ## 48 point array 
lower=( 0 0 0 0 ......0 0  35 35 35 0 0.... 0 0 0 0 0 0 0 0) ## 48 point array 
percent=( 0 0 0 0 ......0 0  10 10 10 0 0.... 0 0 0 0 0 0 0 0) ## 48 point array 
UPPER=$(echo ${upper[$VAR]}  )
LOWER=$(echo ${lower[$VAR]}  )
Percent=$(echo ${percent[$VAR]}  )
Percent=$(echo "Percent / 100 "| bc -l)
x=$(echo " UPPER - LOWER "| bc -l)
y= $(echo "UPPER - alitude  "| bc -l)
SHADING=$(echo "((y/x)*Percent) "| bc -l)

if [  $SHADING > 1 ]; then SHADING=$Percent;  fi
if [  $SHADIN <= 0 ]; then SHADING=1;  fi

((($Group4 * $SHADING )+($G4w * $albedo/100)) + # or which ever group the shading applies too

and as added info to calculate the upper and lower degrees -
lower:
simply stand behind your panels so you can not see the object now walk backwards until you see the object just appear above the solar panel.
now measure the height the solar panels are from the ground ( ie 3 meters). from a plumb line dropped from the top of the panel measure back to where you were standing ie 2 meters and the height you eyes were from the ground 2m –
a= panel height (3) - eye height (2) =1
b= distance from plumb line = 2
use a right angle calculator
alpha angle= 26 degree to be fully shaded
now do the same for the upper - when object is viewed below the solar panel
measure the height from the ground to the bottom of the solar panels 1M
a= panel height (1) - eye height (.5m) =.5
b= distance from plumb line dropped from the bottom edge = .5m
alpha angle = 45 degrees

so from that example the panels will start to shade at <45 degrees and fully shaded by 26 degrees

if you can not figure it out I will rewrite my script at some point to include option for shading

Thanks so much Stephen !! Fantastic info

I will start off with baby steps first – just about to put it live and monitor it against actual production for the 6 sets of panels I have – will then start fiddling with this idea (once I digest it and understand what you are saying !!) and see if this gets me closer to my production curve

Craig

the easiest way to set it up is if you have a day that has ~clear sky. in both predictive and actual - you can test it and configure at the end of the day historically
simply run START in the early morning
then at the end of the day
run the solar script directly ie solar-open
just configure
ENABLE=0
to
ENABLE=1
this is running in debug mode and uses the same file over and over that you downloaded in the morning . and it will simply update up influxdb with the new inputs as you change settings and then you can compare to see if it matches your actual production a little fine tuning and away you go.
then once it tuned just change back to
ENABLE=0
and you are off and running