Has anyone on here any experience of Solcast - in the sense of POSTing PV data to their end?
I’ve seen a few mentions of integrating the predictions, but not seen any mention of feeding the actual data back to improve the accuracy.
I’ve got an idea of what would need to happen - but in terms of putting that into usable code I really have no idea how to start. So wanted to see if anyone else has tried?
I’ve not tried it yet - mainly because I have 2 strings of PV with one string facing East and the other facing North. Solcast only allows one string with their free access, so until I can formulate an equivalent hybrid model I can’t use their fine tuning approach.
I post to Solcast and have done so for the past 6 months or so. My installation is off grid so I only post the output when the batteries are not floating. I use Node Red to average the power output over each 5 minute interval (I think this is the smallest interval Solcast will accept) and only post if the array is not being limited. Even with this limited data set the Solcast site reports that there has been sufficient data to enable PV tuning. Their forecasts appear accurate enough to estimate the spare generation capacity available to know when to turn the air condioner on or schedule larger energy usage.
Would you be willing to share this, to use as a starting point?
I originally had a piece of code running elsewhere pulling data from the SolarEdge API, but it was horribly clunky and unreliable. But it gave enough tuning to see the forecasts getting more and more accurate.
Sure. No problems.
The node red flow that follows does both a download of a forecast which is then uploaded to EmonCMS and well as the posting of my actuals. The code for the posting of the actuals is complicated by making sure I only post when the output of the solar array is not constrained by the battery state. This is identified when the line frequency is above 51Hz and so I discard any measurement period where the frequency rises above 51Hz. You can edit is part out to get it working all the time.
[
{
"id": "e77f9f87.0c622",
"type": "tab",
"label": "Solcast Interface",
"disabled": false,
"info": ""
},
{
"id": "66dca2c9.3eb33c",
"type": "http request",
"z": "e77f9f87.0c622",
"name": "Solcast Forecast Data",
"method": "GET",
"ret": "txt",
"paytoqs": false,
"url": "https://api.solcast.com.au/rooftop_sites/4c59-af78-a1a0-ee25/forecasts?format=json",
"tls": "",
"persist": false,
"proxy": "",
"authType": "basic",
"x": 380,
"y": 190,
"wires": [
[
"4c9657e6.2bab08"
]
]
},
{
"id": "fd8e1b49.3c49b8",
"type": "debug",
"z": "e77f9f87.0c622",
"name": "View Full Output",
"active": false,
"tosidebar": true,
"console": false,
"tostatus": false,
"complete": "payload",
"targetType": "msg",
"x": 640,
"y": 230,
"wires": []
},
{
"id": "4c9657e6.2bab08",
"type": "json",
"z": "e77f9f87.0c622",
"name": "Convert from JSON to Object",
"property": "payload",
"action": "",
"pretty": false,
"x": 620,
"y": 190,
"wires": [
[
"fd8e1b49.3c49b8",
"91d934a.f8843c8"
]
]
},
{
"id": "91d934a.f8843c8",
"type": "function",
"z": "e77f9f87.0c622",
"name": "Reset Daily Forecast Data",
"func": "var daily_forecast_data = [];\nfor (i = 0; i <= 120; i++) { // 0 is the 0530 forecast slot. First forecast at 0500 is 0530.\n forecast_output = msg.payload.forecasts[i].pv_estimate\n daily_forecast_data.push(forecast_output);\n}\nflow.set(\"daily_forecast_data\",daily_forecast_data);\nreturn msg;",
"outputs": 1,
"noerr": 0,
"initialize": "",
"finalize": "",
"x": 970,
"y": 190,
"wires": [
[
"a3b4c0b3.ec825"
]
]
},
{
"id": "1c137609.ce1dba",
"type": "inject",
"z": "e77f9f87.0c622",
"name": "At 0500 each day",
"props": [
{
"p": "payload",
"v": "",
"vt": "date"
},
{
"p": "topic",
"v": "",
"vt": "string"
}
],
"repeat": "",
"crontab": "00 05 * * *",
"once": false,
"onceDelay": 0.1,
"topic": "",
"payload": "",
"payloadType": "date",
"x": 170,
"y": 190,
"wires": [
[
"66dca2c9.3eb33c"
]
]
},
{
"id": "ece67ca9.59b8f",
"type": "mqtt out",
"z": "e77f9f87.0c622",
"name": "",
"topic": "emon/Solcast/DailyForecast",
"qos": "2",
"retain": "true",
"broker": "36372260.fb268e",
"x": 1520,
"y": 240,
"wires": []
},
{
"id": "42495fd6.c9b48",
"type": "inject",
"z": "e77f9f87.0c622",
"name": "Test Poke",
"repeat": "",
"crontab": "",
"once": false,
"onceDelay": 0.1,
"topic": "",
"payload": "",
"payloadType": "date",
"x": 460,
"y": 310,
"wires": [
[
"a5bfbde5.eff4b"
]
]
},
{
"id": "a5bfbde5.eff4b",
"type": "function",
"z": "e77f9f87.0c622",
"name": "Get forecast_object",
"func": "msg.payload = flow.get(\"forecast_object\")\nreturn msg;",
"outputs": 1,
"noerr": 0,
"x": 650,
"y": 310,
"wires": [
[
"a3b4c0b3.ec825"
]
]
},
{
"id": "568c3c8b.1cbb94",
"type": "debug",
"z": "e77f9f87.0c622",
"name": "",
"active": false,
"tosidebar": true,
"console": false,
"tostatus": false,
"complete": "true",
"targetType": "full",
"x": 1493,
"y": 434,
"wires": []
},
{
"id": "a3b4c0b3.ec825",
"type": "function",
"z": "e77f9f87.0c622",
"name": "Format Forecast to Load into Feed, Update Daily Total Estimate",
"func": "latest_forecast_time = flow.get(\"latest_forecast_time\") || 0;\nthis_forecast_call_time = global.get(\"hour_of_day\");\ndaily_forecast_data = flow.get(\"daily_forecast_data\");\nrecord_id_2130_tomorrow = 80 - ((this_forecast_call_time - 5) * 2);\nvar msg2 = { payload: 0};\nvar msg3 = { method: \"POST\" };\nflow.set(\"forecast_object\",msg.payload); \nfor (i = 0; i < 96; i++) { // Load forecast data into emoncms for next 48 hrs. First forecast is for next 30min.\n forecast_output = msg.payload.forecasts[i].pv_estimate * 1000;\n forecast_time = (new Date(msg.payload.forecasts[i].period_end).getTime() / 1000 );\n if (forecast_time <= latest_forecast_time) {\n msg3.url = \"http://192.168.1.200/emoncms/feed/update.json?id=92&time=\" + forecast_time + \"&value=\" + forecast_output + \"&apikey=5af722e3ef5403b7d1deb3e8ff4450e1\";\n\n } else {\n msg3.url = \"http://192.168.1.200/emoncms/feed/insert.json?id=92&time=\" + forecast_time + \"&value=\" + forecast_output + \"&apikey=5af722e3ef5403b7d1deb3e8ff4450e1\";\n flow.set(\"latest_forecast_time\", forecast_time);\n }\n node.send([null, null, msg3]);\n if (i <= record_id_2130_tomorrow) {\n daily_forecast_data[i + ((this_forecast_call_time - 5) * 2)] = forecast_output/1000; // 0 is the 0530 slot. First forecast at 0500 is 0530.\n }\n}\n// Sum forecast values for daily forecast for current day\nforecast_total = 0;\nfor (i = 0; i <=32; i++) { // 0 is the 0530 slot. 32 is the 2130 slot. First forecast at 0500 is 0530.\n forecast_total += daily_forecast_data[i];\n}\nforecast_total = forecast_total / 2;\nmsg.payload = forecast_total;\nflow.set(\"24hr_forecast\", forecast_total);\n// Sum forecast values for daily forecast for next day\nforecast_total = 0;\nfor (i = 48; i <=80; i++) { // 48 is the 0530 slot. 80 is the 2130 slot. First forecast at 0500 is 0530.\n forecast_total += daily_forecast_data[i];\n}\nforecast_total = forecast_total / 2;\nmsg2.payload = forecast_total;\nflow.set(\"48hr_forecast\", forecast_total);\nflow.set(\"daily_forecast_data\", daily_forecast_data);\nreturn [msg, msg2, null];",
"outputs": 3,
"noerr": 0,
"initialize": "",
"finalize": "",
"x": 1090,
"y": 350,
"wires": [
[
"227a3047.4dbae",
"ece67ca9.59b8f"
],
[
"6dc37916.c105d8"
],
[
"30124093.5d5a9"
]
]
},
{
"id": "30124093.5d5a9",
"type": "delay",
"z": "e77f9f87.0c622",
"name": "Rate Limiter",
"pauseType": "rate",
"timeout": "5",
"timeoutUnits": "seconds",
"rate": "1",
"nbRateUnits": "1",
"rateUnits": "second",
"randomFirst": "1",
"randomLast": "5",
"randomUnits": "seconds",
"drop": false,
"x": 1473,
"y": 374,
"wires": [
[
"568c3c8b.1cbb94",
"91f7e39f.fc9d6"
]
]
},
{
"id": "91f7e39f.fc9d6",
"type": "http request",
"z": "e77f9f87.0c622",
"name": "http post to emoncms of forecast point",
"method": "use",
"ret": "txt",
"paytoqs": false,
"url": "",
"tls": "",
"persist": false,
"proxy": "",
"authType": "",
"x": 1753,
"y": 374,
"wires": [
[
"423efc4c.78f844"
]
]
},
{
"id": "423efc4c.78f844",
"type": "debug",
"z": "e77f9f87.0c622",
"name": "Response",
"active": false,
"tosidebar": true,
"console": false,
"tostatus": false,
"complete": "true",
"targetType": "full",
"x": 1743,
"y": 434,
"wires": []
},
{
"id": "603435d.861a6cc",
"type": "mqtt in",
"z": "e77f9f87.0c622",
"name": "",
"topic": "emon/emontx3cm15/P1",
"qos": "2",
"datatype": "auto",
"broker": "36372260.fb268e",
"x": 180,
"y": 730,
"wires": [
[
"47b07f5.016868"
]
]
},
{
"id": "35809296.8b0cce",
"type": "inject",
"z": "e77f9f87.0c622",
"name": "Every 5 minutes",
"props": [
{
"p": "payload",
"v": "5",
"vt": "num"
},
{
"p": "topic",
"v": "",
"vt": "string"
}
],
"repeat": "",
"crontab": "*/5 5-20 * * *",
"once": false,
"onceDelay": 0.1,
"topic": "",
"payload": "5",
"payloadType": "num",
"x": 170,
"y": 610,
"wires": [
[
"7254cdc7.0401b4"
]
]
},
{
"id": "7254cdc7.0401b4",
"type": "function",
"z": "e77f9f87.0c622",
"name": "Calculate Average Solar Power",
"func": "// If PV panels have not been limited in the past period\n// calculate average solar power\nvar solar_history = [];\nvar frequency_history = [];\nvar average_solar_power = 0;\nvar solar_total = 0;\nvar period = msg.payload;\nsolar_history = flow.get(\"solar_history\") || [];\nfrequency_history = global.get(\"frequency_history\");\n\n// Calculate average solar power\nsolar_intervals = solar_history.length;\nfor (i = 0; i < solar_intervals; i++) {\n solar_total += solar_history[i];\n}\naverage_solar_power = solar_total / solar_intervals;\n\n// Clear solar_history array\nsolar_history = [];\nflow.set(\"solar_history\", solar_history);\n\n// Cleck to see if PV panels have been limited\nfrequency_intervals = period * 6;\nstatus = 0;\nfor (i = 0; i < frequency_intervals; i++) {\n if (frequency_history[i] > 51) {\n status++;\n }\n}\n\n// If PV panels have not been limited send on data\nif (status === 0) {\n msg.solar_total = solar_total;\n msg.solar_intervals = solar_intervals;\n msg.average_solar_power = average_solar_power;\n msg.period = period;\n} else {\n msg = null; \n}\nreturn msg;",
"outputs": 1,
"noerr": 0,
"initialize": "",
"finalize": "",
"x": 470,
"y": 610,
"wires": [
[
"1c3a7ed0.68d9a1",
"7e6b6634.6748d8"
]
]
},
{
"id": "1c3a7ed0.68d9a1",
"type": "debug",
"z": "e77f9f87.0c622",
"name": "",
"active": false,
"tosidebar": true,
"console": false,
"tostatus": false,
"complete": "true",
"targetType": "full",
"x": 550,
"y": 650,
"wires": []
},
{
"id": "47b07f5.016868",
"type": "function",
"z": "e77f9f87.0c622",
"name": "Push into solar_history array",
"func": "var solar_history = [];\nsolar_history = flow.get(\"solar_history\") || [];\ncurrent_solar_power = parseInt(msg.payload);\nif (current_solar_power < 0) {\n current_solar_power = 0;\n}\nsolar_history.push(current_solar_power);\nflow.set(\"solar_history\", solar_history);\nmsg.payload = solar_history;\nreturn msg;",
"outputs": 1,
"noerr": 0,
"initialize": "",
"finalize": "",
"x": 460,
"y": 730,
"wires": [
[]
]
},
{
"id": "7e6b6634.6748d8",
"type": "change",
"z": "e77f9f87.0c622",
"name": "Set timestamp in ISO 8601 format",
"rules": [
{
"t": "set",
"p": "timestamp",
"pt": "msg",
"to": "$now()",
"tot": "jsonata"
}
],
"action": "",
"property": "",
"from": "",
"to": "",
"reg": false,
"x": 820,
"y": 610,
"wires": [
[
"2e4ff927.8cb916"
]
]
},
{
"id": "2e4ff927.8cb916",
"type": "function",
"z": "e77f9f87.0c622",
"name": "Format for publishing to Solcast",
"func": "var measurement = { measurement: { period: \"PT\" + msg.period + \"M\", period_end: msg.timestamp, total_power: msg.average_solar_power / 1000 }};\nmsg.payload = measurement;\nreturn msg;",
"outputs": 1,
"noerr": 0,
"x": 1150,
"y": 610,
"wires": [
[
"65c3c1c7.d02e6",
"dc8ade32.dd41c"
]
]
},
{
"id": "dc8ade32.dd41c",
"type": "http request",
"z": "e77f9f87.0c622",
"name": "http post to Solcast of average solar power",
"method": "POST",
"ret": "txt",
"paytoqs": true,
"url": "https://api.solcast.com.au/rooftop_sites/4c59-af78-a1a0-ee25/measurements",
"tls": "",
"persist": false,
"proxy": "",
"authType": "basic",
"x": 1500,
"y": 610,
"wires": [
[
"d97da441.8ece88"
]
]
},
{
"id": "65c3c1c7.d02e6",
"type": "debug",
"z": "e77f9f87.0c622",
"name": "",
"active": false,
"tosidebar": true,
"console": false,
"tostatus": false,
"complete": "true",
"targetType": "full",
"x": 1170,
"y": 650,
"wires": []
},
{
"id": "d97da441.8ece88",
"type": "debug",
"z": "e77f9f87.0c622",
"name": "Response",
"active": false,
"tosidebar": true,
"console": false,
"tostatus": false,
"complete": "true",
"targetType": "full",
"x": 1540,
"y": 650,
"wires": []
},
{
"id": "697c0642.c844a8",
"type": "inject",
"z": "e77f9f87.0c622",
"name": "Every hour",
"repeat": "",
"crontab": "0 6-20 * * *",
"once": false,
"onceDelay": 0.1,
"topic": "",
"payload": "",
"payloadType": "date",
"x": 150,
"y": 350,
"wires": [
[
"cf26c8cc.a7ddb8"
]
]
},
{
"id": "cf26c8cc.a7ddb8",
"type": "http request",
"z": "e77f9f87.0c622",
"name": "Solcast Forecast Data",
"method": "GET",
"ret": "txt",
"paytoqs": false,
"url": "https://api.solcast.com.au/rooftop_sites/4c59-af78-a1a0-ee25/forecasts?format=json",
"tls": "",
"persist": false,
"proxy": "",
"authType": "basic",
"x": 350,
"y": 350,
"wires": [
[
"60ddbfef.944"
]
]
},
{
"id": "60ddbfef.944",
"type": "json",
"z": "e77f9f87.0c622",
"name": "Convert from JSON to Object",
"property": "payload",
"action": "",
"pretty": false,
"x": 620,
"y": 350,
"wires": [
[
"a3b4c0b3.ec825"
]
]
},
{
"id": "4588b3cc.d924dc",
"type": "comment",
"z": "e77f9f87.0c622",
"name": "Update forecast data each hour",
"info": "",
"x": 170,
"y": 290,
"wires": []
},
{
"id": "c5174864.24a4a8",
"type": "comment",
"z": "e77f9f87.0c622",
"name": "Reset total daily solar forecast",
"info": "",
"x": 160,
"y": 150,
"wires": []
},
{
"id": "5dfe3d80.2eecf4",
"type": "comment",
"z": "e77f9f87.0c622",
"name": "Send acutals to Solcast",
"info": "",
"x": 140,
"y": 560,
"wires": []
},
{
"id": "227a3047.4dbae",
"type": "debug",
"z": "e77f9f87.0c622",
"name": "",
"active": false,
"tosidebar": true,
"console": false,
"tostatus": false,
"complete": "true",
"targetType": "full",
"x": 1450,
"y": 200,
"wires": []
},
{
"id": "dc8b581d.d24778",
"type": "inject",
"z": "e77f9f87.0c622",
"name": "",
"repeat": "",
"crontab": "",
"once": false,
"onceDelay": 0.1,
"topic": "",
"payload": "39.157",
"payloadType": "num",
"x": 1290,
"y": 140,
"wires": [
[
"ece67ca9.59b8f"
]
]
},
{
"id": "fa4446a0.3b08d8",
"type": "comment",
"z": "e77f9f87.0c622",
"name": "Initialise",
"info": "",
"x": 100,
"y": 40,
"wires": []
},
{
"id": "ff28e9b.8532c18",
"type": "inject",
"z": "e77f9f87.0c622",
"name": "Initialise",
"repeat": "",
"crontab": "",
"once": true,
"onceDelay": "0.1",
"topic": "",
"payload": "",
"payloadType": "date",
"x": 140,
"y": 80,
"wires": [
[
"d04d3d3c.d80fe"
]
]
},
{
"id": "d04d3d3c.d80fe",
"type": "function",
"z": "e77f9f87.0c622",
"name": "Initialise Flow Variables",
"func": "var daily_forecast_data = [];\nfor (i = 0; i <= 32; i++) { // 0 is the 0530 forecast slot. First forecast at 0500 is 0530.\n daily_forecast_data.push(0);\n}\nflow.set(\"daily_forecast_data\",daily_forecast_data);\nmsg.daily_forecast_data = daily_forecast_data;\nreturn msg;",
"outputs": 1,
"noerr": 0,
"x": 390,
"y": 80,
"wires": [
[
"17ea05ff.2a1a5a"
]
]
},
{
"id": "17ea05ff.2a1a5a",
"type": "debug",
"z": "e77f9f87.0c622",
"name": "Initialised Flow Variables",
"active": false,
"tosidebar": true,
"console": false,
"tostatus": false,
"complete": "true",
"targetType": "full",
"x": 670,
"y": 80,
"wires": []
},
{
"id": "6dc37916.c105d8",
"type": "mqtt out",
"z": "e77f9f87.0c622",
"name": "",
"topic": "emon/Solcast/NextDailyForecast",
"qos": "2",
"retain": "true",
"broker": "36372260.fb268e",
"x": 1530,
"y": 300,
"wires": []
},
{
"id": "1e2f015b.c3649f",
"type": "inject",
"z": "e77f9f87.0c622",
"name": "Initialise",
"props": [
{
"p": "payload"
},
{
"p": "topic",
"vt": "str"
}
],
"repeat": "",
"crontab": "",
"once": true,
"onceDelay": "45",
"topic": "",
"payload": "",
"payloadType": "date",
"x": 140,
"y": 225,
"wires": [
[
"66dca2c9.3eb33c"
]
]
},
{
"id": "36372260.fb268e",
"type": "mqtt-broker",
"z": "",
"name": "emonbase mqtt broker",
"broker": "localhost",
"port": "1883",
"clientid": "",
"usetls": false,
"compatmode": false,
"keepalive": "60",
"cleansession": true,
"birthTopic": "",
"birthQos": "0",
"birthPayload": "",
"closeTopic": "",
"closeQos": "0",
"closePayload": "",
"willTopic": "",
"willQos": "0",
"willPayload": ""
}
]
Please let me know how you go.
Hi Simon, Just want to say thanks for that.
I’ve imported the whole lot - even though I’ve no need for the forecasts yet, they do work fine!
I’ve managed to strip out all the frequency related code and after checking the solcast measurements API throughout today it seems to be working fine.
Yes, but looks like we can finally get a split system, which has been a big shortcoming for me…
Since we will be removing the PV Tuning capability, we are now providing hobbyists with the ability to add a second PV site at their home location, in order to handle split-array set-ups.
[Edited for legibility - Moderator (RW)]
So there is no value to anyone, even Solcast, from the POST values? I find that hard to believe, but if it is the case then I should probably stop POSTING.
Now I’ve reread it when I’ve not had a beer or two, it seems they’re still continuing with it for hobbyists, but not as a commercial endpoint.
I’m finding it hard to read what that actually means - as surely there’s no longer an incentive for them to keep offering it? If there truely was no difference for hobbyists, why are they offering a 2nd site as a replacement benefit?
I read that as they’ll be retaining the Rooftop PV Sites product, and removing tuning altogether.
Confirmed when I registered a “rooftop” account a few minutes ago. (1125Z 24 Feb 21)
When registration was complete, I received a pop-up indicating tuning was no longer supported.
I was wondering why my site stopped tuning on 30 Dec. I thought they had just taken a break for Christmas and the holidays. 
I’ve been using this for a while but today I noticed values of ‘zero’ for today’s forecast and ‘NaN’ for next daily forecast. If I do an initialise I get this error in the reset daily forecast
“TypeError: Cannot read property ‘pv_estimate’ of undefined”
Is anyone else getting errors?
Seems it no longer likes 120 max is now 96 in
var daily_forecast_data = [];
for (i = 0; i <= 120; i++) { // 0 is the 0530 forecast slot. First forecast at 0500 is 0530.
forecast_output = msg.payload.forecasts[i].pv_estimate
daily_forecast_data.push(forecast_output);
}
flow.set("daily_forecast_data",daily_forecast_data);
return msg;
Solcast are in the process of upgrading their api and the default for their rooftop_sites call is now 96 hours. You can get up to 336 hours by adding ‘hours=336’ to your api call.