PVOutput.org NodeRed

I’ve been using PVOutput.org since my PV solar was installed using a PERL script I wrote to push the data. Now that I’m migrating to EmonBase I’d like to do the same in NodeRed, but the Javascript and the procedural to event driven shift makes it a steep learning curve. So despite being mentioned on this page -

ARCHIVE: : emonPi, NodeRed and MQTT

it doesn’t look like anyone has tried it yet. So, I found this page on Github -

PVoutput-NodeJS/pvoutput-api.js at master · Adeptive/PVoutput-NodeJS · GitHub

that does it in ‘flat’ Javascript. I basically understand how this works, I’ve got code that straightens out the date/time fields but I’m not sure how to do the get request and process the output in NodeRed.

Can anyone point me in the right direction?

Did you successfully setup NodeRed to publish to PVoutput? I’m considering the move too, and also have my history in PVoutput.

Thanks, Geoff

Yes, yes I did, after much head scratching, I learnt enough JS/Node Red to be dangerous. Though I’m posting it to PVO as well as EmonCMS, as the full history of my PV system is on PVO.
So, I’ve coded up a working solution to push data from EmonSD/Node Red to PVO, and it’s been running for about 5 months. It’s hit several problems, similar to my PERL version, but seems to be working fine now. I’ve been meaning to play with GitHub and get it posted up but been too busy with other stuff.

Have a play…
PVOutput.txt (19.1 KB)

The only important thing is you will need to insert your API key/sys id into the two nodes mentioned.
You will also need to tweak or remove the temperature nodes at the bottom to pick up the right ones or not.
Then it should just go. Good luck!

If not, I’ll be around… :grinning:

1 Like

@stryker Any joy with the code?

Hi,

Having this code removes one of two final hurdles in ordering - the other is we will be going 3phase in the future for a car charger, so will need to do more reading on monitoring that too.

Thanks - will let you know how I go but at best I won’t have this installed for a few months yet.

Cheers!
Geoff

Hi there
I’m a relative newbie and would really appreciate a bit of help with this.
I have an emonpi which pulls data from my SMA inverter via Bluetooth (and some other data from other sensors)
I’d like the emonpi to send both power & yield to PVOutput, say every 5 mins
I’ve downloaded the text file but just see a single string of characters in Notepad++
I’d really appreciate it if someone could tell me what I should be doing. I’m reasonably comfortable with Putty, editing config files and that kind of thing but have no coding experience (or aptitude!)
Many thanks in anticipation
Jeremy

Hi Jeremy,

Here’s what I use to send data from emonCMS to PVOutput.org:

#!/bin/bash

PVO_API='xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' #your PVOutput.org API key
PVO_SYSID='xxxxx' # your PVOutput.org system ID

FEED_ID_A='X' # id number of the feed you want to read data from
FEED_ID_B='X'

DATE=$(date "+%Y%m%d")
TIME=$(date "+%H:%M")

A=$(curl -s http://127.0.0.1/emoncms/feed/value.json?id=$FEED_ID_A)
B=$(curl -s http://127.0.0.1/emoncms/feed/value.json?id=$FEED_ID_B)

#A=$(wget -qO- http://127.0.0.1/emoncms/feed/value.json?id=$FEED_ID_A)
#B=$(wget -qO- http://127.0.0.1/emoncms/feed/value.json?id=$FEED_ID_B)

A=${A//\"} # strip quotes from json string
B=${B//\"}

curl -s -d "d=$DATE" -d "t=$TIME" -d "v2=$A" -d "v4=$B" -H "X-Pvoutput-Apikey: $PVO_API" \
-H "X-Pvoutput-SystemId: $PVO_SYSID" http://pvoutput.org/service/r2/addstatus.jsp

#wget --quiet --post-data "d=$DATE&t=$TIME&v2=$A&v4=$B" --header="X-Pvoutput-Apikey:$PVO_API" \
#--header="X-Pvoutput-SystemId:$PVO_SYSID" http://pvoutput.org/service/r2/addstatus.jsp

send-to-pvo.txt (959 Bytes)

As it sits, it uses curl to send the data. If you prefer to use wget, uncomment the wget lines and comment out the curl lines. Either work well, it’s a matter of personal choice.

Set up a cron job to run it every five minutes, and you should be good to go.

That looks great thanks Bill
Does it matter what I name the file or where I save it? Also, how do I tell it what fields in PVOutput these two sets of data are for
Jeremy

Name shouldn’t matter as long as it’s not the same as a bash built-in or any other file on your system that’s in the search path.

I’d recommend your home user directory, but you should be able to save it wherever you like. If you save it in a directory that’s not in the search path, you’ll need to call the file by its full path name, to run it.

Have a look at Help
The relevant info is in the Add Status Service section

Many thanks
I feel I’m getting somewhere but not quite there
I now have a file called sendtopvoutput in the Home directory but when I try to run it (want to check it’s ok before setting up cron) I get “-bash sendtopvoutput Command not found message” - would you mind taking a quick shufty and pointing out any obvious errors. I tried changing the 127.0.0.1 to 192.168.1.xxx etc but that didn’t help
Very grateful for any help
Jeremy

#!/bin/bash

PVO_API='7590274xxxxxxxxxxe8aae11981ea2cxxxxxx' #your PVOutput.org API key
PVO_SYSID='57xx' # your PVOutput.org system ID

FEED_ID_A='14' # id number of the feed you want to read data from
FEED_ID_B='17'

DATE=$(date "+%Y%m%d")
TIME=$(date "+%H:%M")

A=$(curl -s http://127.0.0.1/emoncms/feed/value.json?id=$FEED_ID_A)
B=$(curl -s http://127.0.0.1/emoncms/feed/value.json?id=$FEED_ID_B)

#A=$(wget -qO- http://127.0.0.1/emoncms/feed/value.json?id=$FEED_ID_A)
#B=$(wget -qO- http://127.0.0.1/emoncms/feed/value.json?id=$FEED_ID_B)

A=${A//\"} # strip quotes from json string
B=${B//\"}

curl -s -d "d=$DATE" -d "t=$TIME" -d "v2=$A" -d "v3=$B" -H "X-Pvoutput-Apikey: $PVO_API" \
-H "X-Pvoutput-SystemId: $PVO_SYSID" https://pvoutput.org/service/r2/addstatus.jsp

#wget --quiet --post-data "d=$DATE&t=$TIME&v2=$A&v4=$B" --header="X-Pvoutput-Apikey:$PVO_API" \
#--header="X-Pvoutput-SystemId:$PVO_SYSID" https://pvoutput.org/service/r2/addstatus.jsp

Say for example, the script is in the directory /home/jeremy
The command to run it will be:

/home/jeremy/sendtopvoutput

If the name of the directory isn’t jeremy, substitute the name of the directory the script is in for “jeremy” in the example above

If you’re currently logged in to the same directory the script is in, the command is:
./sendtopvoutput

Have you made the script executable? If not, you’ll need to do that too.
Change to the directory the script is in and run the following command:

sudo chmod 755 sendtopvoutput

Many thanks Bill
I got that far but still ‘command not found’ so I tried hashing out the curl and unhashing wget to try that way

More promisingly it tries to run with the message …
line 22 -H command not found wget missing URL

but the URL looks good to me. Any thoughts?

The line that calls wget (as well as the line that calls curl) has a \ character in it which makes bash treat the two lines as one contiguous line.

By any chance was the \ removed?

Thanks Bill, I don’t think so, the 2 lines read…

wget --quiet --post-data "d=$DATE&t=$TIME&v2=$A&v3=$B" --header="X-Pvoutput-Apikey:$PVO_API" \
#--header="X-Pvoutput-SystemId:$PVO_SYSID" https://pvoutput.org/service/r2/addstatus.jsp

OK, there’s the problem. You forgot to uncomment the second line.

Remove the hash mark from the second line, then:

  1. ensure there’s a backslash (\) at the end of first line,

or

  1. make one long line out of the two.

BINGO!
All sorted now> Had a last minute hitch with my feeds not being public but now sorted and posting every 15 mins. Sincere thanks for your help
Jeremy

Sounds good!

YWV,S!

Just for fun and it is not my own work (unfortunately I can not remember where I found this) **edit: luckily some else did read the beginning of this post it came from the top and I can confirm it works well ** - here is a flow that you could use to send the data to PVO. Once you load the flow in node red the info with in the node is quite well written and I did not have too many issues setting it up.

It has been running now for about 6 months and so far so good.

[
    {
        "id": "d48307fe.7d5148",
        "type": "inject",
        "z": "c2fb8649.3f0648",
        "name": "Send Manually",
        "topic": "",
        "payload": "",
        "payloadType": "str",
        "repeat": "",
        "crontab": "",
        "once": false,
        "x": 100,
        "y": 120,
        "wires": [
            [
                "9302df8.e415d2"
            ]
        ]
    },
    {
        "id": "d3b32b36.3bb128",
        "type": "http request",
        "z": "c2fb8649.3f0648",
        "name": "Post",
        "method": "POST",
        "ret": "txt",
        "url": "",
        "x": 650,
        "y": 120,
        "wires": [
            [
                "240bf226.cdb60e",
                "748ebf48.12ee3"
            ]
        ]
    },
    {
        "id": "9302df8.e415d2",
        "type": "function",
        "z": "c2fb8649.3f0648",
        "name": "Set API key here 1",
        "func": "msg.action = msg.payload;\nmsg.headers = { \n 'X-Pvoutput-Apikey': 'xxxxxxxxxxxxxxxxxxxxxx',\n 'X-Pvoutput-SystemId': 'xxxxxx',\n 'Content-Type': 'application/x-www-form-urlencoded'\n};\n\nreturn msg;",
        "outputs": 1,
        "noerr": 0,
        "x": 310,
        "y": 120,
        "wires": [
            [
                "ab58678a.3f8428"
            ]
        ]
    },
    {
        "id": "240bf226.cdb60e",
        "type": "debug",
        "z": "c2fb8649.3f0648",
        "name": "headercheck",
        "active": false,
        "console": "false",
        "complete": "true",
        "x": 830,
        "y": 120,
        "wires": []
    },
    {
        "id": "d5337235.a0797",
        "type": "inject",
        "z": "c2fb8649.3f0648",
        "name": "Every 5 mins",
        "topic": "",
        "payload": "",
        "payloadType": "str",
        "repeat": "300",
        "crontab": "",
        "once": false,
        "x": 100,
        "y": 160,
        "wires": [
            [
                "9302df8.e415d2"
            ]
        ]
    },
    {
        "id": "1ba6ca5d.b7a3a6",
        "type": "comment",
        "z": "c2fb8649.3f0648",
        "name": "Send data to PVOutput.org and check if it was recieved ok",
        "info": "This flow allows you to send data to PVOutput.org\n\nYou'll need to sign up an account (it's free) and get\nAPI key.\n\nYou need to put this API key and the System ID\ninto both the \"Set API Key here\" boxes.\n\nMake sure you have set the correct Time Zone in the console - \n  sudo dpkg-reconfigure tzdata\n  \nAnd also in the EmonCMS webpage account settings.\n\nOtherwise you'll get offset graphs and data.",
        "x": 209.00003051757812,
        "y": 74.00006103515625,
        "wires": []
    },
    {
        "id": "ef123ee3.30e0f",
        "type": "debug",
        "z": "c2fb8649.3f0648",
        "name": "Debug1",
        "active": false,
        "console": "true",
        "complete": "payload",
        "x": 650,
        "y": 160,
        "wires": []
    },
    {
        "id": "ab58678a.3f8428",
        "type": "function",
        "z": "c2fb8649.3f0648",
        "name": "Set up data",
        "func": "msg.action = msg.payload;\n\nmsg.url = \"http://pvoutput.org/service/r2/addstatus.jsp\";\n\n// set up the data/time\nvar now     = new Date(); \nvar year    = now.getFullYear(); \nvar month   = now.getMonth()+1; // Months start at 0 not 1 - FTFY\nvar day     = now.getDate();\nvar hour    = now.getHours(); \n\n//var hour    = now.getHours()+1; // GMT to BST\n//if (hour === 24) { // BST overflow - This clearly needs some work!\n//    hour = 0;\n//    day = day +1;\n//}     \n\nvar minute  = now.getMinutes();\nvar second  = now.getSeconds(); \n\n// fix any short date/times so all values are two digits\nif(month.toString().length == 1)  { var month = '0'+month; }\nif(day.toString().length == 1)    { var day = '0'+day;}\nif(hour.toString().length == 1)   { var hour = '0'+hour; }\nif(minute.toString().length == 1) { var minute = '0'+minute; }\nif(second.toString().length == 1) { var second = '0'+second; }\n\nuploaddate = year.toString()+month.toString()+day.toString(); // string the dates or we get errors!\n\n//pull back stored data\nvar power1 = global.get('gridwatts')||0;  // Power1 - Grid power import/export\nvar power2 = global.get('panelwatt')||0;  // Power2 - Solar power import\nvar power3 = global.get('batterywatt')||0;  // Power3 - My EV charger (you may use this for something else - or not)\n//var vrms   = flow.get('vrms')||0;    // Grid voltage\nvar contot = global.get('invkwh')*1000||0;  // Consumption total for the day (watts)\nvar gentot = global.get('panelkwh')*1000||0;  // Generation total for the day (watts)\n//var power1max = flow.get('power1max')||0;  // Power1max - maximum Power1 of the last 5mins\nvar temp1 = global.get('temp1')||0;\nvar invwatt = global.get('invwatt')||0;\nvar battvolts = global.get('battvolts')||0;\n\n//contot = (Math.round(contot*100))/100;     // trim to 2dp\nvar powerc = power1 + power2; // Consumption power now (need to convert back from string to int in decimal). Use Power1 or Power1max as described in function StorePower1\n// v1=gentotal v2=solar v3=contotal v4=con_max v5=t v6=EV v7=t_hall v8=t_landing v9=t_attic v10=t_outside v11=t_top v12=t_bottom\n//msg.payload = {\"d\":year+month+day,\"t\":hour+':'+minute,\"v1\":gentot,\"v2\":power2,\"v3\":contot,\"v4\":power1max,\"v6\":vrms,\"v8\":\"28.1\"};\nmsg.payload = {\"d\":uploaddate,\"t\":hour+':'+minute,\"v1\":gentot,\"v2\":power2,\"v3\":contot,\"v4\":invwatt,\"v5\":temp1,\"v6\":battvolts};\n\n//flow.set('power1max',0); // reset maximum Power1 for the next five min period\n\nreturn msg;",
        "outputs": 1,
        "noerr": 0,
        "x": 510,
        "y": 120,
        "wires": [
            [
                "d3b32b36.3bb128",
                "ef123ee3.30e0f",
                "384e9ccb.f184e4"
            ]
        ]
    },
    {
        "id": "748ebf48.12ee3",
        "type": "function",
        "z": "c2fb8649.3f0648",
        "name": "Check Status 1",
        "func": "if (msg.statusCode == 200) { \n   var stat = \"\";\n   flow.set('pvostat','200');  // this clears the outstanding PVO data, so it won't be uploaded again\n   var msg1 = null;\n   var msg2 = null;\n} else {\n   var time = new Date().toString();\n   flow.set('pvostat', msg.statusCode);\n   var stat = \"FAILED: Time:\" + time + \"  StatusCode:\" + msg.statusCode + \"  StatusMsg:\" + msg.payload;\n   var msg1 = null;\n   var msg2 = null;\n//   msg1.payload = stat;\n//   msg2.payload = stat;\n   var msg1 =  { payload: stat };\n   var msg2 =  { payload: stat };\n}\nreturn (msg1, msg2);",
        "outputs": "2",
        "noerr": 0,
        "x": 840,
        "y": 160,
        "wires": [
            [
                "da23c62c.e5ccc8",
                "e4b50911.ce24c8"
            ],
            [
                "b6561318.02785"
            ]
        ]
    },
    {
        "id": "da23c62c.e5ccc8",
        "type": "debug",
        "z": "c2fb8649.3f0648",
        "name": "Post dump1",
        "active": false,
        "console": "false",
        "complete": "payload",
        "x": 1050,
        "y": 120,
        "wires": []
    },
    {
        "id": "2b8d93c.585f76c",
        "type": "inject",
        "z": "c2fb8649.3f0648",
        "name": "Every Midnight",
        "topic": "",
        "payload": "",
        "payloadType": "date",
        "repeat": "",
        "crontab": "",
        "once": false,
        "x": 100,
        "y": 200,
        "wires": [
            [
                "9302df8.e415d2"
            ]
        ]
    },
    {
        "id": "b6561318.02785",
        "type": "file",
        "z": "c2fb8649.3f0648",
        "name": "PVO error log1",
        "filename": "/home/yourfolder/pvolog/pvo_errors.log",
        "appendNewline": true,
        "createDir": true,
        "overwriteFile": "false",
        "x": 1060,
        "y": 200,
        "wires": []
    },
    {
        "id": "384e9ccb.f184e4",
        "type": "function",
        "z": "c2fb8649.3f0648",
        "name": "Store PVO data",
        "func": "//store the PVO data in case it doesn't upload correctly\nflow.set ('pvodata',msg.payload);\nreturn msg;",
        "outputs": 1,
        "noerr": 0,
        "x": 480,
        "y": 200,
        "wires": [
            [
                "15a81555.395eab"
            ]
        ]
    },
    {
        "id": "e4b50911.ce24c8",
        "type": "file",
        "z": "c2fb8649.3f0648",
        "name": "PVO Data",
        "filename": "/home/yourfolder/pvolog/PVO_data.log",
        "appendNewline": true,
        "createDir": true,
        "overwriteFile": "false",
        "x": 1040,
        "y": 160,
        "wires": []
    },
    {
        "id": "15a81555.395eab",
        "type": "debug",
        "z": "c2fb8649.3f0648",
        "name": "Data stored",
        "active": false,
        "console": "false",
        "complete": "payload",
        "x": 650,
        "y": 200,
        "wires": []
    },
    {
        "id": "54f64e90.bca2e",
        "type": "http request",
        "z": "c2fb8649.3f0648",
        "name": "Post2",
        "method": "POST",
        "ret": "txt",
        "url": "",
        "tls": "",
        "x": 670,
        "y": 280,
        "wires": [
            [
                "2c6fa3ea.1c302c",
                "f9edd6d0.919298"
            ]
        ]
    },
    {
        "id": "8fed7ee7.bbb9c",
        "type": "function",
        "z": "c2fb8649.3f0648",
        "name": "Set API key here 2",
        "func": "msg.action = msg.payload;\nmsg.headers = { \n 'X-Pvoutput-Apikey': 'xxxxxxxxxxxxxxxxxxxxxxxxxxxx',\n 'X-Pvoutput-SystemId': 'xxxxxxx',\n 'Content-Type': 'application/x-www-form-urlencoded'\n};\n\nmsg.url = \"http://pvoutput.org/service/r2/addstatus.jsp\";\n\nreturn msg;",
        "outputs": 1,
        "noerr": 0,
        "x": 510,
        "y": 280,
        "wires": [
            [
                "54f64e90.bca2e",
                "b4c3f030.efa5e"
            ]
        ]
    },
    {
        "id": "2c6fa3ea.1c302c",
        "type": "debug",
        "z": "c2fb8649.3f0648",
        "name": "headercheck2",
        "active": false,
        "console": "false",
        "complete": "true",
        "x": 860,
        "y": 280,
        "wires": []
    },
    {
        "id": "e4b80edf.cff53",
        "type": "inject",
        "z": "c2fb8649.3f0648",
        "name": "Check every 10 min",
        "topic": "",
        "payload": "",
        "payloadType": "str",
        "repeat": "300",
        "crontab": "",
        "once": false,
        "x": 120,
        "y": 280,
        "wires": [
            [
                "88480d4e.63edb"
            ]
        ]
    },
    {
        "id": "b4c3f030.efa5e",
        "type": "debug",
        "z": "c2fb8649.3f0648",
        "name": "Debug2",
        "active": false,
        "console": "true",
        "complete": "payload",
        "x": 650,
        "y": 320,
        "wires": []
    },
    {
        "id": "f9edd6d0.919298",
        "type": "function",
        "z": "c2fb8649.3f0648",
        "name": "Check Status 2 ",
        "func": "if (msg.statusCode == 200) { \n   var time = new Date().toString();\n   var stat =\"\";\n   flow.set('pvostat', 200);  // this clears the outstanding PVO status, so it won't be uploaded again\n   var stat = \"SENTOK: Time:\" + time + \"  StatusCode:\" + msg.statusCode + \"  StatusMsg:\" + msg.payload;\n//   var msg1 = null;\n//   var msg2 = null;\n   var msg1 =  { payload: stat };\n   var msg2 =  { payload: stat };\n} else {\n   var time = new Date().toString();\n   flow.set('pvostat', msg.statusCode);\n   var stat = \"ReFAIL: Time:\" + time + \"  StatusCode:\" + msg.statusCode + \"  StatusMsg:\" + msg.payload;\n//   var msg1 = null;\n//   var msg2 = null;\n   var msg1 =  { payload: stat };\n   var msg2 =  { payload: stat };\n}\nreturn (msg1, msg2);",
        "outputs": "2",
        "noerr": 0,
        "x": 860,
        "y": 320,
        "wires": [
            [
                "2ce42647.1ed3ca"
            ],
            [
                "55f55a40.fa3a54"
            ]
        ]
    },
    {
        "id": "2ce42647.1ed3ca",
        "type": "debug",
        "z": "c2fb8649.3f0648",
        "name": "Post dump2",
        "active": false,
        "console": "false",
        "complete": "payload",
        "x": 1070,
        "y": 280,
        "wires": []
    },
    {
        "id": "55f55a40.fa3a54",
        "type": "file",
        "z": "c2fb8649.3f0648",
        "name": "PVO Error log2",
        "filename": "/home/yourhome/pvolog/pvo_errors.log",
        "appendNewline": true,
        "createDir": true,
        "overwriteFile": "false",
        "x": 1080,
        "y": 320,
        "wires": []
    },
    {
        "id": "88480d4e.63edb",
        "type": "function",
        "z": "c2fb8649.3f0648",
        "name": "Fetch PVO data",
        "func": "//store the PVO data in case it doesn't upload correctly\nvar pvostat = flow.get('pvostat')||200;\nif (pvostat == 200) {\n    msg = null;\n} else {\n   var pvodata = flow.get('pvodata')||\"\";\n    msg.payload = pvodata;\n}\nreturn msg;",
        "outputs": 1,
        "noerr": 0,
        "x": 320,
        "y": 280,
        "wires": [
            [
                "8fed7ee7.bbb9c"
            ]
        ]
    },
    {
        "id": "50538838.385258",
        "type": "comment",
        "z": "c2fb8649.3f0648",
        "name": "Check data sent OK",
        "info": "The PVO website is busy and sometimes can't respond quickly enough and we get timeouts.\nThe next bit of code catches any errors and resubm its the code until it's sent OK.\n\nThat said, if it can't resend the data before the next data packet has to be sent (5 mins) then\nthe current packet will be lost. This is pretty rare though and won't be noticable on the graphs.\n",
        "x": 90,
        "y": 240,
        "wires": []
    }
]

Hi @Paul_F_Prinsloo you found it at the top of this thread! :grinning:

1 Like

:rofl::rofl::joy: - Serves me for just posting and not reading the whole tread :zipper_mouth_face: - post was edit accordingly thank you!