Community
OpenEnergyMonitor

Community

PVOutput Python Script (emoncms.org > PVOutput)


(Mick Wall) #1

For the past few months i’d been using the bash script kindly donated by @Bill.Thomson

To get my emoncms data up in to PVOutput to have an alternative view on things.

But I found that in my setup the bash script occasionally dropped out and sent zero watts (both solar and usage) to PVOutput.

So I decided to try and re-write the whole thing in Python. I’m only a hobbyist coder at best so this has taken a little time.

I’ve tried to add some logging as well as some error checking.

Running this new python script alongside the bash script i’ve found mine doesn’t drop out when the bash one does. So an improvement of sorts.

I also added new functionality to grab the current temperature from DarkSky and also the current sun angle based upon your latitude and longitude.

There are comments within the code and a description up top. Hopefully this can help people out getting this to work in their system.

I decided to pull the data from emoncms.org so that I could code this in my lunch hours. :wink:

I’m sure you could amend the code to talk to your local instance if required.

#!/usr/bin/env python

# pvoutput_upload.py
# Pulls Solar and Home use data from your emoncms.org account and squirts the data up to PVOutput
#
# This can run on any internet host running Python3 and you should schedule the script to run every 5 mins
# You may be able to change to talk to your internal emonpi instance if you don't want it to go external.
#
# */5 * * * * /usr/bin/python3 /home/PVOutput/pvoutput_upload.py
#
# 5 mins is the shortest time period you can send to PVO (in donation mode), otherwise its 15 mins if you don't have a donation PVO account
# https://forum.pvoutput.org/c/donation-features
#
# Based on your longtitude and latitude the script also pulls down current tempearture (from DarkSky) and uses Pysolar to calculate the current sun angle.
# Use a website like https://www.latlong.net/ to calculate your lon and lat
#
# Note: For DarkSky you'll need to apply for a free API key via https://darksky.net/dev
# Note: The script is sending the sun angle data to v6 in PVO which is normally reserved for Voltage.  But I do this so I see angle on the live displayed graph.
# 
# As well as needing Python 3 installed you'll need to install the following modules
# sudo pip3 install pysolar
# sudo pip3 install numpy
# sudo pip3 install pytz

import time
import datetime
import json
import requests

from pysolar.solar import *
from pytz import timezone

##############################################################

def get_darksky_weather():

  option_list = "?units=si&exclude=minutely,hourly,alerts,daily"
  url = 'https://api.darksky.net/forecast/%s/%s,%s?%s' % (darkskyapi,lat,lon,option_list)

  if DEBUG ==1:
    app_log.info(url)

   # Get the data from Dark Sky.
  try:
    response = requests.get(url)
    json_res = response.json()
  except requests.exceptions.RequestException as e:
    if DEBUG ==1:
        app_log.error(e)

  if response.status_code == 200:
    current_temp_fan = round(json_res['currently']['temperature'],1)
    current_temp_cel = round(((current_temp_fan - 32) / 1.8),1)

    #Round temperature to nearest 0.5 otherwise PVOutput graph is very shaky
    current_temp_cel= round(current_temp_cel * 2) / 2
  
  return (current_temp_cel)

##############################################################

def get_emoncms_data(FEED_ID):

  payload = {'id': FEED_ID, 'apikey': EMONPI_APIKEY, 'start': UNIX_MILLI_START, 'end': UNIX_MILLI_END, 'interval': INTERVAL}

  header = {'content-type': 'application/json'}

  try:
	  response = requests.get("http://emoncms.org/feed/data.json", params=payload, headers=header)
  except requests.exceptions.RequestException as e:
    if DEBUG ==1:
        app_log.error('Feed ID: %s', FEED_ID)
        app_log.error(e)

  if response.status_code == 200:
    str_response = response.content.decode("utf-8")
  
    if DEBUG ==1:
        app_log.info(str_response)
        app_log.info(response.url)

    mylist = json.loads(str_response)

    READING = 0
    COUNT = 1
    TOTAL = 0
    TEXT_VALUES = ''

    for x in mylist: 
      #print (COUNT,x)
      READING = (x[1])
      TOTAL = TOTAL + READING
      COUNT = COUNT + 1
      TEXT_VALUES = TEXT_VALUES + (str(int(READING))) + ', '

    #print ()
    COUNT = COUNT - 1
    #print (COUNT)
    #print (TOTAL)
    OUTPUT = (int(TOTAL/COUNT))
    #print (OUTPUT)

    if DEBUG ==1:
        app_log.info('Number of numbers pulled: %s', COUNT)

  else:
    OUTPUT = 0

  return (OUTPUT, TEXT_VALUES)

#########################################################################

# Stick 'Sun Angle' in V6 (Voltage)
# You could always change to v7 to stick it in extended parameters (another PVO donation mode feature)

def post_pvoutput(): # may raise exceptions
  url = 'https://pvoutput.org/service/r2/addstatus.jsp'
  headers = {
    'X-Pvoutput-SystemId': str(PVO_SYSID),
    'X-Pvoutput-Apikey': str(PVO_API)
  }
  params = {
    'd': DATE,
    't': TIME,
    'v2': SOLAR_GEN,
    'v4': HOME_USAGE,
    'v5': TEMP,
    'v6': ALTITUDE
  }
  app_log.info('posting data to pvoutput')
  resp = requests.post(url, headers=headers, data=params, timeout=10)
  if resp.status_code != 200:
    app_log.error(resp.status_code)
    app_log.error('pvoutput returned code %s', resp.status_code)
    app_log.error(resp.text)
  return


##########################################################################

def get_sun_angle():
    uk_tz = timezone('Europe/London')
    # find your timezone from here https://stackoverflow.com/questions/13866926/is-there-a-list-of-pytz-timezones

    ALTITUDE = round((get_altitude(lat, -lon, datetime.datetime.now(uk_tz))),2)
    if ALTITUDE < 0:
        ALTITUDE = 0

    return ALTITUDE

##########################################################################

import logging
from logging.handlers import RotatingFileHandler

log_formatter = logging.Formatter('%(asctime)s %(levelname)s %(funcName)s(%(lineno)d) %(message)s')

# Windows current directory
logFile = 'log.txt'

# Linux absolute path
#logFile = '/home/PVOutput/log.txt'

my_handler = RotatingFileHandler(logFile, mode='a', maxBytes=5*1024*1024, backupCount=2, encoding=None, delay=0)
my_handler.setFormatter(log_formatter)
my_handler.setLevel(logging.INFO)

app_log = logging.getLogger('root')
app_log.setLevel(logging.INFO)

app_log.addHandler(my_handler)

###################################################################################

DEBUG=1

# PVOutput.org details
PVO_API='xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' #your PVOutput.org API key
#PVO_SYSID='99999' # your PVOutput.org system ID
PVO_SYSID='99999' # Testing ID

# EmonCMS.org details
EMONPI_APIKEY='xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
FEED_ID_USE=999999
FEED_ID_SOLAR=399999

# How many seconds between data points 
# (ie, EmonCMS updates every 10 seconds and PVOuput only accepts every 5 mins.... so using 10 seems to catch most of the 29/30 unique values)
INTERVAL=10

# DarkSky API details
lat = 99.4032481162862
lon = -10.52260641495603
darkskyapi = 'xxxxxxxxxxxxxxxxxxxxxx'

# Time right now in milliseconds
UNIX_MILLI_END=int(round(time.time() * 1000))
# 5 minutes ago in milliseconds (PVOutput only accepts 5 minute updates)
UNIX_MILLI_START=((UNIX_MILLI_END-300000))
# If you need to change this to 15 mins then its 900,000


# Date for PVOutput Use* conda install openssl
DATE = time.strftime('%Y%m%d')
TIME = time.strftime('%R')

#print (UNIX_MILLI_START)
#print (UNIX_MILLI_END)

# Reset working variables
SOLAR_GEN = 0
HOME_USAGE = 0
TEMP = 0

# Get Current Angle of the Sun
ALTITUDE=get_sun_angle()
if DEBUG ==1:
    app_log.info('Solar Altitude %s', ALTITUDE)

# Get Home Usage figure from EmonCMS API
FEED_ID = FEED_ID_USE 
HOME_USAGE, TEXT_VALUES = get_emoncms_data(FEED_ID)
if DEBUG ==1:
    app_log.info('Home Usage Raw %s', TEXT_VALUES)
#print (TEXT_VALUES)

# Get Solar Generation figure from EmonCMS API
FEED_ID = FEED_ID_SOLAR
SOLAR_GEN, TEXT_VALUES = get_emoncms_data(FEED_ID)
if DEBUG ==1:
    app_log.info('Solar Usage Raw %s', TEXT_VALUES)
#print (TEXT_VALUES)

# Get temperature from DarkSky API
TEMP = get_darksky_weather()

#print ()
#print (DATE)
#print (TIME)
#print ()
#print ('Solar: ',SOLAR_GEN)
#print ('Usage: ',HOME_USAGE)
#print ('Temp: ',TEMP)


app_log.info('Solar %s', SOLAR_GEN)
app_log.info('Usage: %s', HOME_USAGE)
app_log.info('Temp: %s', TEMP)
app_log.info('Solar Altitude: %s', ALTITUDE)


# Punt collected data up to PVPoutput.org
post_pvoutput()

pvout.zip (2.9 KB)

Edit - zipped and attached file. BT - moderator


(Bill Thomson) #2

Nice work! thumbsup

I rewrote it entirely in Python, but as I use a Modbus based meter, I didn’t post the update.
(Figured it would be of little use to anyone not using Modbus)