Python script to import Octopus Agile consumption data into emoncms

@TrystanLea

Yes - I finally figured out what was needed for feedid.

And with @borpins’s steer to Python3, I got the script to run OK. It also worked 24 hrs later just doing an incremental update.

Octopus sends kWh data to 2 decimal places which emoncms rounds (or truncates?) and so I’ve multiplied the value line by 1000 and show the units as Wh’s – this preserves all the Octopus data.

Because of the time I spent, I’ve added comments in my copy of the script:


# Script by Trystan Lea ref: https://community.openenergymonitor.org/t/python-script-to-import-octopus-agile-consumption-data-into-emoncms/15230

# Requires Python 3

# To run the script in an SSH terminal do: /usr/bin/python3 /home/pi/script-name.py (octopus_consumption_perTL.py)

# To run the script as a daily cronjob do: exactly the same

# Unlike other 'devices' which provide INPUT data as it occurs at n secs intervals, Octopus is providing a FEED of historical 30mins consumption data

# In consequence, it is necessary to FIRST create an emoncms FEED with a chosen tag, name & interval. The ID NUMBER for this FEED must then be entered in the script below at feedid =

These may/may not be of help to others.

Thx for yr work on this.
Yr new feed/insert.json functionality is a great addition which I think may have many uses.

That shouldnt be the case, mine have all imported to 3 decimal places (watt hour resolution). Are you looking at the values via the graph module? There’s a decimal place option to show up to 3 dp in the graph UI.

Great thanks!

never noticed that!

@TrystanLea

Indeed there is - just found it - one lives & learns …

It’s in the Feeds in View box under the graph ‘hidden’ under a column heading DP - pls give us oldies a break - there’s enough space there for a column heading Dec pts or something similar.

But banter to one side - Octopus appears to send the data at 10 Wh resolution - so more trailing zero’s is just spurious accuracy.

So it’s down to user choice (add *1000 to one script line). And personally I favour Wh’s have just invested heavily in PowerWalls and am hoping the Octopus numbers will be low.

:slight_smile:

thanks tristan, another user of your script, just got it working, plus discovered your app as well, thanks again

1 Like

Hi @TrystanLea

I think I have found an issue with this where you have a solar feed.

My first attempt did not show Agile Import Actual data.

Settings:-

Second test attempt leaving out solar feed works as expected.

settings:-

j

Thanks @ian yes correct, it does not yet support showing the smart meter data in Solar PV mode. Do your agile consumption half hourly values otherwise show both import and export values? If you look at them outside the app…

Its not a problem to me, it just confused me first time I tried it. The real value to me was the ability to get correction factors and I can run the app without solar to get those.
I have all the data outside the app in legacy dashboards but the app is much slicker and what I tend to look at.

@TrystanLea, I now have some Agile data - only taken 6 months to get meters etc!

Just playing with the script.

  1. Is it possible to auto create the feed (on first run) then check to see if the Feed exists on each run rather than having to create the feed manually?
  2. One disadvantage of this method (over my Node-Red version) is it doesn’t create a cumulative feed. Can that be done via post process?
  3. How do I compare a measured feed with this data?

I think this script should go into the useful scripts repository with an install script to make it a bit more accessible to others.

Is this now in stable?

1 Like

Would love to try this but not a clue how to add Node Red to my my EmonPi looked at the instructions but have no idea how to get to the prompt to type in the command to load it. Would it be possible to put a quick idiots guide to putting all of this great functionality onto a EmonPi latest software version.

John

For this you do not need to install Node Red, it just uses Python3.

How comfortable are you with SSH and the command line?

Reasonable, but did try to connect via SSH but it came back with connection refused so not sure what I was doing wrong or what I should do if it did let me in

John

@borpin

Per yr request in another thread – here’s my variant on Trystan’s great Python script to download historic Octopus Agile 30 mins consumption data. It now includes CUM consumption.

octopus_consumption_TL+JB.zip (1.8 KB)

I’ve removed personal info.

And you’ll probably want to prune my verbose comments – just memory joggers for me.

If you’ve already downloaded historic data, running this new script will not re-download it. So temporarily change the 0 on line 37 to 10000 to work around this problem.

Setting up FEEDs is a bit of a hassle – but you don’t need to do that just to try out this script – just run it under Python3 in an SSH terminal.

These screenshots illustrate the end result …

1 Like

If to the emonbase, what user/password combo did you use?

@borpin

OOPS!
My revised script resets the CUM each day as the previous day’s history is added.
And that kinda defeats the object
I’m on the case …

I’d had a look and was wondering. You need the last data value out of the feed as your starting point!

https://YOURIP/feed/value.json?id=1

@borpin

I fixed the problem with the cum starting point yesterday - both for first run and subsequent runs.
And yesterday with two fresh empty feeds, it ran OK.
And the daily cronjob ran successfully this AM - the previous cum advanced - so good to go …

octopus_consumption_TL+JB_v2.zip (1.8 KB)

Recommendation - if you’ve previously run Trystan’s original script and only have daily data, it’s best to create two fresh empty feeds before running this modified script.

1 Like

managed to get in to SSH now (was not enabled) Any help with what to do now. I suspect I need to copy the script somewhere first (onto the EmonPi), not a clue how to do this I can then see that you run the command to set it to run at the prompt. Sorry for being so daft :frowning:

John

@TrystanLea, can you drop this into UsefulScripts please?

Done: usefulscripts/octopus at master · emoncms/usefulscripts · GitHub

I’ve added this in

I havent added this in yet.

I use the Octopus Agile emoncms app to do this.

Yes