Octopus Agile emoncms app

[BigJacko (Neil Jackson)]

@TrystanLea

have a new version of the demandshaper on its way, with a more modular structure, it would be best to use that for any new integration.

Will check it out. Thanks for the tip. Presumably I can just git-clone that locally in place of my existing DemandShaper, and not screw up my emoncms updates in the future (sorry, haven’t played with bespoke components in emoncms yet, so I’m being ultra-safe/paranoid, LOL).

@TrystanLea

Let me know how you get on with the control side of things with NodeRed.

Will do, most definitely. If I can get something working, it’d be nice to contribute back to the project that’s been so helpful to me over the last year or so.

Sorry Tale...

(long story… but gist is previous-previous energy supplier bankrupt, OFGEM replacement supplier of last-resort issued no bills, Ombudsman involved, case won largely thanks to me having OEM/emoncms proofs, allowed to escape, and so switched to Octopus, commence bliss and restored faith in humanity. Phew!)

HomeAssistant? This one? https://www.home-assistant.io/ Joy Another new platform to learn! LOL. Any particular reason you chose this one, @borpin ? Just curious if there’s an issue with Node-RED that makes it less reliable, a pain, etc. Cheers.

[borpin (Brian Orpin)]

@BigJacko

Just curious if there’s an issue with Node-RED that makes it less reliable, a pain, etc. Cheers.

I just try and keep all my scheduling on one platform. I use NR for processing data feeds mainly.

[Bramco (Simon)]

There are plenty of ways to do it in NR to save adding yet another environment.

Take a look at Pete Scargill’s Big Timer for instance. It’s been around for a long time and would probably do what you need.

Simon

[BigJacko (Neil Jackson)]

Will do. TBH I’ve already had to rule out Home Assistant because it’s a replacement OS for the Pi, and I’m already committed to another.
Thanks for the suggestion, Simon.

One quick question though @TrystanLea - I don’t seem to have the choice to load in my Octopus API data. Does this mean I’m not on the right app version? I’m guessing it’s not part of the public emoncms code yet, correct? What’s the best process to install it, on a standalone Pi running emon SD? Cheers, Neil

[borpin (Brian Orpin)]

@BigJacko

I’ve already had to rule out Home Assistant because it’s a replacement OS for the Pi

I have come to the decision that running everything on RPis is overrated IMHO. I have boosted the memory on an old laptop, installed Proxmox and run everything in containers/VMs on that now (well about to migrate my HA install to it).

Advantages

• Reuse old kit
• Builtin UPS
• Automated backups (whole system not just data)
• No SD Card failures
• Add a second laptop and generate a cluster for resilience
• Rapid testing
• Separate out different components of your system (HA, NR, MQTT, Pi-Hole, Emoncms, Unifi Controller).

Disadvantages

• None yet

[TrystanLea]

@BigJacko

What’s the best process to install it, on a standalone Pi running emon SD

It’s a standalone script at the moment to pull in the data, here’s the forum post that describes how to use it Python script to import Octopus Agile consumption data into emoncms

@borpin

I have boosted the memory on an old laptop, installed Proxmox and run everything in containers/VMs on that now

That sounds great, interesting approach!

[Zarch (Mick Wall)]

Just to reiterate @borpin point about Proxmox.

Almost same here, at first just reused an old desktop (i3 / 8gb ram) for Proxmox, now using an old HP Microserver.

Has grafana, node-red, home assistant, influx, octolux etc etc all running as separate VMs, and/or as docker containers.
Gives you much finer grain control of the individual components if you wish to upgrade/shutdown etc/backup.

Well worth looking into.

[BigJacko (Neil Jackson)]

@TrystanLea

@BigJacko

What’s the best process to install it, on a standalone Pi running emon SD

It’s a standalone script at the moment to pull in the data, here’s the forum post that describes how to use it Python script to import Octopus Agile consumption data into emoncms

Ah, sorry, wires crossed I think, @TrystanLea. I have a feed (based on @borpin’s NodeRed solution) which has all my historic meter-reads under ‘consumption’ and timestamped according to the half-hour periods in question.

What I don’t have is the part in my version of your EmonCMS Octopus Agile App, which matches this:

@TrystanLea

If the checkbox is ticked at the bottom of the page, it will calculate the total consumption and total costs based on the smart meter data where available.

I see no checkbox, ergo, I think I have an older version. It doesn’t appear to be part of the current EmonCMS core (i.e. that App never updated when I tried updating EmonCMS a couple of days ago), so I’m guessing it’s only available via your git page? If that’s the case, are there any special steps I need to perform in order to safely patch it in, without risking messing up future ‘normal’ EmonCMS updates? Or is it just a git-fetch and job done?

Cheers.

Also, @borpin and @Zarch - thanks for the idea of virtualisation and running HA on a bit of old (non-RPi) kit. I hadn’t thought of that. Virtualisation is still not in my ‘go-to’ mindset yet, but I’m slowly changing! LOL. I’ll give that a look… I have too much old kit laying about, that might lend itself to this…

[borpin (Brian Orpin)]

@Zarch

now using an old HP Microserver.

I was using one of those until I put a power meter on it and discovered how much power it used compared to the laptop.

@Zarch

all running as separate VMs, and/or as docker containers.

For me, most of my ‘parts’ are running in native PVE lxc containers built off the Ubuntu template rather than docker in a VM. AIUI, these are the equivalent of running docker on an OS.

The only VM is the HA OS one.

@BigJacko

Virtualisation is still not in my ‘go-to’ mindset yet

I had run VMs in the free version of ESXi for a number of years on the HP Microserver, but backup was a real pain. Proxmox is just so easy in comparison - well worth a dabble.

[TrystanLea]

@BigJacko

App never updated when I tried updating EmonCMS a couple of days ago

It should just come in with an update. It’s in the stable branch. Perhaps try running update again and doing a full page refresh on the app page?

[Zarch (Mick Wall)]

From messing with various disk drives, if you have a SSD knocking about or indeed can stretch £25 for a low end 256GB, i’d highly recommend you install Proxmox (and VMs) on an SSD.

Takes away any nagging doubts from the outset when you get hung up on performance later.

[BigJacko (Neil Jackson)]

@TrystanLea

@BigJacko

App never updated when I tried updating EmonCMS a couple of days ago

It should just come in with an update. It’s in the stable branch. Perhaps try running update again and doing a full page refresh on the app page?

Ah. Thanks for the hint, @TrystanLea - as a result, I think I’ve sussed it:

Updating /var/www/emoncms/Modules/app module 
------------------------------------------ 
- git branch: stable 
- git tags: 2.1.1-42-ge5d51a4 
WARNING local changes in /var/www/emoncms/Modules/app - Module not updated 
- git status: 
 
On branch stable 
Your branch is up to date with 'origin/stable'. 
 
Changes not staged for commit: 
  (use "git add ..." to update what will be committed) 
  (use "git checkout -- ..." to discard changes in working directory) 
 
	modified:   apps/OpenEnergyMonitor/costcomparison/rates.js 
 
no changes added to commit (use "git add" and/or "git commit -a") 

Some time ago, I added data to the tariffs dropdown for the Cost Comparison App, so that I had the current regional version of the Octopus Fixed tariff that I was on at the time. I’m guessing this has marked my git as unstaged/uncommitted, and all bets are off. Derp. I can probably ditch those changes now anyway, because I’m on Agile.

It does highlight that the process of adding personalised tariffs to the Cost Comparison App might need simplification in the future, if that app is to become really useful to all, but I’m guessing you knew that already and development time-availability is the issue?

[borpin (Brian Orpin)]

@Zarch

if you have a SSD knocking about or indeed can stretch £25 for a low end 256GB, i’d highly recommend you install Proxmox (and VMs) on an SSD

Yes agree. As it stands I’m just using the original one in the laptop. I have a 256 SSD in my desktop, but I want to upgrade that so I’ll swap them around at some time. However, backups go to my NAS drive and a connected USB drive so restoring should be fairly painless.

[BigJacko (Neil Jackson)]

Sorted. Now we’re cooking on gas. Or rather, we’re not.

Oh, and in case you (or anyone else) has any use for the following South Western area tariffs, @TrystanLea, I’ll leave them here for posterity. They were the changes to rates.js that was hanging up my auto-update.

energy_rates.push({
identifier: "OCTOPUSGOSOUTHWEST",
label: "Octopus Go EV Tariff",
supplier: "Octopus Energy",
region: "SOUTHWEST",
updated_epoc: 1591996360,
// Rate 0 = peak 14.29 pence
// Rate 1 = off-peak 5.00 pence
rates: [{
    cost: 0.1429,
    colour: colour1
},
{
    cost: 0.05,
    colour: colour2
}
],
// Daily Standing Charge  = 25.00 pence
dailystandingcharge: 0.25,
rate_bucket: OCTOPUSGO_SOUTHWEST
});

energy_rates.push({
identifier: "OCTOPUS12MFIXED2020",
label: "Octopus 12M Fixed 2020 Tariff",
supplier: "Octopus Energy",
region: "SOUTHWEST",
updated_epoc: 1591996360,
// Rate 0 = peak 15.29 pence
rates: [{
    cost: 0.1529,
    colour: colour1
}
],
// Daily Standing Charge  = 21.14 pence
dailystandingcharge: 0.2114,
rate_bucket: OCTOPUS12M_FIXED_2020
});

Thanks again for the help. Much appreciated.

[nanoamp (Nick Kaijaks)]

I’ve got Agile data importing as a feed, and can select it in the app for meter_kwh_hh. But I don’t see any sign that it’s working. No historic bars are dark blue, and there’s no calibration data in the console. I saw in another thread that smart meter readings won’t currently coexist with a solar PV feed. But I can’t get the app to ignore my solar feed! It defaults to auto, and even if I set the dropdown back to Select solar_kwh feed and hit OK, it just ignores it and uses it anyway. Any suggestions on how to persuade it to stop?

[BigJacko (Neil Jackson)]

Have you also ticked the little checkbox below the graph on the Octopus Agile App page (not the settings, the display itself)?

Have you confirmed that you do actually have valid data for previous days, in your Agile data feed? i.e. if you go to the Graphs area, and select that Agile data (whatever you’ve called it), and use a data-interval of 1800 (half-hour), and a bar-chart style, do you see bars for each half-hour of the previous day(s)?

I’m not sure about the Solar PV issues myself (don’t have it, sorry)… but best to eliminate the obvious items first anyway, I think.

[nanoamp (Nick Kaijaks)]

Yes to both. In the CSV view I see data points from July up until 00.30 today.

[ian]

Creating a new copy of the app was how I got rid of the solar feed.

[nanoamp (Nick Kaijaks)]

Hmm, I’ve tried that, but it auto-selects the solar feed on creating a fresh one, and I still can’t find a way to un-select it.

[ian]

Just realised I don’t use standard feed names so auto does nothing. Other than changing feed names which is a bit drastic I think you may be stumped.

I just thought. Take a zero input that never changes. Create a feed and try selecting that feed as your solar input.