Octopus Agile emoncms app

Emoncms
106

Hello Trystan,

This is wonderful, thank you. I have recently switched over to the Octopus Agile tariff, and now I am looking at ways to update my data logging in emoncms.

I like what you have done with the Agile “App” in emoncms, but I would love to be able to create an ordinary “feed” in emoncms containing the half-hourly agile tariff prices. I may be mistaken because I’m still coming up to speed with this, but I don’t think that your Agile app creates a feed with the data. Am I mistaken?

Is there an easy way to do this please? I am assuming that the code base must already exist because you have created precisely that as part of the Agile app. Failing that I shall have to improve my dodgy python skills and try to code something myself. I shall need to get my head around submitting data to emoncms using a specific timestamp.

The ultimate goal is to combine the rates feed with my smart meter usage feed, in a emoncms dashboard graph. Thankfully, I see that Stuart has already conquered the usage part of the puzzle, and I shall be exploring his python code in due course. GitHub - stuartpittaway/emoncmsOctopusSmartMeter: SmartMeter API integration with Octopus Energy (UK) and emoncms

Any help gratefully received please.

Thanks,
Rich

107

That is correct, the app pulls in the agile data for the particular view that you are looking at but does not record it to a local feed.

Stuarts work looks great, does it pull in the tariff data as well?

108

OK, thanks Trystan.

I guess I shall have to try and code something, but my Python is amateur at best!

Stuart’s work is only fetching the consumption data from what I can see, which is shame. I have been doing a bit of playing today with some of the Agile data, and I would love to represent the half-hourly values in a bar chart like you have done with the Agile app. I can’t work out how to do that in a dashboard graph, because it always represents it as a line graph with the values smoothed from one half-hourly slot to another. Is there something I am missing please? What style of dashboard visualization should I use to get the same imagery as your Agile app please?

Kind regards,
Rich

109

Hello Rich, I assume you are referring to the stepped look? e.g:

image

Again, Im afraid, that one is just available via the app at the moment.

110

Yes, that’s the look I was after. Oh well never mind; I’ll see what I can create in the existing dashboard.

Thank you,
Rich

111

I’ve added support for integrating Octopus smart meter consumption data where available in the app *. If the smart meter data is available it will show the half hours as shaded darker blue, otherwise it shows the half hourly consumption from the main feed, recorded via the emonTx/emonPi or other hardware that you may be using.

If the checkbox is ticked at the bottom of the page, it will calculate the total consumption and total costs based on the smart meter data where available.

Now that Im on the tariff (since just over a week!) I wanted to compare the smart meter data to my emontx data and to be able to see the smart meter data in the app. It works well to combine both as you have to wait until the day after to see the Octopus smart meter data and it may not always come through see Aug 24 and Aug 25. The realtime data from the EmonTx fills in the gaps nicely.

image
image1383×989 84.9 KB

Importing Octopus smart meter data to emoncms:
The above assumes the consumption data is pulled separately from the Octopus API and is available as a emoncms feed for the app to use. See importing Octopus smart meter consumption data, either via a node-red flow thanks to @borpin: Import Octopus consumption data or my python script Python script to import Octopus Agile consumption data into emoncms

Comparison, error calculation and calibration
The percentage error between the smart meter data and the emonTx/emonPi feed values is printed to the browser console, alongside suggested calibration parameters.

image

It turns out that in my case my un-calibrated EmonTx is in error by 3% above the smart meter data. Though the calibration suggestion suggests applying a scaling calibration equivalent to 2% alongside a small offset calibration.

The calibration parameters are based on a line of best fit between the emonTx/emonPi data on the x-axis and smart meter data on the y-axis. I need to think through how to apply the intercept to the incoming power data, it should be applied as an offset but needs to be translated from units of kWh to Watts.

The above can all be pulled in on an emonPi/emonBase by running the Emoncms Updater via the Admin panel. I’ve pushed the changes to the app module stable branch and updated emoncms core to include the bulk feed data upload API in stable as well.

3 Likes
112

Looking forward to trying this because I noticed a similar discrepancy (feed kWh greater than billed amount). Suspect some of it comes from rounding, which makes more difference in summer when the amount in most half-hours is small.

1 Like
113

@TrystanLea Hi - I just wanted to say a stonking great thank-you for your work with this (and the whole emonCMS project team generally, tbf).

I finally got switched on to Agile on Friday night/Saturday morning (Sep 11/12) - after having had my SMETS2 meter installed on 8th July! Yeah, we’ve had a few issues - mostly all to do with AES, the installer. Octopus have been golden, tbh.

Anyway - the reason I’m posting this thanks, is because your Agile app gave me the ability to ‘backtest’ my past year-or-so’s meter monitoring, against the equivalent I would’ve paid, if I’d been on Agile then instead.

Obviously I didn’t check every single day, but I checked plenty, and on every single one, the Agile tariff would’ve been way, way cheaper than the Fixed (15.29p/kWh) tariff I was on at Octopus. I had always thought this would be the case, but it was excellent to actually have it confirmed indisputably! I had always been secretly a bit worried that Agile was a gimmick - and that maybe the peak usage band would offset savings throughout the rest of the day. As it turned out, with my usage pattern (high, home all day/night, many computers, homeworking family), Agile was a total no-brainer.

So, thanks to you, we got the SMETS2 (eventually) and waited for it to be commissioned. And waited. And waited… And waited… LOL. And of course we shot through the 14-day expected commissioning time, and, well, that was nearly three months ago now.

Thing is, I’m now in a position to go back to Octopus and show them my own ‘meter readings’ (I have a homebrew pulse-counter hooked up to a Pi running EmonCMS) on, if needs be, a per-minute watt reading. More usefully, I have been able to simply screenshot the days since that 14-day SMETS2-commissioning expectation, and show them the true ‘opportunity cost’ of my not being able to get Agile up and running, due to the meter still being SNAFU. So far, we’re running to about a £57 difference between what I’m paying on ‘Fixed’ vs what I would’ve paid on Agile, if they (or rather, AES) had gotten my meter commissioned on time.

Whether I’ll actually get that back or not, we shall see… but it’s certainly fantastic to be able to (fairly easily) see exactly what’s been going on, and what’s been missed out on! So thanks for that!

Going forward, I’m hoping things will be a tad more stable (though not necessarily simpler), and I can start integrating my Zappi2 with EmonCMS and DemandShaper (which I have not looked at yet), and get that all ‘Agile-Aware’.

But I would not have been on this road at all, had it not been for EmonCMS, and especially, the Agile App you built, so again, thank you so much. It’s been a great help, and a real inspiration. :smiley:

2 Likes
114

Thanks a lot @BigJacko much appreciated! That makes it all worthwhile!

Does the Zappi2 have a way to interface with it, e.g to send a timer command or on/off state change? We’d likely need to write an interface to get the DemandShaper to talk to it.
Looks like there may be something to work with here GitHub - twonk/MyEnergi-App-Api: Investigation of the MyEnergi App…

1 Like
115

On it right now, @TrystanLea :smiley:

It’s a bit of a grind - the Zappi API is, er, a tad sketchy…but I’m slowly bending it to my will in Node-RED. I’m able to interrogate the Zappi for its immediate condition, and I’m currently attacking the whole ‘get historical per-minute data’ from MyEnergi’s servers (IMHO it would be so much smarter if that could be localised, but hey, their choice, their servers to pay for! LOL). Once I’ve got that, a feed into EmonCMS is only minutes away (fingers crossed).

Then, yes, Demandshaper looks like the next step forward, but I confess at the moment I know absolutely nothing about its finer points. But, having said that, I was in a similar situation with Node-RED before this Sunday, so given a following wind and a couple of days headscratching, I’ll probably work something out.

As far as I can tell, yes, absolutely the Zappi can be instructed when to ‘boost’ (their vernacular for ‘suck from grid, rather than local solar or battery’). I believe it has a number of boost slots that can be allocated, so presumably it can receive a number of candidate windows in one hit, perhaps on a daily basis, once the Agile tariff has been loaded in, munged for ‘best slots’ and then pumped out to the Zappi. Rinse-repeat each day, is my guess… But we’ll see…

116

Great to hear. It has been suggested to make it possible to control a NodeRed flow from the DemandShaper, that may be the easiest route.

I have a new version of the demandshaper on its way, with a more modular structure, it would be best to use that for any new integration. https://github.com/emoncms/demandshaper/tree/modular_forecasts

Let me know how you get on with the control side of things with NodeRed. I think you can do the scheduling from NodeRed itself as well, which may be easier. @borpin may be able to give pointers.

1 Like
117

All my scheduling is done via HomeAssistant not NR.

1 Like
118

Will check it out. Thanks for the tip. Presumably I can just git-clone that locally in place of my existing DemandShaper, and not screw up my emoncms updates in the future (sorry, haven’t played with bespoke components in emoncms yet, so I’m being ultra-safe/paranoid, LOL).

Will do, most definitely. If I can get something working, it’d be nice to contribute back to the project that’s been so helpful to me over the last year or so.

Sorry Tale...

(long story… but gist is previous-previous energy supplier bankrupt, OFGEM replacement supplier of last-resort issued no bills, Ombudsman involved, case won largely thanks to me having OEM/emoncms proofs, allowed to escape, and so switched to Octopus, commence bliss and restored faith in humanity. Phew!)

HomeAssistant? This one? https://www.home-assistant.io/ Joy :slight_smile: Another new platform to learn! LOL. Any particular reason you chose this one, @borpin ? Just curious if there’s an issue with Node-RED that makes it less reliable, a pain, etc. Cheers.

119

I just try and keep all my scheduling on one platform. I use NR for processing data feeds mainly.

1 Like
120

There are plenty of ways to do it in NR to save adding yet another environment.

Take a look at Pete Scargill’s Big Timer for instance. It’s been around for a long time and would probably do what you need.

Simon

1 Like
121

Will do. TBH I’ve already had to rule out Home Assistant because it’s a replacement OS for the Pi, and I’m already committed to another.
Thanks for the suggestion, Simon.

One quick question though @TrystanLea - I don’t seem to have the choice to load in my Octopus API data. Does this mean I’m not on the right app version? I’m guessing it’s not part of the public emoncms code yet, correct? What’s the best process to install it, on a standalone Pi running emon SD? Cheers, Neil

122

I have come to the decision that running everything on RPis is overrated IMHO. I have boosted the memory on an old laptop, installed Proxmox and run everything in containers/VMs on that now (well about to migrate my HA install to it).

Advantages

  • Reuse old kit
  • Builtin UPS
  • Automated backups (whole system not just data)
  • No SD Card failures
  • Add a second laptop and generate a cluster for resilience
  • Rapid testing
  • Separate out different components of your system (HA, NR, MQTT, Pi-Hole, Emoncms, Unifi Controller).

Disadvantages

  • None yet :slight_smile:
1 Like
123

It’s a standalone script at the moment to pull in the data, here’s the forum post that describes how to use it Python script to import Octopus Agile consumption data into emoncms

That sounds great, interesting approach!

124

Just to reiterate @borpin point about Proxmox.

Almost same here, at first just reused an old desktop (i3 / 8gb ram) for Proxmox, now using an old HP Microserver.

Has grafana, node-red, home assistant, influx, octolux etc etc all running as separate VMs, and/or as docker containers.
Gives you much finer grain control of the individual components if you wish to upgrade/shutdown etc/backup.

Well worth looking into.

1 Like
125

Ah, sorry, wires crossed I think, @TrystanLea. I have a feed (based on @borpin’s NodeRed solution) which has all my historic meter-reads under ‘consumption’ and timestamped according to the half-hour periods in question.

What I don’t have is the part in my version of your EmonCMS Octopus Agile App, which matches this:

I see no checkbox, ergo, I think I have an older version. It doesn’t appear to be part of the current EmonCMS core (i.e. that App never updated when I tried updating EmonCMS a couple of days ago), so I’m guessing it’s only available via your git page? If that’s the case, are there any special steps I need to perform in order to safely patch it in, without risking messing up future ‘normal’ EmonCMS updates? Or is it just a git-fetch and job done?

Cheers.

Also, @borpin and @Zarch - thanks for the idea of virtualisation and running HA on a bit of old (non-RPi) kit. I hadn’t thought of that. Virtualisation is still not in my ‘go-to’ mindset yet, but I’m slowly changing! LOL. I’ll give that a look… I have too much old kit laying about, that might lend itself to this…