Emoncms Demand Shaper module

Hi guys, I’m trying to get the demand shaper working with a SonOff S20 loaded with the emonESP timer branch.

The smartplug is not being discovered. How could we go about debugging this?

On a separate note, I’ve also tested loading the S20 with tasmota 6.6.0 and am able to publish mqtt to the emonbase.

Hi

Have you logged direct into the ESP in Access point mode and set the WiFi up first. If so you should be able to see the IP address allocated to the ESP on your network. You then access the ESP at the allocated address. From there you can set up MQTT. There are a few issues with the MQTT defaults and you may need to complete broker address, port, credentials and password and hit save. I don’t know what they are as I have a custom broker. Once you hit save the ESP shows MQTT connection status. Until you are connected nothing will happen. I have found it sometimes takes quite a while to connect and I re-enter the broker IP address and hit save several times to get it to connect.

Thanks. All good now got it working. Had to do a bit of waiting and refresh the module page.

I’ve been doing quite a bit of work on the demandshaper module over recent weeks as part of work on the EnergyLocal Bethesda Trial. I’ve created a new release v1.1:
Release v1.1 · emoncms/demandshaper · GitHub

Changelog

  • Modularisation and refactor of scheduler function, seperated out the forecast processing from the smart and timer schedule generation.
  • Matching javascript implementation of php scheduler function.
  • Improvements to schedule object: fixed schedule settings and always changing runtime settings are now seperated providing a better structure.
  • Fixed schedule settings only are saved to disk in order to reduce SD card wear
  • Forecast is preloaded in js client and schedules calculated using js scheduler to reduce bandwidth requirements and speed up UI.
  • Fixed DST timezone support
  • Higher resolution scheduling support to 15mins
  • Basic demandshaper schedule log to /var/log/emoncms/demandshaper.log
  • Improvements to EnergyLocal forecast
  • Missing ip address fix
  • Fixed standard timer implementation (timer1 and timer2 now work)
  • Confirmation of whether settings have transferred to Smartplug/OpenEVSE/Heatpump controller.
  • Emoncms V10 menu support.

Important: Due to substantial differences in settings object structure, after updating to version 1.1 the demandshaper schedules database entries need resetting.
This can be done by going to the URL:

http://emonpi/emoncms/demandshaper/clearall

Timezones: The module currently has the Europe/London timezone hardcoded, making this configurable is on the development list for a later version, most of the forecasts are currently UK based as it is, so its likely of limited use elsewhere for now. The EmonESP firmware uses UTC time and the OpenEVSE firmware the local browser time. The demandshaper module applies a correction based on device type.

Schedule log: It can be useful when testing to see what schedules have been generated and when they are recalculated. This information is now logged to /var/log/emoncms/demandshaper.log if this file exists. To create this file the steps are:

sudo touch /var/log/emoncms/demandshaper.log
sudo chmod 666 /var/log/emoncms/demandshaper.log

This log can be accessed from an api:

http://emonpi/emoncms/demandshaper/log                         (all)
http://emonpi/emoncms/demandshaper/log?filter=openevse         (filter by device)
http://emonpi/emoncms/demandshaper/log?filter=openevse&last    (show only last schedule)

Stability & Testing
I’m not confident yet that this development is stable, it generally appears to be working well on my system and the DST fixes and state confirmation make it a substantial improvement over earlier versions, there are times where the schedule does not appear to get written correctly to the smartplug and the interface responds with a ‘settings mismatch’ error, re-sending/setting a new schedule usually clears this issue.

Im away from the office from July 16th (Tuesday) until the 31st so will only be able to give limited support in that period. Thanks :slight_smile:

1 Like

Here’s a screenshot of the OpenEVSE interface, showing an improved 24h view of the agile octopus forecast, it wraps the forecast around if the forecast does not extend far enough, the schedule gets recalculated when a new forecast is available ensuring the schedule is based on the most up to date forecast:

1 Like

The latest version of the demandshaper module now supports all Octopus Agile regions, selectable from a dropdown:

2 Likes

Did this ever get completed?

Yes, see last post :wink:

I’ve added a brief guide on setting up the demand shaper for EV charging to the OpenEnergyMonitor guide Emoncms Demand Shaper Module (beta) - Guide | OpenEnergyMonitor Im still working on the system diagram and will extend with a section discussing in more detail the ‘beta’ state of this development and intention for further development, e.g support for different vehicle API’s.

I’ve fixed a number of issues with the demandshaper module, now available on new image via standard update process, or via git if installed manually.

  • Fixed missing carbon intensity forecast and associated undefined error.
  • Fixed missing ‘Saved’ message on OpenEVSE schedule update
  • Fixed OpenEVSE tariff info to now show p/mile
  • Fixed an issue with timezone handling when schedules are near midnight, hour rollover/under issue.
  • Fixed an issue relating to Setting mismatch error where start of schedule is in the past.
  • Add device wizard hides automatically when first device appears
  • UDP broadcast related setting was missing in emoncms core, now added in, results in much faster setup for smart plugs and wifi relay module.

Version is now v1.1.0: GitHub - emoncms/demandshaper: Appliance, Smartplug, OpenEVSE demand scheduler: Find the best time to run household loads

1 Like

Hi Trystan
I’ve been playing with Demandshaper - the problem I have is getting the present state of charge from the car and setting it in Demandshaper. is the ‘20%’ value hard coded? If so could you make it setable via the UI??
My Leaf (40kWh version) apparently doesn’t yet work with the OVMS so can’t get Demandshaper to talk to the car. However - there seems to be an app called ‘MyLeaf’ on Android that which someone has clearly reverse engineered to get the SoC and other things from the Nissan. wondering how easy it would be to grab the SoC from the Demandshaper script?! I searched github and came across GitHub - glynhudson/leaf-python-mqtt: Extract data from Nissan Leaf API and post to mqtt ( hi @glyn.hudson !) so perhaps this could be added to Demandshaper to determine how long to charge for. I’m not really a coder but happy to play and test, given suitable instructions. I ran into issues installing it on my emon pi (IOError: [Errno 28] No space left on device) so haven’t played yet. Will try on a spare Linux box in a bit…

BR
Tim

1 Like

Hello @gr0mit great to hear your testing the module. Yes wider integration with different API’s would be great. I need to think through how to make this more modular and document a process for developing/attaching a suitable script here.

In the meantime here are a couple of pointers to where things are happening in the code:

The default 20% value is set here:
https://github.com/emoncms/demandshaper/blob/master/demandshaper-module/general.js#L53

This is the line where the front-end interface pulls in the OVMS info
https://github.com/emoncms/demandshaper/blob/master/demandshaper-module/general.js#L131

This is the API end point on the server side for the OVMS info request:
demandshaper/demandshaper_controller.php at master · emoncms/demandshaper · GitHub

To add another service we would need to add an API end point in the demandshaper_controller and then make a way of selecting between services in the front end…

I think it might be possible to make the start SOC selectable in the short term and perhaps provide an enter kWh to charger option, or / and an option to specify say x number of miles to add in the charge?

Hi @TrystanLea

I think it might be possible to make the start SOC selectable in the short term and perhaps provide an >enter kWh to charger option, or / and an option to specify say x number of miles to add in the charge?i

I think that would be a great plan in the short term. I’m not really a programmer (other than copy/paste of other folks code!) but to be able to set the SoC (which I can get from the Leaf app) and either kWh or miles/km to add to the car, it would be cool.

I suppose ideally this could be the first API endpoint from which the Demandshaper reads the values. Then other API endpoints can be added more easily. Just a thought.

1 Like

I have couple of loosely related questions. I just download the new emoncms image to a 32GB card and it working great - thank you.

The Octopus Agile ‘Current Price’ seems to be reading the maximum price rather than the price at the current time ? (Sorry I didn’t realise the posts here Octopus Agile emoncms app)

Can the ‘Demand Shaper’ output to any MQTT device ?, if not is a .bin available of the emonESP - that makes life a lot easier to re-flash from Tasmota rather than remove it from where it is and take it apart to re-flash it !

Dave.

@TrystanLea - Great work on the demand shaper. This is going to be what we need. I am keen to help and contribute.

I have started looking at the code because I am suffering from death by “settings mismatch” with a Sonos S20 Smart Plug. I think this problem started when you implemented the timer-based scheduling command rather than the simple on/off state (but I agree that this is definitely the way to go). I have been testing to try to identify the circumstances when “settings mismatch” is raised:

  • If the demand shaper is set with “ok to interrupt” and it then calculates that more than four periods are required, on the same calendar day, you always get a “settings mismatch” error with a Sonos S20 smartplug.

  • If the demand shaper calculates a period which has already started , you always get a “settings mismatch” error with a Sonos S20 smartplug.

  • If the demand shaper calculates a period which will start in less than 30 minutes time , you always get a “settings mismatch” error with a Sonos S20 smartplug.

I have had no issues flashing Sonos S20 smartplugs and connecting them to the Demand Shaper Module. Very easy to setup and connect. But… I am having no luck connecting a new, single relay MQTT to the demand shaper. The relay publishes to whatever topic I give it and you can see it in a feed… but I can’t get it to show in the demand shaper module.

I must be missing a step. I configured the MQTT settings (ref. screenshot), but do I need to configure anything on the demandshaper side so that there is something to subscribe to?

image

@TrystanLea @gr0mit

Those links to the code you provided looks like it’s reading the SoC from OVMSServers, putting that in emonCMS, and then the demand shaper reads that? Have I understood that correctly?

Would it be feasible to have an option to read the current SoC via an MQTT topic? That would allow it to be posted to a local server in a standard way.

Edit: Just read the rest of the thread and it looks like there’s a few of us asking similar things.

Thanks for testing @kylelamb

Have you tried the latest version of the module? I fixed quite a few of these in the last update. In particular the period that has already started one, and there where a couple resulting from timezone and midnight related issues.

Aha, sorry, you need to flash the EmonESP timer branch firmware to the relay module, same firmware as used for the smartplugs. Just change the node_type and node_description settings to be ‘wifirelay’ before compilation and upload, see this line: https://github.com/openenergymonitor/EmonESP/blob/timer/src/src.ino#L69

Yes, although ‘putting that in emonCMS’ is maybe more clearly understood as also happening within the demandshaper module. There’s a server and client component to the demandshaper module, the reading of the OVMS happens on the server component and then that’s read in to the client…

That’s not a bad idea, ideally we need to request the SoC when we need it rather than periodic polling that may be waking the car computer up each time, so we still need a way for the demandshaper module to trigger a request for the SoC…