Emoncms Demand Shaper module

I’ve tried sending changed vehicle SoC at various points in the cycle from off / smart.

I’ve tried changing to timer rather than off to kick the schedule.

I agree with you though, no orange bars, no charge.

Hopefully Tristan can provide some insight.

Thanks

Hello @johnbanks

Sorry for the delay, thanks for persisting with this.

I think what might be happening here is that the demandshaper taken note of the amount of time left to charge on a schedule. After completing a schedule this will be zero and unless it is reset the new schedule for the next day wont be created.

Does it show ‘0 mins left to run’ in the interface when there are no schedule bars shown?

Im going to try adding another step to the script, will post an amended version shortly.

I was referring to the target SOC in that thread, but the same applies to the EV SOC.

Ok the following should work. Looks like I needed to add the calculation of the timeleft into the script manually as its not possible as I had originally thought to trigger this in the background demandshaper script. The demandshaper UI usually does this step

import requests, json

emoncms_apikey = ""
emoncms_server = "http://emonpi.local"

reply = requests.get(emoncms_server+"/demandshaper/get", params={'device':'openevse','apikey':emoncms_apikey})
ds_data = json.loads(reply.text)

ds_data['schedule']['settings']['target_soc'] = 0.4

kwh_required = (ds_data['schedule']['settings']['target_soc']-ds_data['schedule']['settings']['current_soc'])*ds_data['schedule']['settings']['battery_capacity']
ds_data['schedule']['settings']['period'] = kwh_required/ds_data['schedule']['settings']['charge_rate']

if ds_data['schedule']['settings']['balpercentage'] < ds_data['schedule']['settings']['target_soc']:
    ds_data['schedule']['settings']['period'] += ds_data['schedule']['settings']['baltime']
ds_data['schedule']['runtime']['timeleft'] = ds_data['schedule']['settings']['period'] * 3600;

reply = requests.post(emoncms_server+"/demandshaper/save", params={'apikey':emoncms_apikey}, data={'schedule':json.dumps(ds_data['schedule'])})

print(reply.text)

@TrystanLea

You are correct.
The EV did a SMART charge last night and this am, SMART was highlighted but there were no schedule bars,.
Here’s the json dump for this am.
Scroll to the extreme right and you’ll see … ‘timeleft’: 0}}}

{'schedule': {'settings': {'device': 'openevse', 'device_type': 'openevse', 'device_name': 'openevse', 'ctrlmode': 'smart', 'period': 0.5434782608695657, 'end': 8, 'end_timestamp': 1610611200, 'interruptible': 0, 'runone': 0, 'on_completion': 'smart', 'forecast_config': [{'name': 'octopusagile', 'weight': 1, 'gsp_id': 'H'}], 'forecast_units': 'generic', 'timer': [{'start': 0, 'end': 0}, {'start': 0, 'end': 0}], 'ip': '', 'soc_source': 'input', 'battery_capacity': 75, 'charge_rate': 6.9, 'target_soc': 0.9, 'current_soc': 0.85, 'balpercentage': 0.9, 'baltime': 2, 'car_economy': 3.4, 'charge_energy': 3.75, 'charge_distance': 12.8, 'distance_units': 'miles', 'ovms_vehicleid': '', 'ovms_carpass': '', 'divert_mode': 1}, 'runtime': {'started': True, 'periods': [], 'timeleft': 0}}}

For what it’s worth …
Inspecting the element for the buttons seems to indicate that the timeleft will be changed if it’s a different button being pressed:

function() {
  var last_ctrlmode = schedule.settings.ctrlmode
  schedule.settings.ctrlmode = $(this).attr("mode");
  if (schedule.settings.ctrlmode != last_ctrlmode) {
    schedule.runtime.timeleft = schedule.settings.period * 3600;
  }
  on_UI_change();
}

As schedule.settings.period is available from the json dump, I was about to try calclating a new timeleft (*3600) and saving that to the json.
Then restarting demandshaper.service … just to see what happens.

But I very much look forward to your solution - thx

Great, thanks @johnbanks, I’ve posted my solution above :slight_smile:

@TrystanLea

The EV charged overnight and, as usual this am, the Demand Shaper webpage showed SMART highlighted but no schedule bars for a future charge.

I then ran a test script incorporating yr suggested lines for which many thx:


import requests, json, os, time

# Emoncms server configuration ...
emoncms_apikey = "xxxxxxxxxxxxxxxxxx"   # use appropriate key
emoncms_server = "http://127.0.0.1"

# Get the Demand Shaper data ...

params = (
    ('device', 'openevse'),
    ('apikey', emoncms_apikey)
)

reply = requests.get(emoncms_server+"/demandshaper/get", params=params)
jsonstr = reply.text.rstrip()
ds_data = json.loads(jsonstr)

print(reply.status_code)    # Debugging only
print (ds_data)             # Debugging only


# Extract relevant data from the json
# The following section for debugging only ...

data1 = ds_data['schedule']['settings']['ctrlmode']
data2 = ds_data['schedule']['settings']['on_completion']
data3 = ds_data['schedule']['settings']['target_soc']
data4 = ds_data['schedule']['settings']['divert_mode']
data5 = ds_data['schedule']['settings']['period']
data6 = ds_data['schedule']['runtime']['timeleft']

print (data1)
print (data2)  
print (data3)
print (data4)
print (data5)
print (data6)

# End of above debugging section

# Demand Shaper modes ...
# divert_mode = 1 = ECO mode ON
# divert_mode = 2 = ECO mode OFF
# Other modes: 'smart'  'off'  'on'  'timer'

## FIXED DEMAND SHAPER PARAMETERS HERE ...

ds_data['schedule']['settings']['ctrlmode'] = 'smart'
ds_data['schedule']['settings']['on_completion'] = 'smart'
ds_data['schedule']['settings']['divert_mode'] = 1         # ECO mode ON
ds_data['schedule']['settings']['battery_capacity'] = 75   # kWh ... depends on EV
ds_data['schedule']['settings']['charge_rate'] = 6.9       # A ... for a single phase openEVSE charger

# ===================================================================
## SEASON VARIABLE PARAMETER ...
# Dependent on the Season and the need to leave 'space' in the EV battery for next days ANTICIPATED charge from excess solar gen
# Max is 0.9 - recognises the limit set to conserve battery life
# WINTER season = 0.9
# MID season = ???
# POOL season = ???

ds_data['schedule']['settings']['target_soc'] = 0.9 

# ===================================================================

## CHANGES TO ENSURE SMART CHARGING DOES OCCUR - per @TrystanLea REF: https://community.openenergymonitor.org/t/emoncms-demand-shaper-module/9097/164
   
kwh_required = (ds_data['schedule']['settings']['target_soc']-ds_data['schedule']['settings']['current_soc'])*ds_data['schedule']['settings']['battery_capacity']
ds_data['schedule']['settings']['period'] = kwh_required/ds_data['schedule']['settings']['charge_rate']

if ds_data['schedule']['settings']['balpercentage'] < ds_data['schedule']['settings']['target_soc']:
    ds_data['schedule']['settings']['period'] += ds_data['schedule']['settings']['baltime']
ds_data['schedule']['runtime']['timeleft'] = ds_data['schedule']['settings']['period'] * 3600;

# ===================================================================

# Now save all the changed parameters ...
reply = requests.post(emoncms_server+"/demandshaper/save", params=params, data={'schedule':json.dumps(ds_data['schedule'])})

print (data1)    # Debugging only
print (data2)    # Debugging only  
print (data3)    # Debugging only
print (data4)    # Debugging only
print (data5)    # Debugging only    
print (data6)    # Debugging only

print (ds_data)              # Debugging only
print(reply.status_code)     # Debugging only
print(reply.text)            # Debugging only

The script ran without error with following result:

200
{'schedule': {'settings': {'device': 'openevse', 'device_type': 'openevse', 'device_name': 'openevse', 'ctrlmode': 'smart', 'period': 0.5434782608695657, 'end': 8, 'end_timestamp': 1610697600, 'interruptible': 0, 'runone': 0, 'on_completion': 'smart', 'forecast_config': [{'name': 'octopusagile', 'weight': 1, 'gsp_id': 'H'}], 'forecast_units': 'generic', 'timer': [{'start': 0, 'end': 0}, {'start': 0, 'end': 0}], 'ip': '', 'soc_source': 'input', 'battery_capacity': 75, 'charge_rate': 6.9, 'target_soc': 0.9, 'current_soc': 0.85, 'balpercentage': 0.9, 'baltime': 2, 'car_economy': 3.4, 'charge_energy': 3.75, 'charge_distance': 12.8, 'distance_units': 'miles', 'ovms_vehicleid': '', 'ovms_carpass': '', 'divert_mode': 1}, 'runtime': {'started': True, 'periods': [], 'timeleft': 0}}}
smart
smart
0.9
1
0.5434782608695657
0
smart
smart
0.9
1
0.5434782608695657
0
{'schedule': {'settings': {'device': 'openevse', 'device_type': 'openevse', 'device_name': 'openevse', 'ctrlmode': 'smart', 'period': 0.5434782608695657, 'end': 8, 'end_timestamp': 1610697600, 'interruptible': 0, 'runone': 0, 'on_completion': 'smart', 'forecast_config': [{'name': 'octopusagile', 'weight': 1, 'gsp_id': 'H'}], 'forecast_units': 'generic', 'timer': [{'start': 0, 'end': 0}, {'start': 0, 'end': 0}], 'ip': '', 'soc_source': 'input', 'battery_capacity': 75, 'charge_rate': 6.9, 'target_soc': 0.9, 'current_soc': 0.85, 'balpercentage': 0.9, 'baltime': 2, 'car_economy': 3.4, 'charge_energy': 3.75, 'charge_distance': 12.8, 'distance_units': 'miles', 'ovms_vehicleid': '', 'ovms_carpass': '', 'divert_mode': 1}, 'runtime': {'started': True, 'periods': [], 'timeleft': 1956.5217391304366}}}
200
{"success":true,"message":"Saved to redis only"}
pi@emonpi-node-15:~/ev-charging-scripts

timeleft is now showing in the json but still not periods

I restarted demandshaper.service to no avail. The schedule bars for the future charge were still not showing.
However per the screenshot, the Time left and future charge amount were showing – which is certainly on the way to a solution:

Finally I did a text search on file: main.js for periods, start & end:

Search for: periods, start & end
pi@emonpi-node-15:/opt/emoncms/modules/demandshaper/demandshaper-module/js $ grep -n 'periods\|start\|end' /opt/emoncms/modules/demandshaper/demandshaper-module/js/main.js
47:    // Convert hour into end timestamp
48:    schedule.settings.end_timestamp = hour_to_timestamp(schedule.settings.end)
54:            data: "config="+JSON.stringify(schedule.settings.forecast_config)+"&interruptible="+schedule.settings.interruptible+"&period="+schedule.runtime.timeleft+"&end="+schedule.settings.end_timestamp,
59:                schedule.runtime.periods = result;
60:                $("#schedule_json").html(JSON.stringify(schedule.runtime.periods));
69:            data: "config="+JSON.stringify(schedule.settings.timer)+"&forecast_start="+forecast.start+"&forecast_end="+forecast.end,
74:                schedule.runtime.periods = result;
75:                $("#schedule_json").html(JSON.stringify(schedule.runtime.periods));
80:        schedule.runtime.periods = [];
121:    $("#end input[type=time]").val(timestr(schedule.settings.end,false));
125:    $("#timer_start1 input[type=time]").val(timestr(schedule.settings.timer[0].start,false));
126:    $("#timer_stop1 input[type=time]").val(timestr(schedule.settings.timer[0].end,false));
127:    $("#timer_start2 input[type=time]").val(timestr(schedule.settings.timer[1].start,false));
128:    $("#timer_stop2 input[type=time]").val(timestr(schedule.settings.timer[1].end,false));
153:    // Output schedule periods
154:    var periods = [];
155:    for (var z in schedule.runtime.periods) {
157:        date = (new Date(schedule.runtime.periods[z].start[0]*1000));
161:        date = (new Date(schedule.runtime.periods[z].end[0]*1000));
165:        periods.push(timestr(h1+m1,true)+" to "+timestr(h2+m2,true));
168:    if (periods.length && schedule.settings.ctrlmode=="smart") {
169:        out = jsUcfirst(schedule.settings.device_name)+" scheduled to run: <b>"+periods.join(", ")+"</b><br>";
186:    for (var time=forecast.start; time<forecast.end; time+=forecast.interval) {
193:        for (var p in schedule.runtime.periods) {
194:            if (time>=schedule.runtime.periods[p].start[0] && time<schedule.runtime.periods[p].end[0]) active = true;
251:    // Shade out time after end of schedule
254:        if (schedule.runtime.periods.length) markings.push({ color: "rgba(0,0,0,0.1)", xaxis: { from: schedule.settings.end_timestamp*1000 } });
320:    if (name=="period" || name=="end") {
325:    else if (name=="timer_start1") schedule.settings.timer[0].start = hour;
326:    else if (name=="timer_stop1") schedule.settings.timer[0].end = hour;
327:    else if (name=="timer_start2") schedule.settings.timer[1].start = hour;
328:    else if (name=="timer_stop2") schedule.settings.timer[1].end = hour;
436:    }).appendTo("body").fadeIn(200);
471:    // Convert hour into end timestamp
pi@emonpi-node-15:/opt/emoncms/modules/demandshaper/demandshaper-module/js $

This may be useful.
Thx for yr help till now.

@TrystanLea

By way of update …

The EV (Tesla) was connected all day yesterday (Thurs) and, as posted above, after running my script (incl yr suggestions) the Demand Shaper webpage had no charging bars but did have Time left and the amount of charge.

This am (Fri) it was clear the EV had not charged overnight.

So I ran the script – no change. I ran the script again and then clicked on the openEVSE button on the Demand Shaper webpage. Voila! charging bars appeared – the full monty.

The EV was not connected at this time – different to yesterday.

I will continue ‘experimenting’ overnight.

It does beg the question of how, in a script, to emulate clicking the openEVSE button on the Demand Shaper webpage. There are many different RAPI commands for controlling openEVSE to choose from and maybe it needs more than one?

Yr advice on this would be great - thx

Hello @johnbanks

I’d like to try a different approach, rather than try and do the calculations in the python script I think moving the hard work to a new “set-device-settings” API provided by the demandshaper module should hopefully work better.

I’ve pushed the new API to the latest version of the demandshaper, available via emonpi update. You only need to provide the target_soc as a parameter in the request now and it does all of the schedule calculation for you:

URL example:

http://emonpi.local/demandshaper/set-device-settings?device=openevse&settings={"target_soc":0.6}

Python script:

import requests, json

emoncms_apikey = ""
emoncms_server = "http://emonpi.local"

reply = requests.get(emoncms_server+"/demandshaper/set-device-settings", params={'device':'openevse','settings':'{"target_soc":0.6}','apikey':emoncms_apikey})
print(reply.text)

How are you setting the SOC of the car? Were you using the emoncms input method for that?

I’ve added the option to set this from the above api as well. You need to specify that this is the source if you do wish to use the api otherwise the input method or ovms method will overwrite the setting. You dont need to select this if your setting the other settings e.g target soc

It’s then possible to set the current car SOC with:

http://emonpi.local/demandshaper/set-device-settings?device=openevse&settings={"current_soc":0.3,"target_soc":0.8}

You can also use this api to set the other settings e.g:

Battery capacity:

/demandshaper/set-device-settings?device=openevse&settings={"battery_capacity":24}

Charge rate:

/demandshaper/set-device-settings?device=openevse&settings={"charge_rate":3.8}

Hi @TrystanLea

I’ve added the call to the set-device-settings API to set the schedule as part of the script @johnbanks shared in post 155, and it appears to have been successful.

I inelegantly tested by adding immediately after setting the apikey and server variables.

I’ll leave it running for a few days, and see if I just truck lucky while testing!

Thanks

@TrystanLea
This is looking great - thx

I did a full update from the Admin webpage and my version increased 10.2.6 > 10.2.7

I then tried yr suggested line in a Python script with apikey, etc:

reply = requests.get(emoncms_server+"/demandshaper/set-device-settings", params={'device':'openevse','settings':'{"target_soc":0.6}','apikey':emoncms_apikey})

But I got:

406
{'success': False, 'message': '406 Not Acceptable. Route not found'}

I tried restarting demandshaper.service but still the same problem.

I also noticed that in the spanner dropdown section of the Demand Shaper webpage, I do NOT have the ‘set-device-settings api’ option.

I suspect I’ve not updated correctly - the log file is:

Starting update via service-runner-update.sh (v3.0) >
- emonSD version: emonSD-24Jul20
- supported images: emonSD-24Jul20 emonSD-02Oct19 emonSD-17Oct19
- emonSD base image check passed...continue update
git pull /opt/openenergymonitor/EmonScripts
  master
* stable
On branch stable
Your branch is up to date with 'origin/stable'.

nothing to commit, working tree clean
Already up to date.
-------------------------------------------------------------
Main Update Script
-------------------------------------------------------------
Date: Mon 18 Jan 14:12:06 GMT 2021
EUID: 1000
openenergymonitor_dir: /opt/openenergymonitor
type: all
firmware: none
Reading package lists...
Building dependency tree...
Reading state information...
python3-pip is already the newest version (18.1-5+rpt1).
The following package was automatically installed and is no longer required:
  rpi-eeprom-images
Use 'sudo apt autoremove' to remove it.
0 upgraded, 0 newly installed, 0 to remove and 76 not upgraded.
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Requirement already satisfied: redis in /home/pi/.local/lib/python3.7/site-packages (3.5.3)
Hardware detected: rfm2pi
Running apt-get update
Get:1 http://raspbian.raspberrypi.org/raspbian buster InRelease [15.0 kB]
Get:2 http://archive.raspberrypi.org/debian buster InRelease [32.8 kB]
Get:3 http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages [13.0 MB]
Get:4 http://archive.raspberrypi.org/debian buster/main armhf Packages [351 kB]
Fetched 13.4 MB in 12s (1,118 kB/s)
Reading package lists...
git pull /opt/openenergymonitor/emonpi
* master
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean
Fetching origin
From https://github.com/openenergymonitor/emonpi
   f915a88..06a689e  master     -> origin/master
 * [new tag]         2.9.3      -> 2.9.3
Updating f915a88..06a689e
Fast-forward
 firmware/platformio.ini | 2 +-
 firmware/readme.md      | 5 +++++
 firmware/src/src.ino    | 3 ++-
 3 files changed, 8 insertions(+), 2 deletions(-)
git pull /opt/openenergymonitor/RFM2Pi
* master
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean
Fetching origin
Already up to date.
Start emonhub update script:
-------------------------------------------------------------
emonHub update
-------------------------------------------------------------
git fetch --all --prune -v
Fetching origin
POST git-upload-pack (984 bytes)
From https://github.com/openenergymonitor/emonhub
 = [up to date]      stable                 -> origin/stable
 = [up to date]      MQTTSubscriberInterfacer -> origin/MQTTSubscriberInterfacer
 = [up to date]      emon-pi                -> origin/emon-pi
 = [up to date]      env_example            -> origin/env_example
 = [up to date]      include_serial_number  -> origin/include_serial_number
 = [up to date]      lanip-status           -> origin/lanip-status
   2baf836..39927d2  master                 -> origin/master
 = [up to date]      redis_config_interface -> origin/redis_config_interface
 = [up to date]      unitless_decoder       -> origin/unitless_decoder
git status
On branch stable
Your branch is up to date with 'origin/stable'.

Untracked files:
  (use "git add <file>..." to include in what will be committed)

	src/src

nothing added to commit but untracked files present (use "git add" to track)
git pull origin stable
From https://github.com/openenergymonitor/emonhub
 * branch            stable     -> FETCH_HEAD
Already up to date.

Running emonhub install & update script emonSD_pi_env:1
--------------------------------------------------------------------
Reading package lists...
Building dependency tree...
Reading state information...
python3-configobj is already the newest version (5.0.6-3).
python3-pymodbus is already the newest version (2.1.0+dfsg-1).
python3-serial is already the newest version (3.4-4).
bluetooth is already the newest version (5.50-1.2~deb10u1+rpt2).
libbluetooth-dev is already the newest version (5.50-1.2~deb10u1+rpt2).
python3-pip is already the newest version (18.1-5+rpt1).
The following package was automatically installed and is no longer required:
  rpi-eeprom-images
Use 'sudo apt autoremove' to remove it.
0 upgraded, 0 newly installed, 0 to remove and 88 not upgraded.
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Requirement already satisfied: paho-mqtt in /usr/local/lib/python3.7/dist-packages (1.5.0)
Requirement already satisfied: requests in /usr/lib/python3/dist-packages (2.21.0)
Requirement already satisfied: pybluez in /usr/local/lib/python3.7/dist-packages (0.23)
Requirement already satisfied: py-sds011 in /usr/local/lib/python3.7/dist-packages (0.9)
Requirement already satisfied: sdm_modbus in /usr/local/lib/python3.7/dist-packages (0.4.3)
Requirement already satisfied: pyserial in /usr/lib/python3/dist-packages (from py-sds011) (3.4)
Requirement already satisfied: pymodbus>=2.3.0 in /usr/local/lib/python3.7/dist-packages (from sdm_modbus) (2.4.0)
Requirement already satisfied: six>=1.15.0 in /usr/local/lib/python3.7/dist-packages (from pymodbus>=2.3.0->sdm_modbus) (1.15.0)
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Requirement already satisfied: RPi.GPIO in /usr/lib/python3/dist-packages (0.7.0)
The user `emonhub' is already a member of `gpio'.
Unit /etc/systemd/system/[email protected] is masked, ignoring.
useradd: user 'emonhub' already exists
- installing emonhub.service
- Service ActiveState=active
--------------------------------------------------------------------
Running emonhub automatic node addition script
[[5]]
Node 5 already present
[[6]]
Node 6 already present
[[7]]
Node 7 already present
[[8]]
Node 8 already present
[[9]]
Node 9 already present
[[10]]
Node 10 already present
[[11]]
Node 11 already present
[[12]]
Node 12 already present
[[13]]
Node 13 already present
[[14]]
Node 14 already present
[[15]]
Node 15 already present
[[16]]
Node 16 already present
[[19]]
Node 19 already present
[[20]]
Node 20 already present
[[21]]
Node 21 already present
[[22]]
Node 22 already present
[[23]]
Node 23 already present
[[24]]
Node 24 already present
[[25]]
Node 25 already present
[[26]]
Node 26 already present

Start emoncms update:
-------------------------------------------------------------
Update Emoncms Core
-------------------------------------------------------------


Checking status of /var/www/emoncms git repository
- git branch: stable
- no local changes
- running: git pull origin stable

Fetching origin
From https://github.com/emoncms/emoncms
   f781bed0..1a5aee34  stable              -> origin/stable
 * [new branch]        fix_lastvalue_error -> origin/fix_lastvalue_error
   c452583b..58ab20fe  master              -> origin/master
 * [new tag]           10.2.7              -> 10.2.7
From https://github.com/emoncms/emoncms
 * branch              stable     -> FETCH_HEAD
Updating f781bed0..1a5aee34
Fast-forward
 Lib/process_old_settings.php                       |   6 +-
 Modules/admin/admin_controller.php                 |   5 +-
 Modules/admin/admin_main_view.php                  |   2 +-
 Modules/admin/locale/fr_FR/LC_MESSAGES/messages.mo | Bin 6705 -> 5595 bytes
 Modules/admin/locale/fr_FR/LC_MESSAGES/messages.po | 330 +++++++++++----------
 Modules/feed/feed_controller.php                   |   3 +-
 Modules/feed/locale/fr_FR/LC_MESSAGES/messages.mo  | Bin 13991 -> 13992 bytes
 Modules/feed/locale/fr_FR/LC_MESSAGES/messages.po  |  60 ++--
 Modules/input/Views/input_api.php                  |   2 +-
 Modules/input/Views/input_view.js                  |   6 +-
 Modules/input/input_methods.php                    |   2 +
 Modules/input/locale/fr_FR/LC_MESSAGES/messages.mo | Bin 15818 -> 15767 bytes
 Modules/input/locale/fr_FR/LC_MESSAGES/messages.po | 173 +++++------
 .../locale/fr_FR/LC_MESSAGES/process_messages.mo   | Bin 26825 -> 30741 bytes
 .../locale/fr_FR/LC_MESSAGES/process_messages.po   | 234 +++++++++++----
 Modules/user/locale/fr_FR/LC_MESSAGES/messages.mo  | Bin 7745 -> 7796 bytes
 Modules/user/locale/fr_FR/LC_MESSAGES/messages.po  | 106 +++----
 Modules/vis/Views/vis_main_view.php                |   4 +-
 .../vis/locale/fr_FR/LC_MESSAGES/vis_messages.mo   | Bin 13087 -> 13475 bytes
 .../vis/locale/fr_FR/LC_MESSAGES/vis_messages.po   |  26 +-
 scripts/phpmqtt_input.php                          |   7 +
 scripts/services/emoncms_mqtt/emoncms_mqtt.php     |   9 +-
 version.txt                                        |   2 +-
 23 files changed, 561 insertions(+), 416 deletions(-)

Update Emoncms database
[]

-------------------------------------------------------------
Update Emoncms Services
-------------------------------------------------------------
emoncms_mqtt.service already installed
feedwriter.service already installed
service-runner.service already installed

Reloading systemctl deamon
Restarting Services...
- sudo systemctl restart feedwriter.service
--- ActiveState=active ---
- sudo systemctl restart emoncms_mqtt.service
--- ActiveState=active ---
- sudo systemctl restart emonhub.service
--- ActiveState=active ---
/opt/openenergymonitor/EmonScripts/sudoers.d/emoncms-rebootbutton: parsed OK
emonPi emoncms admin reboot button sudoers updated

-------------------------------------------------------------
Update Emoncms Modules
-------------------------------------------------------------
------------------------------------------
Updating /var/www/emoncms/Modules/app module
------------------------------------------
- git branch: stable
- git tags: 2.2.2-2-gd345c22
- no local changes
- running: git pull origin stable

Fetching origin
From https://github.com/emoncms/app
   d345c22..7c5f348  stable     -> origin/stable
   2d51198..1a0a8eb  master     -> origin/master
 * [new tag]         2.2.4      -> 2.2.4
Already on 'stable'
Your branch is behind 'origin/stable' by 10 commits, and can be fast-forwarded.
  (use "git pull" to update your local branch)
Updating d345c22..7c5f348
Fast-forward
 apps/OpenEnergyMonitor/myelectric2/myelectric2.php |   3 +-
 apps/OpenEnergyMonitor/octopus/octopus.php         | 208 ++++++++++++++++++++-
 module.json                                        |   2 +-
 3 files changed, 203 insertions(+), 10 deletions(-)

------------------------------------------
Updating /var/www/emoncms/Modules/config module
------------------------------------------
- git branch: stable
- git tags: 2.0.5-6-gd28ff1e
- no local changes
- running: git pull origin stable

Fetching origin
Already on 'stable'
Your branch is up to date with 'origin/stable'.
Already up to date.

------------------------------------------
Updating /var/www/emoncms/Modules/dashboard module
------------------------------------------
- git branch: stable
- git tags: 2.0.8-9-gdc07d12
- no local changes
- running: git pull origin stable

Fetching origin
From https://github.com/emoncms/dashboard
   dc07d12..79c8d4c  master     -> origin/master
Already on 'stable'
Your branch is up to date with 'origin/stable'.
Already up to date.

------------------------------------------
Updating /var/www/emoncms/Modules/device module
------------------------------------------
- git branch: stable
- git tags: 2.0.7
- no local changes
- running: git pull origin stable

Fetching origin
Already on 'stable'
Your branch is up to date with 'origin/stable'.
Already up to date.

------------------------------------------
Updating /var/www/emoncms/Modules/graph module
------------------------------------------
- git branch: stable
- git tags: 2.0.9-3-g6c18b0c
- no local changes
- running: git pull origin stable

Fetching origin
From https://github.com/emoncms/graph
 * [new branch]      edit_feed_scalerange -> origin/edit_feed_scalerange
Already on 'stable'
Your branch is up to date with 'origin/stable'.
Already up to date.

------------------------------------------
Updating /var/www/emoncms/Modules/setup module
------------------------------------------
- git branch: stable
fatal: No names found, cannot describe anything.
- git tags: 
- no local changes
- running: git pull origin stable

Fetching origin
Already on 'stable'
Your branch is up to date with 'origin/stable'.
Already up to date.

------------------------------------------
Updating /var/www/emoncms/Modules/wifi module
------------------------------------------
- git branch: stable
- git tags: 2.0.3
- no local changes
- running: git pull origin stable

Fetching origin
Already on 'stable'
Your branch is up to date with 'origin/stable'.
Already up to date.

/opt/openenergymonitor/EmonScripts/sudoers.d/wifi-sudoers: parsed OK
wifi sudoers entry updated
------------------------------------------
Updating /opt/emoncms/modules/backup module
------------------------------------------
- git branch: master
- git tags: 2.2.6-2-ge4ab2c2
- no local changes
- running: git pull origin master

Already up to date.
Already on 'master'
Your branch is up to date with 'origin/master'.

------------------------------------------
Updating /opt/emoncms/modules/demandshaper module
------------------------------------------
- git branch: stable
- git tags: 2.1.2
- git status:

On branch stable
Your branch is up to date with 'origin/stable'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

	modified:   devices/openevse.php

no changes added to commit (use "git add" and/or "git commit -a")


------------------------------------------
Updating /opt/emoncms/modules/postprocess module
------------------------------------------
- git branch: stable
- git tags: 2.1.4
- no local changes
- running: git pull origin stable

Already up to date.
Already on 'stable'
Your branch is up to date with 'origin/stable'.

------------------------------------------
Updating /opt/emoncms/modules/sync module
------------------------------------------
- git branch: stable
- git tags: 2.0.5-3-g109816f
- no local changes
- running: git pull origin stable

Already up to date.
Already on 'stable'
Your branch is up to date with 'origin/stable'.

------------------------------------------
Updating /opt/emoncms/modules/usefulscripts module
------------------------------------------
- git branch: stable
fatal: No names found, cannot describe anything.
- git tags: 
- no local changes
- running: git pull origin stable

From https://github.com/emoncms/usefulscripts
   644c6e7..4b4ffbe  stable     -> origin/stable
   29ef84a..ec1b24e  master     -> origin/master
Updating 644c6e7..4b4ffbe
Fast-forward
 .gitignore                                   |  1 +
 carbonintensity/carbonintensity.py           | 69 ++++++++++++++++++++++++++++
 carbonintensity/default.carbonintensity.conf |  5 ++
 carbonintensity/readme.md                    | 25 ++++++++++
 4 files changed, 100 insertions(+)
 create mode 100644 carbonintensity/carbonintensity.py
 create mode 100644 carbonintensity/default.carbonintensity.conf
 create mode 100644 carbonintensity/readme.md
Already on 'stable'
Your branch is up to date with 'origin/stable'.

--------------------------------------------
Backup module installation and update script
--------------------------------------------
- Loading EmonScripts config.ini
- Copying default.config.cfg to config.cfg
- Setting config.cfg settings
- Backup module symlink already exists
- PHP Version: 7.3
- Creating /etc/php/7.3/mods-available/emoncmsbackup.ini
post_max_size = 3G
upload_max_filesize = 3G
upload_tmp_dir = /var/opt/emoncms/backup/uploads
- phpenmod emoncmsbackup
- /var/opt/emoncms/backup already exists
- /var/opt/emoncms/backup/uploads already exists
- restarting apache

Update Emoncms database
[]

-------------------------------------------------------------
EmonPi LCD Update
-------------------------------------------------------------
Reading package lists...
Building dependency tree...
Reading state information...
i2c-tools is already the newest version (4.1-1).
python3-smbus is already the newest version (4.1-1).
python3-gpiozero is already the newest version (1.5.1).
The following package was automatically installed and is no longer required:
  rpi-eeprom-images
Use 'sudo apt autoremove' to remove it.
The following additional packages will be installed:
  python-rpi.gpio rpi.gpio-common
The following packages will be upgraded:
  python-rpi.gpio python3-rpi.gpio rpi.gpio-common
3 upgraded, 0 newly installed, 0 to remove and 85 not upgraded.
Need to get 47.3 kB of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 http://archive.raspberrypi.org/debian buster/main armhf python3-rpi.gpio armhf 0.7.0-0.1~bpo10+4 [20.7 kB]
Get:2 http://archive.raspberrypi.org/debian buster/main armhf python-rpi.gpio armhf 0.7.0-0.1~bpo10+4 [20.7 kB]
Get:3 http://archive.raspberrypi.org/debian buster/main armhf rpi.gpio-common armhf 0.7.0-0.1~bpo10+4 [5,952 B]
apt-listchanges: Reading changelogs...
debconf: unable to initialize frontend: Dialog
debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.)
debconf: falling back to frontend: Readline
debconf: unable to initialize frontend: Readline
debconf: (This frontend requires a controlling tty.)
debconf: falling back to frontend: Teletype
dpkg-preconfigure: unable to re-open stdin: 
Fetched 47.3 kB in 0s (387 kB/s)
(Reading database ... 
(Reading database ... 5%
(Reading database ... 10%
(Reading database ... 15%
(Reading database ... 20%
(Reading database ... 25%
(Reading database ... 30%
(Reading database ... 35%
(Reading database ... 40%
(Reading database ... 45%
(Reading database ... 50%
(Reading database ... 55%
(Reading database ... 60%
(Reading database ... 65%
(Reading database ... 70%
(Reading database ... 75%
(Reading database ... 80%
(Reading database ... 85%
(Reading database ... 90%
(Reading database ... 95%
(Reading database ... 100%
(Reading database ... 51784 files and directories currently installed.)
Preparing to unpack .../python3-rpi.gpio_0.7.0-0.1~bpo10+4_armhf.deb ...
Unpacking python3-rpi.gpio (0.7.0-0.1~bpo10+4) over (0.7.0-0.1~bpo10+1) ...
Preparing to unpack .../python-rpi.gpio_0.7.0-0.1~bpo10+4_armhf.deb ...
Unpacking python-rpi.gpio (0.7.0-0.1~bpo10+4) over (0.7.0-0.1~bpo10+1) ...
Preparing to unpack .../rpi.gpio-common_0.7.0-0.1~bpo10+4_armhf.deb ...
Unpacking rpi.gpio-common:armhf (0.7.0-0.1~bpo10+4) over (0.7.0-0.1~bpo10+1) ...
Setting up rpi.gpio-common:armhf (0.7.0-0.1~bpo10+4) ...
Setting up python3-rpi.gpio (0.7.0-0.1~bpo10+4) ...
Setting up python-rpi.gpio (0.7.0-0.1~bpo10+4) ...
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Requirement already satisfied: redis in /home/pi/.local/lib/python3.7/site-packages (3.5.3)
Requirement already satisfied: paho-mqtt in /usr/local/lib/python3.7/dist-packages (1.5.0)
Requirement already satisfied: xmltodict in /home/pi/.local/lib/python3.7/site-packages (0.12.0)
Requirement already satisfied: requests in /usr/lib/python3/dist-packages (2.21.0)
- reinstalling emonPiLCD.service
Removed /etc/systemd/system/emonPiLCD.service.
Removed /etc/systemd/system/multi-user.target.wants/emonPiLCD.service.
- installing emonPiLCD.service
Created symlink /etc/systemd/system/multi-user.target.wants/emonPiLCD.service → /opt/openenergymonitor/emonpi/lcd/emonPiLCD.service.
Created symlink /etc/systemd/system/emonPiLCD.service → /opt/openenergymonitor/emonpi/lcd/emonPiLCD.service.
- Service ActiveState=active
Stopping emonPiLCD service
Display update message on LCD

-------------------------------------------------------------
emonPi update done: Mon 18 Jan 14:13:57 GMT 2021
-------------------------------------------------------------
restarting service-runner

More help pls - and sorry for being a pain

Looks like you have a local modification to the demandshaper module:

------------------------------------------
Updating /opt/emoncms/modules/demandshaper module
------------------------------------------
- git branch: stable
- git tags: 2.1.2
- git status:

On branch stable
Your branch is up to date with 'origin/stable'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

	modified:   devices/openevse.php

no changes added to commit (use "git add" and/or "git commit -a")

Do you want to keep that modification?

If you do not, the steps to clear it are:

cd /opt/emoncms/modules/demandshaper
git checkout .

You can then run the updater again and it should pull in the latest

@TrystanLea T
Thx – that was the problem. And why openevse.php had fully open permissions I don’t recall.

Yr modifications worked great. So I’ve incorporated them into a script which can easily be adapted to run as a cronjob late every evening.

I’ve re-jigged things a bit as I don’t like long lines of code esp when the important bit is on the extreme right and often not readily in view.

# PURPOSE ...

# To adjust the parameters which control the Demand Shaper SMART schedule timing 
# This is achieved WITHOUT needing to access the Demand Shaper webpage - hence can be incorporated into a cron run script, for example

# Facility available from emon ver 10.2.7 and later
# Ref: https://community.openenergymonitor.org/t/emoncms-demand-shaper-module/9097/170

# NOTE - it may be necessary to select the set-device-settings api in the 'spanner dropdown' section of the Demand Shaper webpage
# More detail is needed on this ......

# ---------------------------------------------------------------

import requests, json

# emoncms server configuration ...
emoncms_apikey = "18f63483****************abe048da"   # For Node 15 - use appropriate key
emoncms_server = "http://127.0.0.1"


params = (     
    ('apikey', emoncms_apikey),
    ('device', 'openevse'),
    ('settings', '{"target_soc":0.85, "battery_capacity":75, "charge_rate":6.9}'),   # Adjust values as appropriate ...
)

reply = requests.get(emoncms_server+"/demandshaper/set-device-settings", params=params)

print(reply.text)           # Debugging only
print(reply.status_code)    # Debugging only


# FOR VALIDATION ONLY - print out the full Demand Shaper json ...

reply = requests.get(emoncms_server+"/demandshaper/get", params=params)
jsonstr = reply.text.rstrip()
ds_data = json.loads(jsonstr)

print (ds_data)        
print(reply.status_code)

My objective is for SMART charging to happen every night automatically with no user intervention except connecting the EV. I’ll see how things go over the next few days and let you know.

With regard to yr question about setting current_soc:

When I first set up the openevse device, emon created a bunch of Inputs incl one called soc. I’ve logged this to a Feed and published it to the MQTT topic emon/openevse/soc.

It’s a Tesla EV and I have a script which interrogates the Tesla Mobile App to get data for emon Inputs. One such is Charge % which I’ve logged to a Feed and published to the same MQTT topic emon/openevse/soc.

The EV is currently connected and both Feeds are showing identical. The EV data shows up whether the Tesla is home or away. But I’m not sure where openevse gets its soc data from. Over several days, the plots are identical despite the fact a different EV was charged on one of the days.

Thx for all yr help

1 Like

@Robert.Wall
Oops! I didn’t xxxxxxxxxxxxx out the apikey in my last post
Pls oblige
Many thx

I don’t think it will matter unless you’ve got ports open to the outside world - but it’s got a chunk missing now. It’s when it’s your emoncms.org R/W APIkey that it really matters.

@TrystanLea
The script has been working OK - so many thx for yr prior help with this.

However, in the early hours of 22 May, Demand Shaper stopped charging at 74% when target_soc was set at 86%.

The script is run by cron every night at 11pm to set up Demand Shaper.

I checked emon history for that time. The openEVSE SOC feed was then showing null and the EV SOC was showing 1% and that it was ASLEEP (a separate script is run as a service to constantly poll key EV parameters). The EV had gone to SLEEP previously at 7pm when its SOC was 58%.

Looking further back in emon history I discovered that when the EV is ASLEEP, it sometimes shows a valid SOC but at other times shows a default 1%.

Via Input Processing, I publish to emon/openevse/soc:

  • EV SOC if it is more than 4%
  • The input that was automatically created – openevse SOC

Demand Shaper is set to use Battery Charge Level (input)

My questions are:

  • What triggers Demand Shaper to do its charge time calculation?
  • Where does the INPUT openevse SOC come from?
  • Exactly what info does Demand Shaper use as the EV current SOC?

With that info, I can perhaps devise a way to ‘store’ and use the last valid EV current SOC.

Thx for any help.

Hello @johnbanks

demandshaper_run triggers the update every 60 seconds, this is the line that runs the relvant method in the associated device class: https://github.com/emoncms/demandshaper/blob/master/demandshaper_run.php#L243

This calls this method for the OpenEVSE: demandshaper/devices/openevse.php at master · emoncms/demandshaper · GitHub

The method limits the update rate to 600s here: demandshaper/devices/openevse.php at master · emoncms/demandshaper · GitHub

Here the method fetches the soc value from the emoncms input demandshaper/devices/openevse.php at master · emoncms/demandshaper · GitHub

Is that from your script that fetches the soc from the tesla api?

The value that you write to the input? it does multiply it by 0.01 e.g demandshaper/devices/openevse.php at master · emoncms/demandshaper · GitHub

Hope that helps?

@TrystanLea

Thx for yr further insights.

(Incidently you have responded on 2 topics today but I did not receive the usual emails – and I checked Spam).

By co-incidence, in the early hours today, another ‘rogue’ SMART charge happened – started 4:50am/finished 7:00am.

This SMART charge was not intended and should not have happened – the Demand Shaper json had SMART on and target_soc set at 0.5 and the EV was actually charged to 53%.

In more detail …

I poll the EV (Tesla) every 30 secs by script and get INPUTS – state (online or asleep) and SOC. Both of these are logged to FEEDS. SOC is also MQTT published to emon/openevse/soc but only if greater than 4% – so I assume the MQTT topic value will always be 5% or more. The Tesla EV seems to have a mind of its own – periodically going to sleep which sometime causes SOC to be reported as 1% - the normal min is 5%.

When I first installed the charger/set up Demand Shaper, a whole bunch of openevse INPUTS were automatically created incl one called soc. I log this to a FEED and also MQTT publish to emon/openevse/soc.

I don’t know where from/how this INPUT gets its data but it does seem to track with the EV SOC.

The screenshot shows what happened late pm to early hrs am …

The EV SOC FEED dropped from actual 53% to asleep 1% at 12:30am and was only awoken at 4:50 am when SMART charging started.

In parallel the openevse soc INPUT/FEED was constant at 53% until charging started at 4:50am.

I’m not familiar with php so perhaps my misunderstandings are:

  • Demand Shaper re-calculates SMART charge timings every 60 secs but does not change things more frequently than every 10 mins
  • I assume this happens both before and during the charge
  • The current_soc used for charge timings is the current value of INPUT openevse soc. And during the relevant period (per screenshot), the log to FEED of this INPUT was constant at 53%
  • The topic value emon/openevse/soc is irrelevant

Could it be that having a target_soc LESS THAN current_soc confuses things? Should the default in that case be – don’t do SMART charging?

In Demand Shaper settings I use – Battery charge level (input) rather than set-device-settings api. Is this correct? What exactly does each do?

Perhaps I should change my script to turn off SMART charging if current_soc is greater than target_soc?

If so, how in Python can I use the emon/openevse/soc value – to avoid the issue of the Tesla EV going to sleep?

Sorry for the confusion and all the questions

Thx

Hello @johnbanks

The soc input should be from your script that’s polling the tesla API, the openevse has no way of otherwise knowing the car’s soc.

It looked like demandshaper schedule was started following the period where the soc was reported at near 0, which makes sense.

If we can get to the bottom of why it’s reporting the incorrect soc that seems like the best course of action. Can you remind me of the script that you are using to poll the tesla API?