Update script does not include DemandShaper as far as I can tell

ian · 2019-06-14T11:11:36.686Z

Hi

I have used the update script several times the last few days in order to get a fully up to date system
(Self hosted RPI) having resolved a few issues discussed in other topics.
I am using demandshaper and I did a git pull on the demandshaper directory and changes were pulled in.
I therefore think that demandshaper has not been included in the update script. Am I correct and should it be added?

TrystanLea · 2019-06-14T11:19:43.209Z

Hello Ian, that’s correct, the demandshaper isnt yet in the update script. I’ve been doing more work on the demandshaper over the last few days, adding in a mechanism to confirm that the state has changed on the smartplug/device. The interface turns grey if the smartplug is turned off at the wall with a ‘device unresponsive’ message. Im still working through a number of niggling issues with it but its getting there.

ian · 2019-06-14T11:53:07.827Z

Hi @TrystanLea

I have just pulled in the latest changes and my smartplugs are no longer showing up.
They all show mqtt as connected when I log into the individual IP addresses. They have all appeared in demandshaper asking to be allowed. That’s as far as it gets.I have cleared cache but no difference.

Update on this. Its because the 4 smartplugs that do not really exist are now persistant and reappear as inputs every time you delete then. I have always had to delete these inputs to get the actual smartplugs to show up in demandshaper.

Also the big orange flash screen you first see in demandshaper has text not visible at the bottom.

ds977×428 41.6 KB

There is still the issue of old smartplug names no longer in use showing up in inputs. I have a hunch this is connected with old names being stored in the ESP somewhere. One of my devices shows up on the router as SMARTPLUG2 and on the device web menu as smartplugWM Both on the same ip address http://192.168.1.77/

Ian

TrystanLea · 2019-06-14T12:41:17.696Z

Hello @ian Im not sure why that would be. Did you restart the demandshaper service?

sudo systemctl restart demandshaper

You could if you wish roll back to before my recent changes with:

git checkout 54c7186de70354fc6c311899371d8049a0d5a13f

borpin · 2019-06-14T20:07:24.705Z

TrystanLea:

that’s correct, the demandshaper isnt yet in the update script.

@TrystanLea Why not let the update script cycle through the Modules folder and so pick up any Module for an update?

Seems like demandshaper needs a Stable branch

ian · 2019-06-15T13:42:08.787Z

TrystanLea:

sudo systemctl restart demandshaper

Hi

I rolled back and restarted demandshaper. I still had 8 smartplugs showing in inputs and no devices in demandshaper. None of the inputs had the little clock displaying. I then deleted all inputs using the clean API.
From this stage I saw a new (to me) behaviour. Looking at inputs I still had 8 smartplugs 4 of them are non existent and I refer to them as phantoms.

smartplugs1207×390 30.1 KB

But now the 4 genuine smartplugs have the clock icon and I can see and control them in demandshaper.

If only the phantoms could be permanently deleted so they did not keep showing as inputs I think everything would be OK.

I have a question about logs. I have read all the posts about log rotate and understand what is trying to be achieved. Is the demandshaper log included in the new log rotate? If so where can I now find the demandshaper logs?

TrystanLea · 2019-06-16T22:17:27.350Z

Hello @ian I think I’ve managed to first replicate and then fix this now… hopefully

If you pull in the latest master branch

cd /var/www/emoncms/Modules/demandshaper
git checkout master
sudo systemctl restart demandshaper

and then when your on the demandshaper page, navigate to the following url. It will reset the demandshaper and remove the old device entries that are causing the inputs to keep appearing.

http://emonpi/demandshaper/clearall

The valid devices will automatically recreate with a page refresh.

The next time you delete a device in the demandshaper it will now delete the demandshaper schedule entry properly before deleting the device and inputs, so should stop the issue causing the inputs to recreate.

I’ve also, I hope, improved the way the demandshaper checks the status of the device to confirm that the settings have been applied, it now uses mqtt and works for the smartplug, openevse and heatpump monitor. The openevse control page now starts and stops the openevse properly for manual operation.

ian · 2019-06-17T09:09:54.503Z

Hi @TrystanLea

Carried out updates as per you post.

One slight oddity
I did the manual checkout and pull and saw the demandshaper updates.

Did a demandshaper/clearall
cleared cache.
Still had 4 phantom inputs
I then ran the update script

./update_emoncms.sh

and saw updates in demand shaper again

I also ran the update from administration to make sure everything was up to date. I also cleared cache and rebooted.

Did demandshaper/clearall again
Results:-
cleared

Looked in inputs and phantoms still there!

smartplugs1419×425 36.9 KB

However demandshaper does work and I have seen saved message flash on screen next to smartplug name and also a settings mismatch message briefly.

TrystanLea · 2019-06-17T09:26:31.841Z

Great, if you delete the ‘phantom’ inputs now using the cog icon which deletes the complete device that should solve it. If you run demandshaper/clearall again after doing that to be sure. The clearall clear’s all the schedules but doesnt actually delete the devices/inputs, but it will stop them being recreated - hence the need to delete the devices/inputs as well.

ian · 2019-06-17T09:52:07.131Z

Sorry, missed the delete input step.

All good. I have a test plug with audible switch. The response seems better than in the past when manually switching from demandshaper.

What does settings mismatch indicate? It stayed visible over several manual switching operations on one smartplug and then eventually cleared.

The one outstanding issue that I am aware of is the fact that the settings are not being saved correctly in the ESP. I still have to reenter IP address and MQTT password and reconnect if I lose power to the smartplug. Also I have to hard code the names as per your suggestion a while back.

One minor thing on the demandshaper control page. The manual button colours are not persistent. On goes green and the graph goes pink. If you navigate away and come back the graph remains pink but the button is greyed out. Also applies to off button.

TrystanLea · 2019-06-17T13:57:44.292Z

Great

ian:

What does settings mismatch indicate?

It essentially means that there is a mismatch in the state of the smartplug vs the demandshaper. The state check is a secondary processes that happens about 2s after sending the command to change the settings. The command to change the settings goes via a bit of a convoluted route and seems to sometimes get delayed. Its something I want to improve on so that its a bit more responsive.

Il note the ESP issue and the button colours. There’s quite a bit more work required on this in general, still an early beta release.