Rate of change engine

I want to use Emoncms to provide some data to help predict when the heating should come on. My starting point is to look at the rate of change for internal and external temperatures

Had a quick peek at the feed and wondered what engine was best. IIRC the advice was to use PHPTimeseries for everything. Is that correct?


If you are doing calcs involving “time since last” it might be wise to stick with variable timeseries as the fixed interval feeds may distort the calcs I guess.

Engaging my brain a bit more, it seems this feed simply compares last to current for the rate of change which is not really what I want. I suppose what I’m after is trend date over a period of time that I can specify. I had a look at PID and whilst I’ll admit the maths is way beyond me, it strikes me the problem is that it does not really learn over a longer period of time so it can predict when for instance, heating should come on so the set point is reached by a specific time. This is a function of current temp and outside temp and I do not see how that can be fed in to the PID.

Need to do some more research…

It sounds as if you’re mixing up two separate concepts. A PID controller is widely used in process control to stabilise a feedback loop. So you could use a PID controller and proportional control in place of an on-off thermostat to maintain your heating at a precisely defined temperature. What it sounds as if you want is an “optimiser”, which learns the rate at which a building heats up in order to be able to predict the last possible moment to turn on heating so that it hits the desired temperature at the desired time.

PID control comes in 3 parts: P is proportional, so the output is proportional to the error term (the difference between the actual and desired values). I is integral, so the error is accumulated over time, with the aim of reducing the long-term error. D is differential, so a fast change in the error causes a large change in the output, effectively speeding up the response. You can usually change the time constants of the I & D terms, and always change their relative contribution to the output.

I am back on this (2 years later!).

What I need is to determine the rate of change for a specific feed.

So, the difference between Tnow and T-2min.

The use case is the temperature at the top of my tank. I know that if it is dropping at over 1 deg/min over 2 min, then the boiler needs to go on. An absolute value as the trigger is no good as it is only required when someone is having a shower. If the rate of fall is less than this, it will be the UFH drawing the heat and the temp can drop further (depending on time of day).

The only way I have found to calculate a rate, is to use the Trend sensor in HomeAssistant and export the value again. I cannot believe it is the only (or best) way.

Any other suggestions?

Take a look at the “If rate >=, skip next” process.
It only deals with last and current value but i’m sure will work for what you want.
A similar function could be made to support adding the time variable to the equation but if the sensors post at regular intervals you already know that.

Last value is no good as I need to look at the value 2 minutes ago. I suppose you could argue this is not strictly an instantaneous rate more a temperature difference between two points in time, 2 minutes apart.

[edit] - That process could give me an instantaneous rate of 1deg/min (a hot water tap) but it is the sustained rate I am looking for (as in a shower).

Can you explain what are you trying to attain?

I want to monitor the rate at which my hot tank is discharging so that I prempt it falling below the required level. So, if someone is having a shower, I need the boiler to start earlier than if the UFH is running. From analyzing the data I have determined that the best way to do this is that, if the tank temperature has fallen about 2 degrees in the last 2 minutes, I want the boiler to switch on (actual value I’ll work out by T&E).

Without doubt a custom process would be a favorable approach, but just thinking outloud for a work around, I assume the temp is accessible to nodered via Mqtt? If you subscribe to the temp value and resubmit it to emoncms 120s later and save that to another feed (mysql in-memory?), would it then be a simple case of subtracting current temp from delayed temp?

I know this isn’t pretty, but if it works ok, then mimicking that in a custom emoncms process might be achievable by making a call to a feed api for the delayed value from the same current temp feed. It’s not easy to query a specific time, so the process might not be great as it will need to query a period and select a particular time:value from the data returned. What makes this process different to others that use previous data is the others all use the last value rather than cherry picking a specific time or interval.

What are you using for the first intelligent device that handles the output of the temperature sensor?

Because it sounds to me as if you want a rolling value of the rate of change from 2 minutes ago to now, and if you have an emonTx/emonTH (or if you can put up with the pain of updating the front end, an emonPi) or something with similar processing power, the place to do it is there, then send either the number or a flag that says “I call for heat”.

I agree the front end is probably the best place for this, that was my first thought but felt it may have more uses if in emoncms, the more I think about it, the less I think that is the case.

I disagree with the flag as I think the raw change value would have significant value to improve the efficiency of the system, plus if this were a more widely adopted approach for this application, then a staggered flag would have even more benefit, eg if the rate of change is only 1deg and there is spare PV, heat the water, otherwise if theres 2degs of change and we are in a off-peak elec tariff period, heat the water or only if over 3degs during peak tariffs should the water be heated? I would concider doing something like this if it was more accurate than the temp levels directly.

Assuming a device with limited processing and a 10s update a simple 12 value array and a incrementing (+rolling) index to pop the delayed value and write the new value at each read would do the job nicely and fairly cheaply.

It’s been lost in the recent spec. (post no.9) but earlier, the requirement was for a sustained and large rate of change - i.e. there are two parameters to define, the minimum duration of the change and the amount of change during that time.

I think the inference was that a slow reduction in tank temperature over a wider range was acceptable for underfloor heating, but not for a shower. We’re almost back to the top few posts, and require a ‘predictive’ model: ‘shower detected: don’t wait, turn the boiler on now!’

The likelihood of memory being available for an array of recent values, and the essentially ‘stateless’ nature of subsequent processing, was the logic I followed. I think I can see the difficulties in implementing this in emonCMS, but agree that it would have multiple uses there.

Easily added as config parameters to a sketch (eg like node id etc) as I think we can assume if the device and sensors are fitted to one application, eg DHW tank, that is unlikely to change frequently so it is a install time setting.

This multi parameter need would cause alot of complexity in emoncms, ie the input ptocessing page would need re-designing to accomadate more boxes and the db table structure expanded etc etc. So the workaround would possibly end up being a process for each duration eg a RateOfChange120 thru to RateOfChange3600 (and so on?) so that the user defined parameter could be just feed id (for the delayed value), this would clutter up the processes dropdown selector somewhat and be for a quite niche user group.

Either way the level of change can then be managed with “if more than” and similar processes in emoncms or in other locations, the other advantage of doing this at source, is that the change is available sooner, rather than waiting for emoncms to process and publish, the raw change can be picked up as quick as the actual current temperature by other softwares/devices.

@pb66 @Robert.Wall I think we are over thinking this. The intelligent part I can handle through HomeAssistant, I just need a mechanism to give me the difference between the temperature 2 minutes ago and the temperature now. Perhaps all I need is the value for 2 minutes ago, so I can compare it to the one now and decide if the ‘rate of change’ is sufficiently high to turn on the boiler.

It is just the relative temperature change I need. I could store all the values in yet another database but that seems very inefficient.

I think the solution is to use the Trend sensor in Home Assistant.

Or the diff between the temp then and now which is all both Robert and I have suggested, we have just talked thru the reasoning and the usefulness of this to a wider audience, as this might increase the likelyhood of it getting implemented and/or avoid implementing something that is too ridged and specific to be of use to anyone else.

In short, your simple ask is not easily achieved in emoncms from a single temp value. Establishing the RoC or delaying a copy of the value to resend, at source is the way to go.

IMO the RoC would be better than a repeated/delayed value as it would possibly be easier to tell them apart (except for a small window of low temp uses) and the already determined difference might be of benefit to a dumber device, (eg a mqtt relay could subscribe to one value rather than 2) or is that also over thinking it again?

1 Like

Ah Ok, I see where you are coming from.

Just realised that I can make an API call for a 2 minute slice of data where (in theory) I could use the first and the last data item.

I use OpenHAB with InfluxDB to control my dehumidifier in the basement. My requirement is slightly different. I don’t want the dehumidifier to short cycle, because they are horribly inefficient when run that way (but that is exactly what mine and many these days do). So I use a long term average, but they also have derivative:

I was using a more complex thing in NodeRED that also worked, but it needed to keep track of time nd some history and didn’t work well (until it got back in sync) when I restarted NodeRED.

So, NodeRED if you want complete control. OpenHAB with Influxdb as the persistent store if you want something easier to code for the control, but a significant learning curve for all the technology.