Rate process query

Emoncms
1

Trying to use the Rate process to weed out spikes in data.

From the explanation, it seems that it is not a ‘rate’ that is specified, but the maximum absolute change allowed between last and now. Is that correct? If so, I cannot seem to get it to work :frowning: (and yes REDIS is running).

image
image1154×156 13.8 KB

I fear that once it has skipped a value, it cannot then get back on track

  • How many does it skip, using the last ‘good’ value before it resets to current value?
2

@TrystanLea Can you shed any light on this?

3

Hello @borpin

Here’s the code for the process list:

public function ifRateGtEqualSkip($arg, $time, $value, $options)
{
    global $redis;
    if ($redis) {
        $redispath = "process:ifRateGtEqualSkip:".$options['sourcetype'].":".$options['sourceid']."_".$this->parentProcessModel->proc_goto;
        //$this->log->info("ifRateGtEqualSkip() time=$time value=$value redispath=$redispath");
        if ($redis->exists($redispath)) {
            $lastvalue = $redis->hmget($redispath,array('time','value'));
            $change = abs($value - $lastvalue['value']);
            if ($change >= $arg)
                $this->parentProcessModel->proc_skip_next = true;
        }
        $redis->hMset($redispath, array('time' => $time, 'value' => $value));
    }
    return $value;
}

It looks like it starts by saving the last time and value to a redis key. and then if that time and value exists previously it calculated the difference between the current value and the last value as an absolute value.

If that difference is more than the provided argument it should skip the next process.

Testing with the following:

image
image1765×288 32.8 KB

Let’s say the input value is 100, if the change is <10 the multiplier process does not get skipped and so the feed value ends up being e.g 10000. If the change is >=10 the value does not have the multiplier applied and so ends up being the original value.

That seems to work ok for me?

4

What happens if it just logs the value after the rate check?

What happens on subsequent data? How long does it wait for a figure in range before giving up and logging again?

I’m finding it stops logging and never starts again.

5

If I send an input value say 100 twice that gets logged fine as difference is less than 10. If I then send 120 that value is skipped. If I send 120 the second time that gets logged…

6

That’s with:

image
image1753×258 32.4 KB

7

Ok, not understanding why mine stops. I’m still on V10 though - have you had a chance to check on whether the update script will work on a Debian based version?

8

I ran the EmonScripts installer on a new Ubuntu VM the other day and it all went fine, was there a reason that it should not work?

9

No, just wondered :slight_smile:

I can create a snapshot easily enough, just I’ve got a bit behind on updating :frowning:

I’ll force an update of the usefulscripts repo first.

1 Like