Weekly KWh processor

Emoncms
1

Hi
I’m trying to setup a KWh total for the last 7 days but I’m struggling to find the necessary code that needs modifying, Ive created the following code in process_processlist.php.


// Power to KWh Week (Last 7 Days)
    public function power_to_kwhw($feedid, $time_now, $value)
    {
        $new_kwh = 0;

        // Get last value
        $last = $this->feed->get_timevalue($feedid);

        if (!isset($last['value'])) $last['value'] = 0;
        if (!isset($last['time'])) $last['time'] = $time_now;
        $last_kwh = $last['value']*1;
        $last_time = $last['time']*1;

        $current_slot = $this->getstartday($time_now);
        $last_slot = $this->getstartday($last_time);    

        $time_elapsed = ($time_now - $last_time);   
        if ($time_elapsed>0 && $time_elapsed<7200) { // 2hrs
            // kWh calculation
            $kwh_inc = ($time_elapsed * $value) / 3600000.0;
        } else {
            // in the event that redis is flushed the last time will
            // likely be > 7200s ago and so kwh inc is not calculated
            // rather than enter 0 we dont increase it
            $kwh_inc = 0;
        }

        if($last_slot == $current_slot) {
            $new_kwh = $last_kwh + $kwh_inc;
        } else {
            # We are working in a new slot (new day) so don't increment it with the data from yesterday
            $new_kwh = $kwh_inc;
        }
        $this->log->info("power_to_kwhd() feedid=$feedid last_kwh=$last_kwh kwh_inc=$kwh_inc new_kwh=$new_kwh last_slot=$last_slot current_slot=$current_slot");
        $this->feed->update_data($feedid, $time_now, $current_slot, $new_kwh);

        return $value;
    }

For the life of me I cant find the part of the script that resets at midnight.

Also
How easy would it be to have a reset button on the dashboard?

Regards
Dave

2

The way the “daily” processes work is to get a timestamp for the start of the day in question (usually today) and then if the “start of the day” is the same as the previous datapoints “start of the day” it adds the previous and the latest values together or if the latest “start of the day” is not equal to the previous datapoints “start of the day” then it will not sum the values and just save the latest value alone. So it doesn’t “reset” as such, it just doesn’t include the existing accumulating total for what was the current day until this latest post.

At the bottom of the process/process_processlist.php file there is a getstartday() function, you could write a similar “getstartweek()” function to get a running total for “this week”, however that is different to a “KWh total for the last 7 days” that you refer to (unless it happens to be Sunday evening of course) do you want a “total so far this week” that “resets” at the same time each week or a rolling “total for the last 7 days”?

3

Hi Paul @pb66
The reasoning for selecting the last 7 days was because I assumed that the script would use seconds, I thought an mod to copy and modify it to suite me needs would be easy.
I am struggling understanding what part of the script does what, ie I was expecting something like a time_now
What part of the script works out the start of the day?

public function getstartday($time_now)
    {
        $now = DateTime::createFromFormat("U", (int)$time_now);
        $now->setTimezone(new DateTimeZone($this->timezone));
        $now->setTime(0,0);    // Today at 00:00
        return $now->format("U");
    }

Regards
Dave

4

The way I read it is that it establishes the current time and date from the unix timestamp in the first line, which will therefore be UTC so the 2nd line looks like it converts that to the defined timezone.

The most relevant bit I think is the 3rd line where it then set’s the $now time to 00:00, ie midnight.

In the 4th line it returns “$now” in unix timestamp format, which should be the time (00:00am) and date (retained from the original timestamp) of the start of the day for whatever timestamp was originally supplied ($time_now).

But is that what you want? or will the “total for this week” serve your purpose?

Getting a rolling 24hr or 7days via a process should be possible by writing a process that can deduct 7x24x60x60 secs from the $time_now to determine a “time_from” and queries a accumulator feed (eg a power to kwh feed) for the 2 values, (the current kWh and the kWh at the same time a week earlier). Subtracting the latter from the former will give the rolling 7days total. ,

Actually this wouldn’t be as easy as I would have hoped because there is no easy way to query a particular time. only the last value or a range between to timestamps if I recall correctly.

5

I’ve had a go at creating the function and think it should be something like this

public function getstartweek($time_now) {
        $now = DateTime::createFromFormat("U", (int)$time_now);
        $now->setTimezone(new DateTimeZone($this->timezone));
        $startofweek = strtotime ( "-".date('N', $now->format('w'))." day" , $time_now );
        $now->setDate(date('Y',$startofweek),date('m',$startofweek),date('d',$startofweek));
        $now->setTime(0,0);    // Today at 00:00
        return $now->format("U");
    }

I cannot vouch for the $now->setTimezone(new DateTimeZone($this->timezone)); line but the rest I have tested to a degree in this little test script, I commented out that line rather than start playing with timezones.

<?php
function getstartweek($time_now) {
        $now = DateTime::createFromFormat("U", (int)$time_now);
        //$now->setTimezone(new DateTimeZone($this->timezone));
        $startofweek = strtotime ( "-".date('N', $now->format('w'))." day" , $time_now );
        $now->setDate(date('Y',$startofweek),date('m',$startofweek),date('d',$startofweek));
        $now->setTime(0,0);    // Today at 00:00
        return $now->format("U");
    }


if (isset($argv[1])) {
        $time_now = $argv[1];
} else {
        $time_now = time();
}
print "current timestamp is: ".$time_now."\n";
print "this week started at: ".getstartweek(time())."\n";
?>

If you copy the script to a file called getstartofweek.php you should be able to get the start of the week timestamp for any given unix time stamp like so

php  ./getstartofweek.php 1496426656

will return

pi@workshopPi:~ $ php ./getstartofweek.php 1496426656
current timestamp is: 1496426656
this week started at: 1496016000

if you omit a timestamp arg it will return the start of the current week like so

pi@workshopPi:~ $ php ./getstartofweek.php
current timestamp is: 1496426838
this week started at: 1496016000

hopefully this might help you get to where you want. Fingers crossed you should just be able to add the “getstartweek” function to the bottom of your process_processlist.php file and edit the 2 “getstartday” occurrences in your new “power_to_kwhw” function to “getstartweek”, I haven’t tested it in emoncms at all though and my PHP is not strong, so proceed at your own peril :slight_smile:

6

Thanks Paul
I’ve added the code in and I’ll leave it running over the next few days to see if it zeros on Sunday>MIDNIGHT<Monday.
If successful I’ll include a how to on here so other people can benefit.

Many Thanks
Dave

7

Hi Paul
The counter reset last night, I believe the timestamp was Sunday night?

Regards
Dave

8

“N” format was added in PHP 5.1.0 (so it says here).
Could that be the reason?
I would assume that if “N” is not recognised as a valid parameter, it defaults to the week starting on Sunday. (“w” is the parameter for that.)

9

Actually I think this line was an early (wrong) effort that has slipped back in somehow.

$startofweek = strtotime ( "-".date('N', $now->format('w'))." day" , $time_now );

it could be (A)

$startofweek = strtotime ( "-".(date('N', $time_now)-1)." day" , $time_now );

however this might cause timezone issues as the start of the week is based on the unixtimestamp (UTC/GMT), so perhaps (B)

$startofweek = strtotime ( "-".(date('N', $now->format("U"))-1)." day" , $now->format("U"));

would retain any timezone adjustments from the setTimezone line before it.

The original line seems to constantly returns a date 4 days prior to the date provided rather than establish the day of the week number for the date supplied, subtract 1 (so Mon = 0 and Sun = 6) and subtract that many days from that date to get the start of the week date as intended.

@Dave if you try amending that line I think it should reset again tonight and the real test will be if it doesn’t reset on Monday night. I think (A) should reset at 23:00 Sunday night and if (B) is TZ corrected it should reset at Midnight, I would say try (B) first and see if that works, Do you have the TZ set in your emoncms account?

@Robert.Wall, I think we might be ok with the “N” as most of us are using PHP 5.5.6 AFAIK,

1 Like
11

Many thanks Paul
Yes I’ve set the timezone up as Europe/London, I’ll let you know Tuesday how I get on.

Thanks again
Dave

12

Hi Paul
Just to update you, the counter reset on Sunday night but at a different time to the normal KWh/Day, but that might be due to the timezones.
I’m going to wait until next week to check that everything works ok before posting a success and a short how too.

Regards
Dave

1 Like
13

That’s interesting! Is the $now->setTimezone(new DateTimeZone($this->timezone)); line uncommented in both the getstartday and the getstartweek functions? Do your kWh/d feeds normally reset at midnight or midnight UTC?

Although I had originally only commented it out of my test script (not the suggested function), I have since uncommented it and hardcoded a TZ to test it further and it seems to function correctly.

Just to document this for future ref. This is the revised “getstartweek.py” test script (TZ set to Europe/London)

<?php
function getstartweek($time_now) {
        $now = DateTime::createFromFormat("U", (int)$time_now);
        $now->setTimezone(new DateTimeZone("Europe/London"));

        // These 2 lines are added to the `getstartday` function for `getstartweek`
        $startofweek = strtotime ( "-".(date('N', $now->format("U"))-1)." day" , $now->format("U"));
        $now->setDate(date('Y',$startofweek),date('m',$startofweek),date('d',$startofweek));

        $now->setTime(0,0);    // Today at 00:00
        return $now->format("U");
    }

// If an arg is passed from the commandline it is assumed to be the timestamp to test
// otherwise it will default to using the current UTC time if no arg passed.
if (isset($argv[1])) {
        $time_now = $argv[1];
} else {
        $time_now = time();
}
print "current timestamp is: ".$time_now."\n";
print "this week started at: ".getstartweek($time_now)."\n";
?>

and if there’s any interest here’s a similar getstrtmonth script too.

<?php
function getstartmonth($time_now) {
        $now = DateTime::createFromFormat("U", (int)$time_now);
        $now->setTimezone(new DateTimeZone("Europe/London"));

        // This line is  added to the `getstartday` function for `getstartmonth`
        $now->setDate(date('Y',$now->format("U")),date('m',$now->format("U")),0);

        $now->setTime(0,0);    // Today at 00:00
        return $now->format("U");
    }

// If an arg is passed from the commandline it is assumed to be the timestamp to test
// otherwise it will default to using the current UTC time if no arg passed.
if (isset($argv[1])) {
        $time_now = $argv[1];
} else {
        $time_now = time();
}
print "current timestamp is: ".$time_now."\n";
print "this month started at: ".getstartmonth($time_now)."\n";
?>

let us know what happens next Sunday!

1 Like
14

Thanks Paul
I’ve put the month in and we’ll see how we get on.

Regards
Dave

15

Hi Paul
I just noticed an odd one, the weekly and monthly totals don’t survive a reboot, oddly enough the KWh/Day if fine? Is there something we need to code somewhere else that saves the total rather than storing it in the apparent RAM?

Regards
Dave

16

I’ve took a look at the code to work out why that might be and although far from being able to draw any conclusions, I suspect it might be due to the “DAILY” “DataType” which is fixed to a 24hr interval and not editable.

“// Daily datatype automatically adjust feed interval to 1d and user can’t change it from gui.”

I haven’t got to the bottom of it but here’s a couple of links for some relevant code

The DataType “2” (DAILY) gets handled differently here, fixing the interval to 86400secs without a dropdown

This seems to be where the usual phpfina fixed intervals get shortlisted for the dropdown