Log to Feed interval

borpin · 24 October 2016 16:10

Is it possible to see what the interval is for an existing Log to Feed feed. It see a similar question was asked here. It would be useful to be able to see this information. I want to create a second feed that is the same as another created a while ago but I cannot remember what the interval was.

pb66 · 24 October 2016 16:27

Try od -i /path/to/phpfina/123.meta

pi@emoncmsPi:~ $ od -i /var/lib/phpfina/100458.meta
0000000           0           0        3600  1483228800
0000020
pi@emoncmsPi:~ $

3600 is the interval
1483228800 is the start time

from Display RAW feed data from PHP engines for debuging

borpin · 24 October 2016 17:13

OK thanks. Could this information be included in the interface?

pb66 · 4 December 2016 13:16

I too think this would be a useful addition, perhaps raising a feature request “issue” on github would help?

EDIT - I have opened an issue, feel free to add or expand on it…

github.com/emoncms/emoncms

Exposing fixed interval feed meta

opened 01:25PM - 04 Dec 16 UTC

closed 09:56PM - 23 May 17 UTC

pb66

enhancement

See forum thread [Log to Feed interval](https://community.openenergymonitor.org/…t/log-to-feed-interval/1971) The fixed interval meta such as start timestamp and interval size should be made accessible, if not displayed on the feed page directly then perhaps via a "info pop-up" from the feeds page. At least adding the meta data fields to the "get feed fields" feed api would improve accessibility by removing the need to ssh in to run commandline queries, which even that is not possible for emoncms.org users.

hagai-shatz · 6 December 2016 18:47

You can use the undocumented API: https://emoncms.org/feed/getmeta.json?id=1
Result for PHPFINA feed is: {“interval”:10,“start_time”:1461310000,“npoints”:65748}

pb66 · 6 December 2016 19:44

Good find!!!

Just tried it on a self-hosted emoncms v9.7.8 PHPfina and got slightly different results though, not sure why?

{“interval”:10,“start_time”:1474373280}

Thanks for sharing that.

borpin · 6 December 2016 20:31

Works on 9.2 as well (yes I know still not been brave enough to upgrade…)

I do wish the API was properly documented; not that I’ve mentioned it before {GRD}

hagai-shatz · 6 December 2016 22:27

Seems like the npints is returned only after the feed file is accessed via read/write. When you just start the server and get meta, only the start time and interval are available.

pb66 · 6 December 2016 22:33

I’m not sure I understand what you mean there, I ran the same API url as you quoted on a live system, the phpfina feed I tested against is fully initialized and currently logged to.

hagai-shatz · 6 December 2016 22:46

If you get data from the feed using /feed/data.json?.. and then call the /feed/getmeta.json?.. you should get the npoints as well (based on the code, I didn’t test it myself on a local instance).

pb66 · 6 December 2016 22:56

Do you mean like this?

https://myserver/feed/data.json?id=100240&start=1481064538000&end=1481064738000&interval=10

[[1481064548000,45.25],[1481064558000,45.369998931885],[1481064568000,44.944999694824],[1481064578000,45.549999237061],[1481064588000,45.055000305176],[1481064598000,45.650001525879],[1481064608000,45.979999542236],[1481064618000,45.104999542236],[1481064628000,47.185001373291],[1481064638000,46.604999542236],[1481064648000,47.990001678467],[1481064658000,47.439998626709],[1481064668000,45.509998321533],[1481064678000,46.485000610352],[1481064688000,46.599998474121],[1481064698000,46.944999694824],[1481064708000,45.310001373291],[1481064718000,45.900001525879],[1481064728000,43.604999542236],[1481064738000,46.764999389648],[1481064748000,43.229999542236]]

https://myserver/feed/getmeta.json?id=100240

{“interval”:5,“start_time”:1467633890}

pb66 · 7 December 2016 09:00

Looking at the code it appears the npoints has been added to emoncms.org but not the main emoncms repo, although the api and the feed functions do the same job, the PHPfina feed engine and the fields it returns from get_data() are different.

github.com

emoncms/emoncmsorg/blob/master/emoncms/Modules/feed/engine/PHPFina.php#L477-L510

    
      
                  $fh = fopen($this->dir.$id.".dat", 'rb');
                  $size = $meta->npoints*4;
                  fseek($fh,$size-4);
                  $d = fread($fh,4);
                  fclose($fh);
          
          
        $val = unpack("f",$d);
                  $time = date("Y-n-j H:i:s", $meta->start_time + $meta->interval * $meta->npoints);
                  
                  return array('time'=>$time, 'value'=>$val[1]);
              }
              else
              {
                  return array('time'=>0, 'value'=>0);
              }
          }
          
          
public function export($id,$start)
          {
              $id = (int) $id;

This file has been truncated. show original

github.com

emoncms/emoncms/blob/master/Modules/feed/engine/PHPFina.php#L92-L120

    
      
          /**
           * Gets engine metadata
           *
           * @param integer $id The id of the feed to be created
          */
          public function get_meta($id)
          {
              $id = (int) $id;
              $feedname = "$id.meta";
          
          
    if (!file_exists($this->dir.$feedname)) {
                  $this->log->warn("get_meta() meta file does not exist '".$this->dir.$feedname."'");
                  return false;
              }
          
          
    // Open and read meta data file
              // The start_time and interval are saved as two consecutive unsigned integers
              $meta = new stdClass();
              $meta->id = $id;
              $metafile = fopen($this->dir.$feedname, 'rb');

This file has been truncated. show original

hagai-shatz · 7 December 2016 10:03

My mistake.
I was unaware there is a different codebase for emoncmsorg and I was testing the API against emoncms.org.
I was also mistaken the static $metadata_cache to be persistent across multiple requests, and this is not the case in PHP.

pb66 · 7 December 2016 10:10

I wouldn’t say that as I would not of expected these differences even though I was aware there was a separate repo. Thanks for pointing out the api, either does the job, I just didn’t understand the difference in results. Perhaps we could add the npoints to the main repo if it is useful?