Data Viewer questions

pb66 · 23 April 2018 11:05

Written to disk, no longer just in RAM, Redis or Buffer etc, actually written to disk.

Looks like that is right too.

When posting using the buffered-write though, the usual phpfina “post()” function isn’t used.

When the input process calls feed->insert_data it checks for redisbuffer enabled (feed_model.php L814) and uses redisbuffer->post() and not phpfina->post() for low-write installs.

In that instance post() just adds the details to a buffer and then periodically, the bufferedwrite service calls redisbuffer->process_buffers() which in turn calls redisbuffer->process_feed_buffer(). Which (and I’m a little hazy here) I believe will ordinarily call phpfina->post_bulk_prepare(). Which appears to check if ($pos>$last_pos) and only writes the data if the data is later than the last recorded data (not equal to).

github.com

emoncms/emoncms/blob/master/Modules/feed/engine/PHPFina.php#L679


      
                  if ($pos>=0 && $pos < $meta->npoints)
                  {
                      // read from the file
                      fseek($fh,$pos*4);
                      $val = unpack("f",fread($fh,4));
          
                      // add to the data array if its not a nan value
                      if (!is_nan($val[1])) {
                          $value = $val[1];
                      }
                  }
          
                  $split_values[] = $value;
              }
          
              $data[] = array($time,$split_values);
          
              $date->modify($modify);
              $time = $date->getTimestamp();
          }
          fclose($fh);

further down that code, the “else” says "if data is in past, its not supported, … ".

The code is pretty complex so I wouldn’t be at all surprised if I’ve got something not quite right.

A lot of it revolves around what $last_pos actually is. I had arrived at the conclusion that the minus one in the $last_pos = $meta->npoints - 1; lines is just to derive an index from a count eg a count of 3, minus 1, gives us an index of 2 as in 0,1,2 is a count of 3.