Struggles with the EmonCMS API

pb66 · 12 February 2017 13:48

I’m sorry too hear you are not going to be using emoncms, but I do appreciate you taking the time to explain why and although I am not a core emoncms developer I would like to respond to the points you raised, hopefully to help future users even if it is too late to tempt you back. It would be of great benefit to the project to have users with your experience on board.

Silent failures, agreed this is not a good situation, many softwares including previous versions of our own emonhub have relied on this response for confirmation of a successful post before deleting the local copy. It is an error and should be labelled as a bug. it now has “bug” status.
EmonCMS.org differences: Whilst it isn’t clear to many users there are significant differences between the emoncms repo and the code run on emoncms.org. There is a separate repo for emoncmsorg if you are interested in the code, but at the very least the user level differences should be documented or preferably, not exist. This is not a development verses stable version issue, there are very different stratagies applied and these need to be highlighted.
Named-key inputs: The issue here is that the “inputname” or the “inputindex” share the same “key” field so you cannot use both, when using CSV bulk input api the “key” is explicitly set to the input’s index number as an “inputname”, the input pages list “key” and “name” fields, behind the scenes they are actually “name” and " description" fields. I have regularly asked for an additional field so that both “inputindex” and “inputname” can be used simultaneously
API input names: Although poorly documented, this is actually possible using the input/set api see below. Also take a look at the emoncms device module, I use this a lot for bulk creation of inputs, processlists and feeds. Possibly not in the way it was designed but it works well for the purpose.

Set input fields	http://localhost/emoncms/input/set.json?inputid=0&fields={'description':'Input Description'}

github.com

emoncms/emoncms/blob/master/Modules/input/input_controller.php#L280

    
      
          <?php
              /*
               All Emoncms code is released under the GNU Affero General Public License.
               See COPYRIGHT.txt and LICENSE.txt.
          
          
        ---------------------------------------------------------------------
                  Emoncms - open source energy visualisation
                  Part of the OpenEnergyMonitor project:
                  http://openenergymonitor.org
              */
          
          
    // no direct access
              defined('EMONCMS_EXEC') or die('Restricted access');
          
          
function input_controller()
          {
              global $mysqli, $redis, $user, $session, $route, $settings, $param, $device, $path;
              
              // requires at least read access
              if (!isset($session['read'])) return false;
              if (!$session['read']) return false;
          
          
    $result = false;
          
          
    require_once "Modules/feed/feed_model.php";
              $feed = new Feed($mysqli,$redis, $settings["feed"]);
          
          
    require_once "Modules/input/input_model.php";
              $input = new Input($mysqli,$redis, $feed);
          
          
    require_once "Modules/process/process_model.php";
              $process = new Process($mysqli,$input,$feed,$user->get_timezone($session['userid']));
          
          
    if (!$device) {
                  if (file_exists("Modules/device/device_model.php")) {
                      require_once "Modules/device/device_model.php";
                      $device = new Device($mysqli,$redis);
                  }
              }
          
          
    require_once "Modules/input/input_methods.php";
              $inputMethods = new InputMethods($mysqli,$redis,$user,$input,$feed,$process,$device);
              
              // Change default route to json
              $route->format = "json"; 
          
          
    // Write access level
              if ($session["write"])
              {
                  // ------------------------------------------------------------------------
                  // input/post
                  // ------------------------------------------------------------------------
                  if ($route->action == "post") {
                      $result = $inputMethods->post($session['userid']);
                      if ($result=="ok") {
                          if ($param->exists('fulljson')) $result = '{"success": true}';
                          if ($param->sha256base64_response) $result = $param->sha256base64_response;
                      } else {
                          $result = '{"success": false, "message": "'.str_replace("\"","'",$result).'"}';
                          $log = new EmonLogger(__FILE__);
                          $log->error($result." for User: ".$session['userid']);
                      }
                      return $result;
                  }
                  
                  // ------------------------------------------------------------------------
                  // input/bulk
                  // ------------------------------------------------------------------------
                  else if ($route->action == 'bulk') {
                      $result = $inputMethods->bulk($session['userid']);
                      if ($result=="ok") {
                          if ($param->exists('fulljson')) $result = '{"success": true}';
                          if ($param->sha256base64_response) $result = $param->sha256base64_response;
                      } else {
                          $result = '{"success": false, "message": "'.str_replace("\"","'",$result).'"}';
                          $log = new EmonLogger(__FILE__);
                          $log->error($result." for User: ".$session['userid']);
                      }
                      return $result;
                  }
                  // ------------------------------------------------------------------------  
                  else if ($route->action == "clean") {
                      $route->format = 'text';
                      return $input->clean($session['userid']);
                  } else if ($route->action == "cleanprocesslistfeeds") {
                      $route->format = 'text';
                      return $input->clean_processlist_feeds($process,$session['userid']);
                  } else if ($route->action == "set-node-input-descriptions") {
                      if (isset($_GET['node']) && isset($_GET['names'])) {
                           return $input->set_node_input_descriptions($session['userid'],$_GET['node'],$_GET['names']);
                      }
                  }
                  else if (isset($_GET['inputid']) && $input->belongs_to_user($session['userid'],get("inputid")))
                  {
                      if ($route->action == 'set') return $input->set_fields(get('inputid'),get('fields'));
                      else if ($route->action == "delete") return $input->delete($session['userid'],get("inputid"));
                      else if ($route->action == "process") 
                      {
                          if ($route->subaction == "get") return $input->get_processlist(get("inputid"));
                          else if ($route->subaction == "set") return $input->set_processlist($session['userid'], get('inputid'), post('processlist'),$process->get_process_list());
                          else if ($route->subaction == "reset") return $input->reset_processlist(get("inputid"));
                      }
                  }    
                  // Multiple input actions - permissions are checked within model
                  else if (isset($_GET['inputids'])) {
                      if ($route->action == "delete") {
                          $inputids = json_decode(get('inputids'));
                          if ($inputids!=null) return $input->delete_multiple($session['userid'],$inputids);
                      }
                  }
                  
                  // -------------------------------------------------------------------------
                  // HTML Web pages
                  // -------------------------------------------------------------------------
                  else if ($route->action == 'api') {
                      $route->format = "html";
                      textdomain("messages");
                      return view("Modules/input/Views/input_api.php", array());
                  }    
                  else if ($route->action == 'view') {
                      $route->format = "html";
                      textdomain("messages");
                      $device_module = false;
                      if (file_exists("Modules/device")) $device_module = true;
                      return view("Modules/input/Views/input_view.php", array(
                          'path' => $path,
                          'device_module' => $device_module,
                          'feedviewpath' => $settings['interface']['feedviewpath']
                      ));
                  }
              }
              
              // Read access
              
              // --------------------------------------------
              // Fetch inputs by node and node variable names
              // --------------------------------------------
              // input/get                              full list
              // input/get?node=emontx                  {"power1":{"time":0,"value":0},"power2":{"time":0,"value":0},"power3":{"time":0,"value":0}}
              // input/get/emontx                       {"power1":{"time":0,"value":0},"power2":{"time":0,"value":0},"power3":{"time":0,"value":0}}
              // input/get?node=emontx&name=power1      {"time":0,"value":0}
              // input/get/emontx/power1                {"time":0,"value":0}
                  
              if ($route->action == "get") {
                  $dbinputs = $input->get_inputs_v2($session['userid']);
                  
                  if (!$route->subaction && !isset($_GET['node'])) {
                      return $dbinputs;
                  } else {
                      // Node
                      if ($route->subaction) { $nodeid = $route->subaction; } else { $nodeid = get('node'); }
                      $nodeid = preg_replace('/[^\p{N}\p{L}_\s\-.]/u','',$nodeid);
                      
                      // If no node variable name specified return all node variables
                      if (!$route->subaction2 && !isset($_GET['name'])) {
                      
                          if (isset($dbinputs[$nodeid])) {
                              return $dbinputs[$nodeid];
                          } else {
                              return "Node does not exist";
                          }
                      
                      } else {
                          // Property
                          if ($route->subaction2) { $name = $route->subaction2; } else { $name = get('name'); }
                          $name = preg_replace('/[^\p{N}\p{L}_\s\-.]/u','',$name);
                          
                          if (isset($dbinputs[$nodeid])) {
                              if (isset($dbinputs[$nodeid][$name])) {
                                  return $dbinputs[$nodeid][$name];
                              } else {
                                  return "Node variable does not exist";
                              }
                          } else {
                              return "Node does not exist";
                          }
                      }
                  }
              }
          
          
    else if ($route->action == "list") return $input->getlist($session['userid']);
              else if ($route->action == "getinputs") return $input->get_inputs($session['userid']);
              else if ($route->action == "get_inputs") return $input->get_inputs($session['userid']);
          
          
    return array('content'=>$result);
          }

This file has been truncated. show original

API create feeds: I agree. The API’s need an overhaul to be more user friendly or at the very least documented. There are many things that can be done via the api’s that are too complicated or simply not known.
Null datapoints: Although I haven’t tested lately, it is/was possible to pass null values to a local emoncms install. This is a feature I did contribute to emoncms some time ago, See the “Can you ‘load’ nulls in BULK json upload format to emoncms.org?” thread for some info.
Re-uploading history: Not entirely sure I have the right end of the stick here but… If you are trying to bulk upload back-data to phpfina or phptimeseries, the trick is to create the feed whilst there is no incoming data (or to use the feeds api) to create an un initialised feed. A feed is initialised upon recieving it’s first adta and the timestamp of that first data becomes the “start date” of the feed, once created you cannot post data earlier that the “start date” so making the first datapoint the earliest ever is essential to back-fill historic data.

If I can tempt you to retry emoncms I would happily try and answer any questions you have as I have a fairly good working knowledge of it. The position I (we) and future “power users” have is that it is currently complex, incomplete and undocumented, and because of that, many move on or just accepts its shortfalls. the result is nothing gets improved as there isn’t the demand for the more indepth stuff.

In actual fact it, is in the name of “simplification” that emoncms.org runs very different code to the emoncms repo and that the documentation is moving towards a “quick user guide”, so it good to know there is also a demand for in-depth documentation and advanced features too.