Using inputs, feeds and processlists with API

Hi All,

I’m working on a local test set-up for emoncms based on a Vagrant development enviroment. I wan’t to replicate my live setup, and re-create all inputs, feeds and processlists on building the Vagrant development enviroment. I’m using a curl script to set-up the inputs and feeds, which work fine at the moment. The thing I yet do not understand is how to bind the inputs to the feeds by using the input/process/set API call. What I’ve done so far by submitting the following calls via PostMan (gui for API programming)

create input (GET)
http://127.0.0.1:8080/input/post.json?node=test-node&json={test-input:205}&apikey=xxx

{ok} (created input id 30)

create feed (GET)
http://127.0.0.1:8080/feed/create.json?tag=test-feed&name=test-feed&datatype=1&engine=5&options={"interval":10}&apikey=XXX

{"success":true,"feedid":17,"result":true}

create processlist (POST)
http://127.0.0.1:8080/input/process/set.json?inputid=30&processlist=1:17&apikey=XXX

{"success":true,"message":"Input processlist updated"}

Result
All seems OK, but I cannot see the processlist updated for the feed, nor does sending new data result in updating the feed.

http://127.0.0.1:8080/input/get_inputs.json ...{"test-input":{"id":"30","processList":""}}...

What am I doing wrong here?

Although I cannot see anything specifically"wrong" with your api calls, I do wonder if the processlist should be in curly braces as usually there would be several items eg processlist={2:0.01,1:17} to be consistent with the options= and json= etc but documentation is thin on those api calls.

Also you should take a look at the emoncms/device module, it can take care of all this for several inputs and dozens of feeds with complex processing with a couple of mouse clicks or a single api call once the templates are created. (See the Device module - Autoconfigured devices thread on the old forum.)

Hi Paul,

Adding the braces did not solve it, same results still.

Could you give an example on how you are using the device module? I see the example.json file, containing the wanted structure, but grasping it’s logic confuses me still.

Here is an example of one of my earlier templates.

{       "name": "emonTx-L1",
        "description": "Automatic inputs and feeds creation for single L1 device.",
        "inputs": [
                {
                        "name": "1",
                        "description": "L1-AC_voltage",
                        "processList": [
                                                {
                                                        "process": "1",
                                                        "arguments": {"type": "ProcessArg::FEEDID", "value": "L1-voltage" }
                                                },
                                                                                                {
                                                        "process": "27",
                                                        "arguments": {"type": "ProcessArg::FEEDID", "value": "L1-voltage_max" }
                                                },
                                                                                                {
                                                        "process": "28",
                                                        "arguments": {"type": "ProcessArg::FEEDID", "value": "L1-voltage_min" }
                                                }
                                        ]
                },
                {
                        "name": "2",
                        "description": "L1-AC_frequency",
                        "processList": [
                                                {
                                                        "process": "1",
                                                        "arguments": {"type": "ProcessArg::FEEDID", "value": "L1-frequency" }
                                                }
                                        ]
                },
                {
                        "name": "3",
                        "description": "L1-CT1_current",
                        "processList": [
                                                {
                                                        "process": "1",
                                                        "arguments": {"type": "ProcessArg::FEEDID", "value": "L1-grid_current" }
                                                }
                                        ]
                },
                {
                        "name": "4",
                        "description": "L1-CT2_current",
                        "processList": [
                                                {
                                                        "process": "1",
                                                        "arguments": {"type": "ProcessArg::FEEDID", "value": "L1-solar_current" }
                                                }
                                        ]
                },
                {
                        "name": "5",
                        "description": "L1-CT3_current",
                        "processList": [
                                                {
                                                        "process": "1",
                                                        "arguments": {"type": "ProcessArg::FEEDID", "value": "L1-heat_current" }
                                                        }
                                        ]
                },
                {
                        "name": "6",
                        "description": "L1-CT4_current",
                        "processList": [
                                                {
                                                        "process": "1",
                                                        "arguments": {"type": "ProcessArg::FEEDID", "value": "L1-light_current" }
                                                }
                                        ]
                },
                {
                        "name": "7",
                        "description": "L1-CT1_power",
                        "processList": [
                                                {
                                                        "process": "1",
                                                        "arguments": {"type": "ProcessArg::FEEDID", "value": "L1-grid_power" }
                                                },
                                                {
                                                        "process": "4",
                                                        "arguments": {"type": "ProcessArg::FEEDID", "value": "L1-grid_kwh" }
                                                },
                                                {
                                                        "process": "5",
                                                        "arguments": {"type": "ProcessArg::FEEDID", "value": "L1-grid_kwhd" }
                                                },
                                                                                                {
                                                        "process": "25",
                                                        "arguments": {"type": "ProcessArg::NONE", "value": "0" }
                                                },
                                                                                                {
                                                        "process": "1",
                                                        "arguments": {"type": "ProcessArg::FEEDID", "value": "L1-export_power" }
                                                },
                                                {
                                                        "process": "4",
                                                        "arguments": {"type": "ProcessArg::FEEDID", "value": "L1-export_kwh" }
                                                },
                                                {
                                                        "process": "5",
                                                        "arguments": {"type": "ProcessArg::FEEDID", "value": "L1-export_kwhd" }
                                                },
                                                                                                {
                                                        "process": "37",
                                                        "arguments": {"type": "ProcessArg::NONE", "value": "0" }
                                                },
                                                                                                {
                                                        "process": "24",
                                                        "arguments": {"type": "ProcessArg::NONE", "value": "0" }
                                                },
                                                                                                {
                                                        "process": "1",
                                                        "arguments": {"type": "ProcessArg::FEEDID", "value": "L1-import_power" }
                                                },
                                                {
                                                        "process": "4",
                                                        "arguments": {"type": "ProcessArg::FEEDID", "value": "L1-import_kwh" }
                                                },
                                                {
                                                        "process": "5",
                                                        "arguments": {"type": "ProcessArg::FEEDID", "value": "L1-import_kwhd" }
                                                }
                                        ]
                },
                {
                        "name": "8",
                        "description": "L1-CT2_power",
                        "processList": [
                                                {
                                                        "process": "1",
                                                        "arguments": {"type": "ProcessArg::FEEDID", "value": "L1-solar_power" }
                                                },
                                                {
                                                        "process": "4",
                                                        "arguments": {"type": "ProcessArg::FEEDID", "value": "L1-solar_kwh" }
                                                },
                                                {
                                                        "process": "5",
                                                        "arguments": {"type": "ProcessArg::FEEDID", "value": "L1-solar_kwhd" }
                                                },
                                                {
                                                        "process": "29",
                                                        "arguments": {"type": "ProcessArg::FEEDID", "value": "L1-grid_power" }
                                                },
                                                {
                                                        "process": "1",
                                                        "arguments": {"type": "ProcessArg::FEEDID", "value": "L1-use_power" }
                                                },
                                                {
                                                        "process": "4",
                                                        "arguments": {"type": "ProcessArg::FEEDID", "value": "L1-use_kwh" }
                                                },
                                                {
                                                        "process": "5",
                                                        "arguments": {"type": "ProcessArg::FEEDID", "value": "L1-use_kwhd" }
                                                }
                                        ]
                },
                {
                        "name": "9",
                        "description": "L1-CT3_power",
                        "processList": [
                                                {
                                                        "process": "1",
                                                        "arguments": {"type": "ProcessArg::FEEDID", "value": "L1-heat_power" }
                                                },
                                                {
                                                        "process": "4",
                                                        "arguments": {"type": "ProcessArg::FEEDID", "value": "L1-heat_kwh" }
                                                },
                                                {
                                                        "process": "5",
                                                        "arguments": {"type": "ProcessArg::FEEDID", "value": "L1-heat_kwhd" }
                                                }
                                        ]
                },
                {
                        "name": "10",
                        "description": "L1-CT4_power",
                        "processList": [
                                                {
                                                        "process": "1",
                                                        "arguments": {"type": "ProcessArg::FEEDID", "value": "L1-light_power" }
                                                },
                                                {
                                                        "process": "4",
                                                        "arguments": {"type": "ProcessArg::FEEDID", "value": "L1-light_kwh" }
                                                },
                                                {
                                                        "process": "5",
                                                        "arguments": {"type": "ProcessArg::FEEDID", "value": "L1-light_kwhd" }
                                                },
                                                {
                                                        "process": "33",
                                                        "arguments": {"type": "ProcessArg::NONE", "value": "0" }
                                                },
                                                {
                                                        "process": "29",
                                                        "arguments": {"type": "ProcessArg::FEEDID", "value": "L1-use_power" }
                                                },
                                                {
                                                        "process": "30",
                                                        "arguments": {"type": "ProcessArg::FEEDID", "value": "L1-heat_power" }
                                                },
                                                {
                                                        "process": "30",
                                                        "arguments": {"type": "ProcessArg::FEEDID", "value": "L1-light_power" }
                                                },
                                                {
                                                        "process": "1",
                                                        "arguments": {"type": "ProcessArg::FEEDID", "value": "L1-other_power" }
                                                },
                                                {
                                                        "process": "4",
                                                        "arguments": {"type": "ProcessArg::FEEDID", "value": "L1-other_kwh" }
                                                },
                                                {
                                                        "process": "5",
                                                        "arguments": {"type": "ProcessArg::FEEDID", "value": "L1-other_kwhd" }
                                                }
                                        ]
                }
        ],

        "feeds": [
                {
                        "name": "L1-voltage",
                        "type": "DataType::REALTIME",
                        "engine": "Engine::PHPTIMESERIES"
                },
                {
                        "name": "L1-voltage_max",
                        "type": "DataType::DAILY",
                        "engine": "Engine::PHPTIMESERIES"
                },
                {
                        "name": "L1-voltage_min",
                        "type": "DataType::DAILY",
                        "engine": "Engine::PHPTIMESERIES"
                },
                {
                        "name": "L1-frequency",
                        "type": "DataType::REALTIME",
                        "engine": "Engine::PHPTIMESERIES"
                },
                {
                        "name": "L1-grid_current",
                        "type": "DataType::REALTIME",
                        "engine": "Engine::PHPTIMESERIES"
                },
                {
                        "name": "L1-solar_current",
                        "type": "DataType::REALTIME",
                        "engine": "Engine::PHPTIMESERIES"
                },
                {
                        "name": "L1-use_current",
                        "type": "DataType::REALTIME",
                        "engine": "Engine::PHPTIMESERIES"
                },
                {
                        "name": "L1-heat_current",
                        "type": "DataType::REALTIME",
                        "engine": "Engine::PHPTIMESERIES"
                },
                {
                        "name": "L1-light_current",
                        "type": "DataType::REALTIME",
                        "engine": "Engine::PHPTIMESERIES"
                },
                {
                        "name": "L1-other_current",
                        "type": "DataType::REALTIME",
                        "engine": "Engine::PHPTIMESERIES"
                },
                {
                        "name": "L1-grid_power",
                        "type": "DataType::REALTIME",
                        "engine": "Engine::PHPTIMESERIES"
                },
                {
                        "name": "L1-export_power",
                        "type": "DataType::REALTIME",
                        "engine": "Engine::PHPTIMESERIES"
                },
                {
                        "name": "L1-import_power",
                        "type": "DataType::REALTIME",
                        "engine": "Engine::PHPTIMESERIES"
                },
                {
                        "name": "L1-solar_power",
                        "type": "DataType::REALTIME",
                        "engine": "Engine::PHPTIMESERIES"
                },
                {
                        "name": "L1-use_power",
                        "type": "DataType::REALTIME",
                        "engine": "Engine::PHPTIMESERIES"
                },
                {
                        "name": "L1-heat_power",
                        "type": "DataType::REALTIME",
                        "engine": "Engine::PHPTIMESERIES"
                },
                {
                        "name": "L1-light_power",
                        "type": "DataType::REALTIME",
                        "engine": "Engine::PHPTIMESERIES"
                },
                {
                        "name": "L1-other_power",
                        "type": "DataType::REALTIME",
                        "engine": "Engine::PHPTIMESERIES"
                },
                {
                        "name": "L1-grid_kwh",
                        "type": "DataType::REALTIME",
                        "engine": "Engine::PHPTIMESERIES"
                },
                {
                        "name": "L1-export_kwh",
                        "type": "DataType::REALTIME",
                        "engine": "Engine::PHPTIMESERIES"
                },
                {
                        "name": "L1-import_kwh",
                        "type": "DataType::REALTIME",
                        "engine": "Engine::PHPTIMESERIES"
                },
                {
                        "name": "L1-solar_kwh",
                        "type": "DataType::REALTIME",
                        "engine": "Engine::PHPTIMESERIES"
                },
                {
                        "name": "L1-use_kwh",
                        "type": "DataType::REALTIME",
                        "engine": "Engine::PHPTIMESERIES"
                },
                {
                        "name": "L1-heat_kwh",
                        "type": "DataType::REALTIME",
                        "engine": "Engine::PHPTIMESERIES"
                },
                {
                        "name": "L1-light_kwh",
                        "type": "DataType::REALTIME",
                        "engine": "Engine::PHPTIMESERIES"
                },
                {
                        "name": "L1-other_kwh",
                        "type": "DataType::REALTIME",
                        "engine": "Engine::PHPTIMESERIES"
                },
                {
                        "name": "L1-grid_kwhd",
                        "type": "DataType::DAILY",
                        "engine": "Engine::PHPTIMESERIES"
                },
                {
                        "name": "L1-export_kwhd",
                        "type": "DataType::DAILY",
                        "engine": "Engine::PHPTIMESERIES"
                },
                {
                        "name": "L1-import_kwhd",
                        "type": "DataType::DAILY",
                        "engine": "Engine::PHPTIMESERIES"
                },
                {
                        "name": "L1-solar_kwhd",
                        "type": "DataType::DAILY",
                        "engine": "Engine::PHPTIMESERIES"
                },
                {
                        "name": "L1-use_kwhd",
                        "type": "DataType::DAILY",
                        "engine": "Engine::PHPTIMESERIES"
                },
                {
                        "name": "L1-heat_kwhd",
                        "type": "DataType::DAILY",
                        "engine": "Engine::PHPTIMESERIES"
                },
                {
                        "name": "L1-light_kwhd",
                        "type": "DataType::DAILY",
                        "engine": "Engine::PHPTIMESERIES"
                },
                {
                        "name": "L1-other_kwhd",
                        "type": "DataType::DAILY",
                        "engine": "Engine::PHPTIMESERIES"
                }
        ]
}

First off the inputs are listed each has a “name” field (read index or “key” as per the emoncms inputs page), a “description” field (“name” in emoncms input pages) and a “processList” field which has 2 sub-fields for each process in the list, a process id (numeric function name) and the “arguments”, This “arguments” field is the key to this working, basically it is going to use the “value” to lookup the feed id so you must ensure there is a feed defined in the feeds section to suit.

The feeds section is used first, using the “name”, “type” and “engine” fields (plus “interval” and “processList” where needed for fixed interval and virtual feeds) the device module creates all the feeds first and notes the feedids so that it can create the inputs using the correct feedids.

The error handling isn’t great so be sure you have created enough feeds and there are no typo’s, once the feeds are created it immediately created the inputs and if any feed isn’t “known” ie held in memory following creation from within the same template and same run, it will fail and you cannot “pick up where you left off” so you then have to delete it all and start again or complete it by hand.

I use an online JSON validator to check my JSON as a missed comma or brace will also cause a failure. I tend to run the template on a test site before using it on a live site and it is easy to repeat the templates, for example this template I duplicated for L2 and L3 on a 3phase install.

You cannot unfortunately use “+ input” or “+feed” type processes that refer to inputs or feeds that already exist or are being created by another template, so for example when I needed to sum the power of the 3 phases, each created by it’s own template, I had another “totaling template” for that. I can expand on how that works if you like but it’s a bit of a workaround so I won’t confuse this post with it right now.

Once the templates are in emoncms/Modules/device/data you can create as many devices as you wish accross as many accounts as you like , from the “device” page or via the api (there is a link to the api docs in the top right corner of the device page just like the input and feeds pages).

Due to little or no docs it is a bit hard to get to grips with at first but once your rolling it is very easy to use. A bit of trial and error really is the best way to pick it up, I can only assume it’s popularity is held back by the fact many users only create one set of inputs and feeds etc so it’s as easy to create them by hand as it is to create the templates and manual creation permits minor errors to be corrected and changes to be made enroute, but with the device module it is essential everything is in place before hitting the button and then it’s done instantly.

Also see the Emoncms Device Wizard thread for a discussion around auto-configuration of devices and the device module.