I would like to be able to record & graph this data within emonCMS. If I use Node-RED to grab data every 30 minutes, how would co-ordinate this? Perhaps poling the data at Times xx:01 and xx:31 and limiting the data via Parameters: **period_from (datetime, optional)** Show consumption from the given datetime (inclusive). This parameter can be provided on its own. **period_to (datetime, optional)** Show consumption to the given datetime (exclusive). This parameter also requires providing the period_from parameter to create a range.
And then logging the data to a feed & then a graph. Would that work, or am I barking up the wrong tree? There is more here, but as you know this is at the edge of my comfort zone! Octopus Energy APIs — Octopus Energy API 1.0 documentation
make the msg.payload value the consumption and set msg.time as the end time (the ISO format should be fine - check emoncms logs), there will be a debug message along lines of ‘valid time found’.
HTH
[edit]
When I create the Octopus NR node, I’ll store the last value received and then look for the next one after 30 mins to ensure no overlap. I might even attempt to recover missed data.
Also, as the emonCMS node uses the HTTP API, you could use the ‘full JSON’ option and construct the message payload passed to the emonCMS node as
Thanks @borpin ; I have now created a flow which works to an extent - it grabs the data & only passes on one set of consumption / interval_end values to emonCMS push, but an error shows up below the emonCMS node, and no data seems to arrive at my emonPi.local (unlike the Powerwall data which does).
Do I need to re-format / tidy up the data that gets sent to emonCMS, or is this OK as is?
@borpin; herewith some screenshots. The Octopus node does collect “sensible” data, the Function node strips out all but the most recent consumption data (c/w times), the Delete Interval_start mode strips out one of the two time stamps (I thought this worthwhile but could be superfluous), but then there is an error with the emonCMS node.
emonCMS Node 0 is the one I use for the Powerwall, but I am aware that I have not configured this properly…
If you look at the data you are sending the emoncms node i.e. this output
Does it look like that? (hint - it doesn’t)
The node-red node does some of the hard work, but it still has to lokk familiar to it.
Delete the switch node and play with the function to construct a suitable msg to input to emonCMS node.
There is detail in the emonCMS Node Help panel on the data format. I suggest you use a slightly different format to feed into the emoncms node. i.e. a separate time and payload and just send a value.
so e.g.
msg.payload = different formats
msg.time = an ISO time string
Try this in the function - untested you may need to debug it.
Remember your Feed on the emonCMS end needs to be a 30 minute feed and you only want to trigger this every 30 minutes. However, by using the ‘time’ function you should not muck up the Feed data. There is a risk you don’t get the last period data for a max of 30 mins (depending on when it starts, but that is the best you can do without some fancy processing.
If you wanted to post all the historical data to emonCMS, you could actually loop through the array and load all the API will give you.
Amazing help @borpin - thank you. I have set the Timestamp node to action once every 30 minutes.
I needed to tweak a setting within the emonCMS node (to accept “simple” JSON), but then it worked; the Octopus consumption figures are only every 30 minutes, and delayed by 18 hours, so it’ll be a slow process to get the graph to show much - all the more so as I hope to only import anything between 00:30 - 04:30 anyway. But I’ll monitor it over the next 24 hours.
I spoke too soon . It seems that the data is published once a day, about 25 hours delayed. (ie up until 23:30 the night before last night). I therefore want to grab the first 48 thirty minute data points once every 24 hours at, say, 05:00, each day.
So rather than the Function node showing … var newmsg = {}; newmsg.payload = msg.payload.results[0].consumption; newmsg.time = msg.payload.results[0].interval_end; return newmsg;
… it needs to be something along the lines of … var newmsg = {}; newmsg.payload = msg.payload.results[0-47].consumption; newmsg.time = msg.payload.results[0-47].interval_end; return newmsg;
What might the correct syntax for this be? I have googled it, but not found what I am after.
I tried this by way of an experiment … var newmsg = {}; newmsg.payload = msg.payload.results[0].consumption; newmsg.time = msg.payload.results[0].interval_end; newmsg.payload = msg.payload.results[1].consumption; newmsg.time = msg.payload.results[1].interval_end; return newmsg;
but only the “0” data point came back (which surprised me).
If anybody is interested, I’ve put together some simple Python code to import the Octopus energy meter readings (both gas and electric).
The code will automatically create the necessary Inputs and Feeds in emoncms so you don’t need to pre-configure emoncms.
The code automatically finds the first meter reading from Octopus and imports the historic data and then just the incremental data after that (uses the last write date from emoncms to determine the checkpoint)
Schedule this code to run once per day using cron or similar.
Code is here…
Its nice to see how accurate the emonpi is…
Red line is the import recorded by emoncms and the green actual from smart meter