Retrieve data from local emoncms

[poldom (Pol Domingo)]

Hi

I’m trying to retrieve data from some feeds of my local emoncms. I’ll use this data in a python script afterwards.

I’m trying to do it using information from https://emoncms.org/site/api#feed. Is this still valid? I get no results, using, for example:

wget -qO- "https://emoncms.org/feed/value.json?id=7&apikey=$APIKEY"

(I have my read api key stored at the $APIKEY variable).

In particular, I need the last month data (from day 1 of the month to the actual moment) from some feeds.

Thanks a lot

[dlmpryor (Buhay)]

I just so happen to be working on a similar thing on a local Emoncms Version 10.2.6
I am working on a CRON job to retrieve data and put in a file.
I had to add Authorization: Bearer before the API key to get it to read.
Maybe you have a similar problem?

’
#!/bin/bash

#Enter the API Keys here
LOCAL_RO_KEY=‘xxxxxxxxxxxxxxxxxxxxxxxx’

#Set the Feed IDs here
PI_IMPORT_FEED_ID=‘24’

#Fetch the data from the feeds on the emonpi

PI_IMPORT=$(curl -s http://127.0.0.1/emoncms/feed/value.json?id=$PI_IMPORT_FEED_ID -H “Authorization: Bearer LOCAL_RO_KEY”)

echo “–”
echo “This is the raw data…”
echo “PI_IMPORT=$PI_IMPORT”
echo $PI_IMPORT >> file.txt
’

[poldom (Pol Domingo)]

Hi @dlmpryor.

Thanks for your answer!!

I have tried your script (I have an emoncms 10.2.6 too), and I get a route not found error.

PI_IMPORT={"success":false,"message":"406 Not Acceptable. Route not found"}

[pb66 (Paul)]

@poldom

I’m trying to retrieve data from some feeds of my local emoncms.

Do you mean local “on the same machine” or “on the same LAN” to the device you are issuing the api call?

@poldom

wget -qO- "https://emoncms.org/feed/value.json?id=7&apikey=$APIKEY"

This is pointing at emoncms.org, not your local emoncms. You need to swap out the “emoncms.org” to your local emoncms IP or hostname.

@poldom

I have tried your script (I have an emoncms 10.2.6 too), and I get a route not found error.

PI_IMPORT={"success":false,"message":"406 Not Acceptable. Route not found"}

Buhay’s suggestion is pointing at the local, as in the “same machine” since it uses the IP 127.0.0.1 (aka “this same” host as the api call is issued from) so if your local emoncms is on another device on your LAN you would again need to replace “127.0.0.1” with the IP or hostname of your local emoncms device.

Always try your api calls from a browser to test if https://IP.OF.MY.EMONCMS/feed/value.json?id=7&apikey=APIKEY works from a LAN connected browser, it should work fine in your script.

[dlmpryor (Buhay)]

Like Paul said use your browser. Following is my results:
Read API Key xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Using Chrome
http://192.168.1.92/emoncms/feed/value.json?id=24 -H “Authorization: Bearer xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx”
0

Using Firefox
http://192.168.1.92/emoncms/feed/value.json?id=24 -H “Authorization: Bearer xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx”

success false
message “406 Not Acceptable. Route not found”

[poldom (Pol Domingo)]

@pb66

Do you mean local “on the same machine” or “on the same LAN” to the device you are issuing the api call?

I mean “the same machine”. I point to 127.0.0.1.

@pb66

This is pointing at emoncms.org, not your local emoncms. You need to swap out the “emoncms.org” to your local emoncms IP or hostname.

Yes, sorry. Bad example. I’d been trying both pointing to my loopback address and to emoncms.org

@pb66

Always try your api calls from a browser to test if https://IP.OF.MY.EMONCMS/feed/value.json?id=7&apikey=APIKEY works from a LAN connected browser, it should work fine in your script.

Ok, calling it from my browser it works fine (using http in spite of https). Thanks! I’ll investigate why it didn’t work before.

@dlmpryor

success false
message “406 Not Acceptable. Route not found”

Yes, I get the same message.

How can I transform:
https://IP.OF.MY.EMONCMS/feed/value.json?id=7&apikey=APIKEY
To get the last month data, from day 1 to today? I suppose I’ll need to calculate the milliseconds from this date, and use something like this:
https://IP.org/feed/data.json?id=7&start=UNIXTIME_MILLISECONDS&end=UNIXTIME_MILLISECONDS&interval=10&apikey=APIKEY

Thanks a lot, I have been trying it for a lot of hours and I really appreciate some help.

[dlmpryor (Buhay)]

Pol,

Here some links to example scripts.
They may or may not help.
I referenced them because I am thinking about using scripting to send a email alert when Input Threshold is Reached.

Automatic backup

No, sorry, my main system is on Proxmox so I use that to backup the container nightly.

Need to reset kWh manually every two months

So, I have been able to create a cron job that submits a call to an input. That input then zero’s out my 3 feeds. I haven’t tested it extensively, but it seems to work. Here is the cron job I have set up. It outputs a message that cron then emails to me. It’s a bit unnecessary but it’s only once a month: 0 0 27 * * if [[ $(/usr/bin/curl --data "node=cron&data={flag:2}&apikey=XXXb5b4e358bd451001095" "http://10.92.68.198/emoncms/input/post" -s -o /dev/null) -eq 0 ]] ; then echo "emoncms reset s…

Collecting Local EmonCMS debug information

Recently, I came across multiple posts in which users are willing to troubleshoot. When troubleshooting the first time myself, I also asked the question which commands and log files are interesting to look at. Initially conceived as a very rudimentary quick and dirty fix script, it evolved into something more refined. Or so I hope I tried to keep the skill level required to use the script as low as possible. That being said, you should at least be able to: be able to log in on…

[poldom (Pol Domingo)]

Thanks @dlmpryor and @pb66 , this last example (wathdog script) helps me to learn how to work with different time values.

Now I’m able to retrieve the last feed value, but I still have problems when I try to recover the old ones. For example, if I call:

http://192.168.1.17/feed/average.json?id=6&start=1609051779&end=1609134592&interval=10&apikey=MY_API_KEY

I get:

That contains the correct time values, but the feed data is always a null value.

[TrystanLea]

Hello @poldom a good way of working out the correct API request is to first open the feed using the emoncms graph interface and then open your browsers Developer Tools > Network tab to see the requests made to create the graph.

You may need to click the ‘Reload’ button on the graph interface to see the first request:

[poldom (Pol Domingo)]

Thanks @TrystanLea, it works!

Finally, after a lot of hours trying it, I have a new world of experiments to try.

One last question. Is it possible to generate a new feed, similar to the “use_kwh” one, but containing the data from a range of hours (for example, the energy used from 12:00 to 14:00)? Or it will be easier to retrieve this data from the general “use_kwh” feed?

[TrystanLea]

Hello @poldom, the emoncms time of use app uses an API that I realise is not documented that may help with what you want to do. Again if you use the browser network tab you can see how it works.

The steps to load the time of use app are: Navigate to Apps, create new (+ New) then select the time of use app. Select a consumption feed and a cumulative kWh feed.

Note the paramaters in the /feed/data requests made.

http://emonpi.local/feed/data.json?apikey=APIKEY&id=1&start=1606608000000&end=1607644800000&mode=daily&split=[0,1,7]

The result gives the cumulative kWh value at midnight, 1am and 7am for each day. You can then work out what the consumption between 7am and 1am is by subtracting those values.

[poldom (Pol Domingo)]

@TrystanLea

The result gives the cumulative kWh value at midnight, 1am and 7am for each day. You can then work out what the consumption between 7am and 1am is by subtracting those values.

Thanks @TrystanLea, the split option is exactly what I was looking for. I had tried the “time of use” app, but I never looked at the network tab.

Well, now I have sufficent tools for my project. If I’m able to develop it I’ll post it here, maybe it can be useful for others.

Thanks again, I’m always proud of my emonpi, It’s a kind of proof of what open software & hardware can do.

[poldom (Pol Domingo)]

One last question. Is it possible to ask the database for the value of a feed at a specific time?

I had been looking for it at https://emoncms.org/site/api#feed but I have only found how to retrieve the last data, or the data between an interval.

[TrystanLea]

Hello @poldom

You can do this with the /feed/data/ api, its a bit convoluted but it does work:

Set the start time to the timestamp that you wish to request in milliseconds and the end time to 1s later. Set the interval to 1.

http://localhost/emoncms/feed/data.json?id=195&start=1609598790000&end=1609598791000&interval=1

The result will be two values:

[[1609598790000,18.899999618530273],[1609598850000,18.899999618530273]]

the first value is the value corresponding to the requested time.

To make this a bit easier I’ve just pushed a new api to the emoncms master branch for use initially on local emoncms installations. The api is:

/feed/value.json?id=1&time=1609598790

the result gives the value only.

[poldom (Pol Domingo)]

Thanks @TrystanLea, I’ll try to download the changes from github , and if I cannot do it I’ll use the other solution.

I’m really pleasantly surprised by the speed in developing code of this project…