Reading zero values from a feed via the API returns an error string

Q-Branch · 2018-01-18T12:26:38.026Z

Hello all,
I have discovered a new and strange problem ever since I upgraded to the latest firmware on my emonpi.

I have a script which curls a series of values from the pi, which runs locally and fetches the values from 127.0.0.1 . If any of the values in the feed are “0”, then rather than returning a numeric “0” character, the script gets a weird error string returned which looks like this:

{success:false,message:Username or password empty}

The shell script (which I have cut down a bit to make it more manageable on the forum) is as follows:

#!/bin/bash
#
#Enter the API Keys here
LOCAL_RO_KEY='xxxxxxxxxxxxxxxx'
#
#Set the Feed IDs here
PI_IMPORT_FEED_ID='8'
PI_IMPORT_KWH_FEED_ID='9'
PI_IMPORT_KWHD_FEED_ID='16'
#
PI_GRIDPOWER_OFFPEAK_FEED_ID='33'
PI_GRIDPOWER_ONPEAK_FEED_ID='32'
#
#Fetch the data from the feeds on the emonpi
#
PI_IMPORT=$(curl -s http://127.0.0.1/emoncms/feed/value.json?id=$PI_IMPORT_FEED_ID&apikey=$LOCAL_RO_KEY)
PI_IMPORT_KWH=$(curl -s http://127.0.0.1/emoncms/feed/value.json?id=$PI_IMPORT_KWH_FEED_ID&apikey=$LOCAL_RO_KEY)
PI_IMPORT_KWHD=$(curl -s http://127.0.0.1/emoncms/feed/value.json?id=$PI_IMPORT_KWHD_FEED_ID&apikey=$LOCAL_RO_KEY)
#
PI_GRIDPOWER_OFFPEAK=$(curl -s http://127.0.0.1/emoncms/feed/value.json?id=$PI_GRIDPOWER_OFFPEAK_FEED_ID&apikey=$LOCAL_RO_KEY)
PI_GRIDPOWER_ONPEAK=$(curl -s http://127.0.0.1/emoncms/feed/value.json?id=$PI_GRIDPOWER_ONPEAK_FEED_ID&apikey=$LOCAL_RO_KEY)
#
echo "--"
echo "This is the raw data..."
echo "PI_IMPORT=$PI_IMPORT"
echo "PI_IMPORT_KWH=$PI_IMPORT_KWH"
echo "PI_IMPORT_KWHD=$PI_IMPORT_KWHD"
#
echo "PI_GRIDPOWER_OFFPEAK=$PI_GRIDPOWER_OFFPEAK"
echo "PI_GRIDPOWER_ONPEAK=$PI_GRIDPOWER_ONPEAK"

In this case, the “PI_IMPORT” variable should get a “0” because the sun is out and the panels are generating nicely. The values of “PI_GRIDPOWER_OFFPEAK” and “PI_GRIDPOWER_ONPEAK” should return “0” for the same reason. However, what I get is the following:

--
This is the raw data...
PI_IMPORT={"success":false,"message":"Username or password empty"}
PI_IMPORT_KWH=4536.7319325912
PI_IMPORT_KWHD=27.126647777821
PI_GRIDPOWER_OFFPEAK={"success":false,"message":"Username or password empty"}
PI_GRIDPOWER_ONPEAK={"success":false,"message":"Username or password empty"}

I use the values later in the script, and the fact that they’re not numeric really messes up what I am doing. This used to work fine without any issues, so something has apparently changed in the update.

Can anyone help me to understand if this is a bug that needs fixing, or whether I need to take a different approach?

Thanks,
Rich

pb66 · 2018-01-18T13:17:55.706Z

Q-Branch:

This used to work fine without any issues, so something has apparently changed in the update.

Can any help me to understand if this is a bug that needs fixing, or whether I need to take a different approach?

Can you try the same api calls from a browser session to see if you get the same results, I expect you will but please check just to be sure and check you are not logged in within the browser when you do test.

Assuming this is a bug I expect it to get fixed once confirmed, however rather than calling 5 requests, you could also do it in one.

REPLY=$(curl -s http://127.0.0.1/emoncms/feed/fetch.json?ids=$PI_IMPORT_FEED_ID,$PI_IMPORT_KWH_FEED_ID,$PI_IMPORT_KWHD_FEED_ID,$PI_GRIDPOWER_OFFPEAK_FEED_ID,$PI_GRIDPOWER_ONPEAK_FEED_ID&apikey=$LOCAL_RO_KEY)

should return a list like this

[12.3,45.6,78.9,1.23,456]

Q-Branch · 2018-01-18T17:13:09.447Z

Hello Paul,
Thank you for your quick reply.

I am not able to do the same test from a browser, because the emonpi is a headless machine and so I don’t have a browser in which to do an API call to localhost / 127.0.0.1 .

However, I was able to perform the test from a browser in another machine, which was not logged in, and it returned “0” as expected. To help you out I also tried a third test, from a different linux machine again, using the same curl command line mentioned above, but using the IP address of the emonpi, did generate the same error message as reported above. So perhaps this problem is specific to a localhost API call, or something odd with the way that curl works?

Thank you also for the guidance on fetching multiple values in one call. I will give that a go and it should help to tidy up my script.

Kind regards,
Rich

pb66 · 2018-01-18T17:29:29.478Z

Q-Branch:

So perhaps this problem is specific to a localhost API call, or something odd with the way that curl works?

At this point I would say that is more likely, but I cannot be 100% sure.

borpin · 2018-01-21T20:06:30.117Z

Q-Branch:

"Username or password empty"

I can only see this in the usermodel.php file (search on GitHub repository).

It looks like the apikey was not recognised. All I can suggest is editing that file locally to output extra logging and see where the call is going. I don’t know enough about the structure or curl to offer any deeper insights.

Is there anything internal, that makes the feeds that work one type of feed and the ones that do not a different type of feed?

Q-Branch · 2018-01-22T09:37:53.385Z

Hello Brian,
no there was nothing specific to these feeds, other than they were “0” in value at the time they were being read.

I decided to work around the problem for the time being, so I added some additional processing in the script to detect for this specific problem. I doubt that this will be useful to anyone else because it’s so specific to what I am doing but, just in case it ever becomes helpful to anyone, here is what I added:

if [[ “$PI_IMPORT” =~ “Username or password empty” ]]; then
PI_IMPORT=0
fi
if [[ “$PI_IMPORT_KWH” =~ “Username or password empty” ]]; then
PI_IMPORT_KWH=0
fi
if [[ “$PI_IMPORT_KWHD” =~ “Username or password empty” ]]; then
PI_IMPORT_KWHD=0
fi
if [[ “$PI_GRIDPOWER_OFFPEAK” =~ “Username or password empty” ]]; then
PI_GRIDPOWER_OFFPEAK=0
fi
if [[ “$PI_GRIDPOWER_ONPEAK” =~ “Username or password empty” ]]; then
PI_GRIDPOWER_ONPEAK=0
fi

Thank you to everyone who offered help and advice. If it turns out to be a bug then it would be great to have it fixed. If not, I’ll just continue to work around it.

Kind regards,
RIch

chivakaa · 2018-03-18T16:01:31.161Z

Hi Rich,

Testing API calls from localhost, i notify authentication required is Bearer and is required include header calls with -H option, like this example

 curl -i http://127.0.0.1/emoncms/feed/value.json?id=9 -H "Authorization: Bearer APIKEY"

HTTP/1.1 200 OK
Date: Sun, 18 Mar 2018 12:22:58 GMT
Server: Apache/2.4.10 (Raspbian)
Content-Length: 15
Content-Type: application/json

803.96976151626

Test and share results.

Kind Regards,

Dan

borpin · 2018-03-20T21:16:28.471Z

@Q-Branch This works from my limited testing.

@chivakaa good work. The -H bit needs adding to the API Feed help page with an example.