Posting to emoncms via python from local pi

Emoncms
1

Hi,

I have a newly installed emoncms running on ubuntu server (192.168.1.2) and a raspberry pi with a working dallas temperature sensor attached on the same network

On a PC on the network the following URL works in browser to update the temperature feed on the emoncms with an arbitary value (20 here)…

http://192.168.1.2/emoncms/input/post?node=1&json={‘Temp’:‘20’}&apikey=1871xxxxxxxxxxxxxxx120

But testing from the Pi by running the following python script, i am getting a 200 OK message back from emoncms but the feed is not updating, any ideas please, I am stuck…

import os                                                  
import glob                                                
import time
import urllib
import httplib
import json

key = "187xxxxxxxxxxxxxxxxxxxxxx120"

while True:

   params = urllib.urlencode({'Temp' : '20','&apikey=':key})

   headers = {"Content-typZZe": "application/x-www-form-urlencoded","Accept": "text/plain"}

   conn = httplib.HTTPConnection("192.168.1.2:80")

   try:

        conn.request("POST", "/emoncms/input/post.json?node=1", params, headers)
        response = conn.getresponse()
        print response.status, response.reason
        data = response.read()
        conn.close()
2

It seems unlikely that this is the problem, but why is there two uppercase Z’s in the "Content-Type" string?

headers = {"Content-typZZe": "application/x-www-form-urlencoded","Accept": "text/plain"}
3

dont know how that got there :slight_smile: but its not in my test.py script…

4

[ignore - talking rubbish]

5

I usually use MQTT to fire across data like this rather than HTTP. If Node-Red is installed on the Pi just use the emoncms node.

6

unfortunantly, the pi is running libreelec (kodi) it’s locked down tight !

7

i was just replying to your thread, fair enough :). One question though if poss, is there a way of the logs showing everything (debug) just not errors?

8

I’ll try again.

In the input API help there is an example of a curl command using the json input. Does that work?

I think this line is wrong

params = urllib.urlencode({'Temp' : '20','&apikey=':key})

Looking at the curl example, the params should possibly be

"node=1&data={'temp':'100'}&apikey=c8xxxxxxxxx"

and

conn.request("POST", "/emoncms/input/post.json", params, headers)

However, I’d just post to MQTT - far simpler code.

YMMV

9

Yes - change it in settings.php

10

hi

followed advice tried following code, but now getting … "

    File "/usr/lib/python2.7/urllib.py", line 1328, in urlencode TypeError: not a valid non-string sequence or mapping object"

import os                                                  
import glob                                                
import time
import urllib
import httplib
import json

while True:

   params = urllib.urlencode("node=1&data={'Temp':'22'}&apikey=18xxxxxxxxxxxxxxxxxxxxxx20")

   headers = {"Content-type": "application/x-www-form-urlencoded","Accept": "text/plain"}

   conn = httplib.HTTPConnection("192.168.1.2:80")

   try:

        conn.request("POST", "/emoncms/input/post.json", params, headers)
        response = conn.getresponse()
        print response.status, response.reason
        data = response.read()
        conn.close()
11

I don’t use urlllib myself so I apologize if I’m wrong, but I think urlencode is expecting a list of key:value parameters so something along the lines of

params = urllib.urlencode({"node:1,data:{'Temp':'22'},apikey:18xxxxxxxxxxxxxxxxxxxxxx20"})

will probably avoid the error you are currently seeing, but whether that works with emoncms is another thing all together.

Is there any particular reason for using “POST” rather than the usual “GET” requests? I know it seems logical to use “POST” to “post” data to emoncms, but data is prodominantly posted via standard “GET” requests as per your testing in the browser. I think emoncms will accept “POST” requests but I’m not at all familiar with that method.

12

This might be a FWIW type observation, but I noticed the post part of the URL you used with your browser
i.e. http://192.168.1.2/emoncms/input/post?node=1&json={‘Temp’:‘20’}&apikey=1871xxxxxxxxxxxxxxx120

differs from the post part of the URL you’re using in your connection request.
i.e. conn.request("POST", "/emoncms/input/post.json", params, headers)
(specifically the /post.json part)

I haven’t run emonCMS for some time, but I do remember seeing /post.json as one of the options in an
earlier version of emonCMS. You mentioned your installation was new, but didn’t say what the version
number is.

However, /post.json isn’t shown on the currect Input API page.
I don’t know if it’s simply not listed, or if it’s no longer usable.

@glyn.hudson? Is this an issue, or can it be ignored?

13

If I remember correctly, the .json part of the /post.json has been dropped and is just ignored as opposed to being blocked or erroneous, but as you say FWIW it definiatly worth dropping the .json to test that theory.

I spotted that too, there are 2 fundamental pieces of debugging info missing here. Not only is the test url different (so there is no confirmation that the used request format works), there is no printout of the formed request in the script to check what is actually being sent.

I guessed that might be because “POST” (request type, not api endpoint) was being used rather than “GET” requests, the “GET” url is both easier to print and to test in a browser (IMO - I know you can use postman or a similar utility).

14

Thanks, PB. I wasn’t sure, so thought I’d offer the suggestion.

15

Nor I, so definitely worth suggesting :slight_smile:

16

Did you try the curl example you can find on the Input APi help page?

I think this might be your answer as to why it doesn’t work. python - Why I can't use urlencode to encode json format data? - Stack Overflow

[edit] This may be the answer to fix it Python requests module: urlencoding json data - Stack Overflow

17

Good call Brian. I use Python Requests to get data from my modbus instruments. :wink:
Just for the heck of it, I decided to load an instance of emonCMS and see if I could whip up something using Requests. Here’s a snippet of the “essential” part of the script that uses the CSV input method to send data
to version 9.8.31:

import requests

apikey='xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
payload = {'csv':Data, 'apikey':apikey}
url = 'http://192.168.1.205/emoncms/input/post?node=node_name'
r = requests.post(url, params=(payload))
print r.url
print r.text # reponse text

The Requests module will need to be installed.

Here’s the script for reference:
csv.py.txt (1.5 KB)

This script uses the node:key json (i.e. not the fulljson) method to send data to emonCMS:
(but is otherwise the same as the CSV version above)
node-key.py.txt (1.5 KB)

Disclaimer:
Although these scripts do indeed work, I’m not a coder. The scripts were written as proof of concept and to give the OP an example. They have no error handling. The only thing they are guaranteed to do is take up storage space.

18

sorry, I’m only a beginner python programmer trying to get my head around some edited code i found online, been trying to follow the advice, but getting stuck with each edit :slight_smile:

I saw another simpler example which worked and managed to update the feed using the following code…

temp = "17"
url = "http://192.168.1.2/emoncms/input/post?node=1&apikey=1871xxxxxxxxxxxxxxxxxxxxxxxx20&json={'Temp':"
url = url + temp + "}" 
urllib2.urlopen(url)

only issue now is in my script the current dallas probe temperature is sent to the variable called read_temp() as a number, but if i specify it in following line …

url = url + read_temp() + "}"

I get the error: TypeError: cannot concatenate ‘str’ and ‘float’ objects ?

Edit - formatted code for readability - BT, Moderator

19

the quick fix is to cast the float value to a string like so

url = url + str(read_temp()) + “}”

there are more elegant ways to do it, but this will work.

20

thank you thank thank you … its all working now :slight_smile:

I am constrained to using this inelegant way as the dallas temperature sensors are running on a pi running libreelec which is a locked down “kodi os” missing loads of python libraries (I would have to recompile from source to add, way above my head)

now my pi libreelec media centres (used for watching netflix/youtube) are also temperature nodes at home, each dallas probe is under £2 on ebay!

for anyone interested this is my final working code (running on libreelec), now all I have to do is get my head around emoncms :slight_smile: …

import os                                                  
import glob                                                
import time
import urllib2
import httplib
import json

                                                
os.system('modprobe w1-gpio')                              # load one wire communication device kernel modules
os.system('modprobe w1-therm')

base_dir = '/sys/devices/w1_bus_master1/'                  # point to the address
device_folder = glob.glob(base_dir + '28*')[0]             # find device with address starting from 28*
device_file = device_folder + '/w1_slave'                  # store the details
def read_temp_raw():
   f = open(device_file, 'r')
   lines = f.readlines()                                   # read the device details
   f.close()
   return lines

def read_temp():
   lines = read_temp_raw()
   while lines[0].strip()[-3:] != 'YES':                   # ignore first line
      time.sleep(0.2)
      lines = read_temp_raw()
   equals_pos = lines[1].find('t=')                        # find temperature in the details
   if equals_pos != -1:
      temp_string = lines[1][equals_pos+2:]
      temp_c = float(temp_string) / 1000.0                 # convert to Celsius
      return temp_c


temp=str(read_temp())

while True:

    url = "http://192.168.1.2/emoncms/input/post?node=1&apikey=187xxxxxxxxxxxxxxxxx120&json={'Temp':"

    url = url + temp + "}"

    urllib2.urlopen(url)

    time.sleep(300)