Logging data to EmonCMS

Emoncms
1

Hi,

I have an EmonPi system which I am using to log data from my solar panels, and readings from a couple of EmonTH sensors. I’ve been meaning to find a way of logging data from other sources into EmonCMS. For example, the temperature and humidity readings from my Tado heating controller, which can be read by an API, and the data from my MyOlive oil tank monitor which would have to be scraped from their website.

I’ve got a Raspberry Pi that I could use to run some scripts. I’ve written scripts and programs in the past, but haven’t done so for quite a while. What’s the best way to get started?

2

Personally, I find the easiest way to input the data is via MQTT. Just use the base topic of emon, the next level will be the node and the final level will be the input so a topic of emon/mytank/temp1 for instance.

I find Node-red the best mechanism for manipulating the data. You can write javascript functions which I tend to do rather than using nodes to create flows.

The other advantage of MQTT is that other things (like Home Assistant) can pick up that same data from the MQTT broker.

A word of warning if you are building multiple things to use the same data, have a dedicated MQTT broker inside your network for everything to use. The one installed along with EmonCMS would be fine.

3

I just write a script that reads the data source and sends it to emoncms via the input API using the fulljson format. I’d suggest to first choose whichever programming language you prefer (I use perl) then write some code to read data from your sources and just print it. Get that working then add code to format it as JSON in the appropriate format and print that. Finally add code to call the emoncms API. With some if statements around the printing, you’ll be able to debug any teething problems easily. Finally put the script in a cron job to run regularly.

I have scripts to upload data from my solar system’s web page and to upload weather data from a local weather station and from the Met Office.

4

Here’s a Python example. The script reads RS-232 data from a thermocouple interface:

#!/usr/bin/python
import serial, requests, logging
from time import sleep, time
logging.basicConfig(filename='/tmp/stove.log',level=logging.INFO)
ser = serial.Serial('/dev/ttyUSB0', 9600, timeout=1)

def every(delay, task):
  next_time = time() + delay
  while True:
    sleep(max(0, next_time - time()))
    task()
    next_time += (time() - next_time) // delay * delay + delay

def get_data():
  #send command to thermocouple interface to read temperature
  ser.write('$1RD\r')

  #consume full duplex echo
  r = ser.readline()

  r = ser.readline().lstrip('*+0').rstrip('\r\n')
  t = ('Stove temperature={} {}\n'.format(r, int(time())))
  requests.post('http://192.168.1.61:8086/write?db=stove&precision=s', data=t)
  
every(2, get_data)

Notes:
A RasPi 2 ingests data via a RS-232 to USB device.
The every() function calls the get_data() function every 2 seconds. (Configurable to any interval)
The Python module Requests is used to send the data to a local instance of InfluxDB via HTTP.
It would be easy to change the call to send data to an instance of emonCMS.
There is no error handling.

There’s a Python library called Beautiful Soup that pulls data from HTTP / XML files.