Is there an easy way to bring in the data from the EmonPi into something like grafana by way of InfluxDB? I have multiple dashboards in Grafana for other stuff and would like to integrate the power consumption into some of those graphs and dashboards. Ideally, Grafana has a data source plugin that can read the TSDB on the emonpi directly.
Welcome, Ian, to OEM.
My colleague @Bill.Thomson uses Grafana, I think he’ll be able to help you (and he’s in Oklahoma).
Hi Ian, and welcome aboard.
I use Influx and Grafana, and I’ve even got an emonPi, but I’ve not had a reason to get data from
the emonPi and send it to Influx.
(I use the emonPi as a test bed, but do my energy monitoring with another device)
I have done it the other way around, i.e. read the Influx database and send that data to the emonPi.
The emonPi uses a custom database, (it’s part of emonCMS) so as far as I know, no plug-in exists that does the job you’re asking about.
Someone may have a script to do that.
What this does suggest to me though is writing an emonhub Interfacer to send data to Grafana InfluxDB.
Perhaps also an Input Process to do the same.
@TrystanLea, @glyn.hudson thoughts?
That would be the suitable location to do it. Happy to accept a pull request if someone can put this together.
Grafana needs a data source, which is often InfluxDB, but could be Graphite, Prometheus or one of many other databases. So that seems to indicate an emonHub interfacer would need to send data to InfluxDB
(or the database in use, if not Influx) vice directly to Grafana.
Perhaps not the ideal solution, but a Python3 module exists for sending data to Influx, which Grafana could then easily read said data.
More info here: Getting Started with Python and InfluxDB | InfluxData
Yes of course it does and that was actually what I was thinking of (post edited).
Im doing this . Its not pretty but works well.
My process is thus
Python script Polls local modbus tcp dev’s
This Node reports to Emonhub via socket interface.
Emon Hub template applies feed names as needed.
Emon Hubs posts to a centralized Mqtt server
I then have a python script that subscribes to the mqtt topic emon/
data from mqtt now has all relevant tags ie emon/lounge/irms1/x
I then write this directly into an influx DB via same python script.
Grafana has access to the influx data source .
Happy days Data is in Emoncms / emon hub and Grafana / Influx
You can also do some very complex routing in emon hub. ie also send data to a 2nd server via http/s
or use this trick to fettle data into multiple user accounts.
i should add im currently sending approx 100 datapoints per poll once a second per device
I know i use emon hub in a very non standard way but once configured is fire and forget and does not rely on node red etc…
This has been on my emonhub todo list since Trystan indicated he wasn’t keen on a influxdb engine for emoncms in this thread “InfluxDB as new timeseries engine”. I really want to try and use influxdb/grafana, reading the post linked by Bill, bulk upload to influxdb should be pretty straight forward.
2 Likes
However, since emon maintains its own time series database, it seems silly to duplicate the data and feed it to influx and then grafana… It would make more sense for grafana to read the existing time series data directly from emoncms and avoid the data duplication.
Indeed it would. But, as no plug-in exists to do the job, the only other way to accomplish that is
some type of “workaround.”
Agreed and look forward to you code to do that 
In the meantime, InfluxDB would have to be used.
What is the TSDB used in emon?
Is it still on the Todo list? Has it moved up? 
.
It is very much still there, it has indeed increased in priority as v2 is out now. But as much as I want to get stuck in, there are still more pressing things to be done. I have downloaded the latest binaries and found they aren’t really that RPi friendly and adding it to one of my cloud servers would ideally require other maintenance beforehand, so it’s kinda stalled again.
You mean InfluxDB? Why not go the Docker route?
Personally I have repurposed an old Laptop, upped the memory, installed ProxmoxVE and now run most things (including InfluxDB) in their own container on that. I now have a load of spare RPis lying around .
It was really easy to do. Built-in UPS and backup is so simple.
I see there is a Python InfluxDB client library, I’ve worked out putting the received data into a JSON structure with a timestamp (MQTT Interfacer) so I’m thinking it will not be that hard to do the InfluxDB bit (famous last words).
I’ll stick it on the ToDo and let you know if I do actually start it.
[edit]
I found this post Writing Data to InfluxDB with Python | InfluxData looks like the HTTP interfacer might be the best starting point interfacing to the InfluxDB database rather than emoncms database, using the same batch idea and the line protocol.
I had been using the Python library you mentioned, then decided to give Telegraf
(from the InfluxDB folks) a try. Kicked the Python script to the curb.
It’s a single file install. Setup consists of editing the telegraf.conf file which is quite well
self-documented. (docs that go further in-depth are available as well)
Lots of input and output plug-ins are available. (JSON, MQTT, HTTP and Influx Data Base among them) Too many to list here. (link below) Just for the hell of it, I installed Telegraf and the modbus input plugin on an emonPi that already had InfluxDB/Grafana installed on it. Works like a champ, doesn’t eat much memory, and isn’t a CPU hog.
Plugin list: Plugin directory | Telegraf Documentation
I have it querying a WattNode kWh meter via modbus at 5 second intervals. Linux top shows
0.3% CPU and 5.5% memory load respectively.
Might be worth a look. 
1 Like
Apparently influxdb v2 has all the ancillaries (telegraf, Chronograf and Kapacitor) in one seamless package with a new sql-like query and scripting code called flux.
1 Like
No doubt about it.
It’s been out of Beta testing for quite some time now. They’ve changed some things though.
Oddly enough, they use the term bucket (reminds me of Robin’s energy bucket) and as you mentioned
Flux is the new “language.” It’s closer to SQL than the v1.x vesion of the language was, but despite
all of the new bells and whistles, if it aint broke don’t fix it, so I’m still using v1. 