Wondering is there a python/perl/bash script to send csv to emoncms

stephen · 2018-09-05T23:03:59.651Z

wondering if there is a python/perl/bash script to send csv files to emoncms. was just building another back end to my openwrt IOT monitor/hub based on openwrt18. currently it supports influx and other databases even microsoft sql. though all I am doing is rsyncing CSVs to a staging area and the staging area imports the CSV for microsodt sql. some government agency wanted it in microsoft SQL for their analytical engine. I see there a node-red method but that so heavy. as that would bump up my firmware from 30 megs to about hundred megs. if someone has one, knows of one could you point me in the direction of it, no need reinventing the wheel… thank you for your time–

Bill.Thomson · 2018-09-06T00:01:04.275Z

The send-to-eorg.sh bash script in the link below should be very close to what you’re looking for.

Read the SDM220 modbus with the minimalmodbus python module Hardware

Here’s the Python script and bash file… read_meter.py (2.3 KB) send-to-eorg.sh (1.2 KB) Note: You’ll need version 2.1.2 of Advanced Python Scheduler. APS version 3.x is not compatible with the script as written. This should help with getting to know modbus: About Modbus | Simply Modbus Software

stephen · 2018-09-06T18:31:14.221Z

Thank you bill that sort of works with some modification… as Mqtt sends are just a single value in collectd

–curious does emoncms auto create node if you send them… oh well guess I will find out

Bill.Thomson · 2018-09-06T18:37:09.175Z

stephen:

wondering if there is a python/perl/bash script to send csv files to emoncms.

Oh, OK. You’d asked about sending CSVs, so that’s what I based my answer on.
Are the CSVs in a file?

stephen · 2018-09-06T20:05:12.944Z

yeah. it will read a csv file created by collectd MQTT plugin or other collectd plugin ( mqtt, modbus , other collectd plugin or custom) … then pass it on to emoncms or other database. currently I have 30 second send on to database ( but that is adjustable) . the data are logged in a csv based on day, host/ area , datatype Type ( energy, temp, pressure, humidity …etc) and location/header ( kitchen grid, solar) file at what ever rate they are sent at generally I have things set at 1-5 seconds so it will upload between 6 and 30 data points every 30 seconds. and hopefully I can get it to upload days or weeks of info in case of outages so that will be 10s of thousands of data points … be nice if I could get it to autocreate nodes but looking at it maybe that not possible with how data is sent emoncms . currently will try this weekend as it is sunny and warm and I have a lot of outdoor stuff to do- suppose to rain for the next week so I get more involved then with it