Community
OpenEnergyMonitor

OpenEnergyMonitor Community

Read from local emoncms and post data to emoncms.org with node-red

Tags: #<Tag:0x00007fc9b6647cd8>

Hi all,
I am trying to mirror some of my emoncms local server data to emoncms.org server with node red.
I have a working sketch running, I can read from local server and post data, but I can only read 1 feed and post it to 1 node of the remote server.

I would like to read 4 feeds of my local emoncms and post them to the same node of emoncms.org, i don’t know how to merge the data into one string, can you help me?

Hi all, finally i found a solution to this, I build a string with the node red join node, and then it’s posted to emoncms.org.

Hi Ernest,

You can set up emonhub to forward data to emoncms.org as well as your local instance of emoncms.

Was there a reason you decided not to do that?

Hi Bill,

My first intention was to use the emonhub mirroring, but I can’t use emonhub because I have arduinos posting directly to a local emoncms server with a raspi and emonpi. This is the topic talking about that.

Thanks.

Here’s a bash script that reads feed data from a local instance of emonCMS
and forwards it to emoncms.org via curl.

#!/bin/bash
#
RO_KEY='your-local-emoncms-read-only-apikey'
RW_KEY='your-emoncms.org-read-write-apikey'
#
# change FF to the numerical id of the feed you wish to read data from
#
A=$(curl -s "http://127.0.0.1/emoncms/feed/value.json?id=FF&apikey=$RO_KEY")
B=$(curl -s "http://127.0.0.1/emoncms/feed/value.json?id=FF&apikey=$RO_KEY")
C=$(curl -s "http://127.0.0.1/emoncms/feed/value.json?id=FF&apikey=$RO_KEY") 
#
A=${A//\"} # strip quotes from json string
B=${B//\"}
C=${C//\"}
#
# change NN to the number of the node you want to send data to
#
curl -s "https://emoncms.org/input/post?node=NN&csv=$A,$B,$C&apikey=$RW_KEY" > /dev/null 2>&1
#

get-data.txt (638 Bytes)

Delete the .txt extension, or change it to .sh

1 Like

Hi Bill and thanks for your time, I am lost here, I don’t know what is a bash script and where to put it?
How will it mirror the data without api keys, or server ips? Sorry I am a novice :grin:

Thanks.

Not a problem, we were all newbies once.

Here’s some info to get you started with bash scripts:
https://ryanstutorials.net/bash-scripting-tutorial/bash-script.php

I did forget to include that part. I’ve edited my post to include the API keys.

Thanks Bill,

I got it! I just created get-data.sh script on my raspi and when I execute it I get the inputs on emoncms.org without node-red, only one more question, how to auto execute the script every 1s to have the emoncms.org inputs updated?

Thanks again.

Keep in mind there’s a limit to how often you can send data to emoncms.org/
If memory serves, that limit is once every 10 seconds.

Here’s a Python script that should do the job.
I’ve attached APS as well as the Python script.
You’ll need to install Advanced Python Scheduler with pip.

APScheduler-2.1.2.tar.gz (40.8 KB)
no need to unzip the tarball. pip can install APS directly from the archive.
There is a newer version of APS available, but I prefer this one. The newer one (v3) isn’t as simple to use as this one (v2) is, so I stayed with the older version.

jobsked.txt (377 Bytes)

Thanks Bill, I have it working.

It’s autoupdating correctly but if I close the ssh terminal window it stops running. How to configure it to autoexecute the python script, also if the raspi is rebooted?

Do you thing this method is better than the node-red one? More clean? More secure?

Thanks.


Here’s
info on how to autostart the Python script. Using that method, it’ll restart after a reboot.

I can’t say, really. The best tool to use is usually the one you’re most familiar with.
So, for me, that turns out to be bash and Python, vice NodeRED.

Cleaner?
As in simpler? Yes.
But for those who aren’t up to speed on writing bash and Python scripts,
NodeRED is usable. i.e. NR makes it easy for a user to get the job done.

More secure?
Because it’s not as complex as NodeRED, I’d say at least somewhat.
But if somebody wants to exploit a machine/OS/app etc. they’re going to do it anyway.

Thanks Bill,

I get it working! :wink:

I think this situation you should use Python as an alternative scripting language which is simpler and easier for beginners.
But if you want to work with Bash, this tutorial will be a great start for bash scripting.

One thing to mention regarding naming your Bash file, you don’t need to rename it from .txt to .sh and that’s because the only thing needed is to make it executable regardless the extension. even if the file without the extension at all.

Regards,

Quite true. As you said, an extension isn’t necessary. But, it’s common convention to name a bash script with a .sh extension. That way, months later when you go back to look at it, modify it, etc. you know immediately its a shell script. Similar in principle to naming a Python script with a .py extension.

Good day Bill,I’m Christian. I have a data logger with raspberry pi that takes data by serial port(ttyAMA0) and after a script post everything on local emoncms and it works properly.l have followed the OEM guide for post data on emoncms.org,changing emonHub.conf, but some problems appeard:
"
2018-04-26 19:58:32,873 INFO EmonHub Pre-Release Development Version (rc1.2)
2018-04-26 19:58:32,875 INFO Opening hub…
2018-04-26 19:58:36,895 ERROR Unable to create ‘RFM2Pi’ interfacer: Attempting to use a port that is not open
"
Could your script works in my project?
Which is folder installation and owner?

Thank you in advance for your information.
WBR
Christian

What is it you are trying to do?

Your other thread

shows a script that posts direct to emoncms, therefore editing emonhub.conf will have no effect since the data is not flowing through emonhub, it is going direct from your script to emoncms.

if you are simply trying to post data to 2 emoncms instances you could just add a second target to your script so that it posts to 2 emoncms’s directly.

The error you see in emonhub is most likely because you are using the serial port for your datalogger and emonhub is still configured to use the serial port for a rfm2pi or emonpi, removing the “rfm2pi” section from emonhub.conf should resolve that.

Bill’s script would undoubtedly work, but it is going around the houses a bit, there is no reason to post the data to one place and then use another script to read the data from there and post it elsewhere.

The easy option would be to edit your first script to post to 2 places, the best option would be to change your script so that it sends to an emonhub socket and then emonhub can send it to as many emoncms’s as you like and offer error logging and data buffering to boot.

Good day Paul,the first thread concerned the data post on local emoncms and was solved.
This is related to issue that I have now for to post the data on emoncms.org.

" removing the “rfm2pi” section from emonhub.conf should resolve that"
done but not solve.

Thank you Paul I add the new line in my personal script,as you suggested me in your reply and now I can see the data on emoncms.org.
SUPER
Paul, has been a fight this installation…but now I’m winner!!!
Many thanks again

1 Like