Weather data into Emoncms via Node-Red

peteclyde · 2022-07-09T08:12:26.928Z

Hi all,

So I’ve successfully setup Node-red so that I can input weather data from my Ecowitt personal weather station into my local Emoncms system (Ubuntu)

The only issue is that I’m having to load into separate input nodes as when I try to load them into one node to keep them together only one of them gets uploaded to Emoncms.

Is there a way to upload them all into one named node ?

Screenshot (24)2168×697 150 KB

Thanks for any assistance…

Pete

Vster · 2022-07-10T20:24:16.389Z

That’s how I do it.

carl_afgte45 · 2022-07-12T20:04:47.195Z

No idea about NodeRED but I see you are putting data into WeatherUnderground? Another way… I have a Bressar weather station that goes to Wunderground this (I cant directly connect to it - well not without a load of reverse engineering I don’t have time for)

I just use a few lines of Python code to query the WeatherUnderground API download and save the data (so I have my own copy of MY data) and then make a http get call to emonPI

Getting the data:
= ‘https://api.weather.com/v2/pws/observations/current?stationId=**station_id**&format=json&units=m&apiKey=**api_key**&numericPrecision=decimal’

Putting into the emonPI:
http://<PI_IP_ADDRESS>/input/post?node=wxstation&fulljson={“temp”:temp,“pressure”:pressure,“wind_av”:wind_av,“wind_dir”:wind_dir}&apikey=************

It just runs as a cron job on my home PI server ever 10 mins. Writing the code might not be your thing, but if you want to try just say and I can share

peteclyde · 2022-07-13T06:57:54.370Z

Hi Carl,

That’s a clever way of getting the data in. Unfortunately I fear that I haven’t made my issue very clear, so I’ll ty to show some screen shots to explain.

I’m not having any issues getting the data from my weather site (Ecowitt) - I then run it through some converters (the data is all in imperial and I need to convert some of it to metric etc). The issues is that I want all the data collected to go into the one input node, but alas I haven’t figured out how to do that. Hence I am importing the data into Emoncms into multiple input nodes as per my screen shots. I hope that explains it better.

The data is getting into Emoncms ok, and I can use it fine, but it’s pretty untidy in the Input screen.

Screenshot (32)1920×1323 228 KB

Screenshot (31)3840×1677 367 KB

Screenshot (30)3840×1948 408 KB

borpin · 2022-07-13T20:40:43.467Z

It is a log time since I used the emoncms flow in NodeRed - I send everything via MQTT.

Personally, I’d split the input do what was needed and then build a JSON object to send it in one go to emoncms via MQTT in the form

{"Item1":100,"item2":200,"time": UNIXTIME}

Note Javascript generates time in milliseconds and emoncms wants seconds.

Even using the push node, I’d construct a single JSON string as it is more efficient.

However, what does the configuration of each emoncms push node look like (show 2 different ones)?

peteclyde · 2022-07-17T01:43:24.120Z

Hi Brian,

Thanks for your reply - I’ve played with MQTT / Node-red without success in the past, I can get a connection and can confirm it is being sent but I can’t find the data anywhere. Obviously a rookie mistake somewhere along the line.

I’m going to just accept the current setup for what it is - all my feeds are running fine, despite the input page looking a bit messy, but whilst it’s woking I’ll leave it as is.

Thanks for your input.

Pete

borpin · 2022-07-17T19:54:51.039Z

peteclyde:

Thanks for your reply - I’ve played with MQTT / Node-red without success in the past, I can get a connection and can confirm it is being sent but I can’t find the data anywhere.

Did you connect to the MQTT Broker on the Pi (using the username and password) and then use the base topic emon/?

borpin · 2022-07-17T19:55:29.242Z

peteclyde:

I’m going to just accept the current setup for what it is - all my feeds are running fine, despite the input page looking a bit messy, but whilst it’s woking I’ll leave it as is.

I’m sure it can be sorted, I did ask…

borpin:

However, what does the configuration of each emoncms push node look like (show 2 different ones)?

Vster · 2022-07-19T16:40:48.396Z

What setting have you got in the EmonCMS push node? They absolutely can go into the same input node.
From memory you have to set the title as the input within the node and use the legacy mode.

image1324×466 29.8 KB

peteclyde · 2022-07-22T01:47:13.556Z

So I’ve followed the above as best I can, but the input names are all ‘1’ - they populate x 3 the first time the functions are run, but after that only one of them updates. How do I change the output title from ‘1’ to either text or alternate numbers to ‘1’ ?

Screenshot (39)2514×834 173 KB

Screenshot (40)3814×549 84.7 KB

Jaddache · 2022-07-22T03:33:04.434Z

Hi Pete,
The help info for the node you are using (the book button in the top right of your node red editor) indicates the node needs to be fed name value pairs by the preceding function node.

image548×535 13.7 KB

The name should then appear in the node input list.

As @Vster indicated above, and this is the way I do it, you can use the mqtt out node to achieve the same outcome, where the mqtt topic is of the form emon/[node_name]/[input_name] and a simple value is in msg.payload.

peteclyde · 2022-07-22T07:58:29.099Z

I’ve previously tried mqtt to no avail, it’s showing as connected but nothing appears in the emoncms inputs.

Is there anything further that I have to change within emoncms to ensure mqtt is active? I’m running Ubuntu in a VirtualBox.

Screenshot (5)2064×1015 177 KB

Screenshot (6)2582×657 209 KB

Checking via command line:

mosquitto_sub -v -u 'emonpi' -P 'emonpimqtt2016' -t 'emon/#'

shows that data is being correctly received but I can’t see the data anywhere in emoncms

Screenshot (7)1598×230 10.2 KB

I feel that I’ve missed something very simple in the setup…

Jaddache · 2022-07-22T10:39:45.391Z

Pete, Looking for something simple - have you set up the mqtt-broker configuration node in node-red to point to the emoncms mqtt broker?
My mqtt nodes look like this. Note that the node name in the topic is not the node number.

The mqtt-broker setup looks like this, with the mqtt username as password as you have been using in the security tab:

image821×801 35.4 KB

borpin · 2022-07-22T20:11:34.780Z

peteclyde:

shows that data is being correctly received but I can’t see the data anywhere in emoncms

I note you appear to have run that command on your VirtualBox and you do not specify a host, so it will assume localhost.

I suspect you have 2 Brokers running, one on the VB and one on the EmonPi.

In Node Red you need to connect to the EmonPi Broker.

Jaddache:

The mqtt-broker setup looks like this, with the mqtt username as password as you have been using in the security tab:

@Jaddache Only correct if NodeRed is installed on the EmonPi, which I suspect it isn’t in this case.

[edit]
It is much better to build a JSON String and send it as a single publish command. Too many messages to EmonCMS will cause it to drop some as the Input Script cannot keep up.

peteclyde · 2022-07-23T00:18:54.743Z

Thanks for your replies gents.

Node-red is on the same ubuntu server as the emon system - 192.168.1.101

I’ve tried IP address and local host - same result. Like I posted earlier, the result of -

mosquitto_sub -v -u ‘emonpi’ -P ‘emonpimqtt2016’ -t ‘emon/#’

shows data being received by mqtt (at least that’s how I read it) - it’s just not appearing inside Emoncms.

Screenshot (9)1327×1025 13.7 KB

Screenshot (8)1045×970 10.3 KB

borpin · 2022-07-25T06:03:56.352Z

peteclyde:

shows data being received by mqtt (at least that’s how I read it) - it’s just not appearing inside Emoncms.

Ok so your Emoncms setting need to be updated.

How did you install the setup?

Post the [MQTT] sections from the files

/var/www/emoncms/settings.ini
/var/www/emoncms/default-settings.ini

Do NOT edit the default-settings.ini but ADD the additional elements to the settings.ini file.

Note, you mat not have setup the user/password on your MQTT Broker on that server (depending how you installed it).

Personally I have Mosquito, Node-RED and Emoncms in separate containers not on one VM system.

peteclyde · 2022-07-26T00:19:32.573Z

Arrrh Brian, you are a legend, thank you for persevering with me on this topic.

I had previously checked the settings.ini file and had seen that it showed:

[mqtt]
enabled = true
user = ‘emonpi’
password = ‘emonpimqtt2016’

Which to me indicated it was set up and should have been ok. I have updated the settings.ini file now to:

[mqtt]
enabled = true
user = ‘emonpi’
password = ‘emonpimqtt2016’
host = ‘localhost’
port = 1883
basetopic = ‘emon’
client_id = ‘pete’
userid = 2
multiuser = false

Because everything else was working fine I presumed that all the default settings were ok. All is good now and I can keep working on the project, thanks again.

Pete