Getting data from EmonPi to Domoticz

borpin · 2022-10-25T15:28:48.228Z

I’d like to write these data directly in my Domoticz (using MQTT or json).

You don’t ‘write’ to a system via MQTT - MQTT sits in the middle between the 2 systems.

If your Domoticz setup uses a different broker (which I suspect is the issue here), just Bridge the EmonPi Broker to the other broker and the data is immediately available.

Leonardo · 2022-10-25T15:52:54.873Z

Hello Borpin…
I’m sure I used wrong terms… I’m quite new to MQTT so I did some tests…
I can read Emonpi value with mosquitto_sub with the command:

mosquitto_sub -C 1 -h EMONIP -u user -P password -t emon/emonpi/power1

I can update on Domoticz the proper IDX with the command

mosquitto_pub -u user -P password  -h DOMOTICZIP -t 'domoticz/in' -m '{"command": "udevice", "idx": 160, "svalue": "xyz"}'

but… I really don’t know if is possible to bridge two different systems
anyway in both of them is running a MQTT Broker…

borpin · 2022-10-25T15:54:55.059Z

Leonardo:

anyway in both of them is running a MQTT Broker

Yes, what I expected.

Read this Mosquitto MQTT Bridge-Usage and Configuration

It will mean all the topics on one broker can be on the other in either or both directions so you can read the values directly.

Leonardo · 2022-10-25T16:00:00.865Z

borpin:

It will mean all the topics on one broker can be on the other in either or both directions so you can read the values directly.

Shold be really great!!!

Thanks a lot!

borpin · 2022-10-25T16:01:17.784Z

Oh an alternative of course, is to edit emonhub config to send that data to 2 different Brokers. Probably easier

Leonardo · 2022-10-25T18:38:30.619Z

borpin:

Oh an alternative of course, is to edit emonhub config to send that data to 2 different Brokers. Probably easier

I’m not yet skilled enough 'cause, as I said, I’m new on MQTT… I saw topics are very different so I’d have to see where to configure this. It’s all a work in progress, I never used Emonpi before so I’m trying to understand how it works… before it I was using a OWL CM119 with a 433MHz transceiver, for me getting Watt is enough, (voltage is an optional )

borpin · 2022-10-25T19:53:18.476Z

It is quite easy, on the Emonpi; on the LHS click on emonhub, then click on edit config.

You will find a section that says MQTT, just copy it and adjust for your other Broker.

Something like this I suggest (just change the xxxxxx)

[[MQTT2]]
    Type = EmonHubMqttInterfacer
    [[[init_settings]]]
        mqtt_host = 192.168.x.x
        mqtt_port = 1883
        mqtt_user = xxxxx
        mqtt_passwd = xxxxx

    [[[runtimesettings]]]
        subchannels = ToEmonCMS,

        timestamped = True
        # emonhub/rx/10/values format
        # Use with emoncms Nodes module
        node_format_enable = 1
        node_format_basetopic = xxxxxxxxxxx/

        nodevar_format_enable = 0
        node_JSON_enable = 0

You will probably want node_format as that publishes data to individual topics and node_format_basetopic is the base topic your other system expects.

Leonardo · 2022-10-26T08:39:17.812Z

borpin:

It is quite easy, on the Emonpi; on the LHS click on emonhub, then click on edit config.

ok I’m following your suggestion, it seems to be the right way but I have to better understand how to fix… I added the MQTT2 section and, reading on Domoticz the topic sent I can read passed values, infact it receives:
domoticz/in/rx/5/values 148.20000000000002,0,148.20000000000002,227.68,0,0,0,0,0,0,0
(the topic in Domoticz is "domoticz/in/)

the good is: values are correct

the bad is: Domoticz expects something like this:

-t ‘domoticz/in’ -m ‘{ “idx” : $IDX, “nvalue” : 0, “svalue” :$VALUE }’

$IDX is the ID of the connected device I want to update, and $VALUE is obviously the value

I think I have to write some code to convert info I get… there is in Domoticz a plugin for Shelly MQTT that uses its syntax, I think I’ll have to do something similar, changing the topic with (for example) emonhub and working on data with this topic…

borpin · 2022-10-26T09:51:20.243Z

RPi Home Automation Software

the good is: values are correct

Good

RPi Home Automation Software

the bad is: Domoticz expects something like this:

Easily solved

borpin:

node_JSON_enable = 0

Change this value to 1

Leonardo · 2022-10-26T10:07:22.803Z

borpin:

Change this value to 1

Yes yes I did, (reading over the Forum I saw you are the developer of the node_JSON)

setting it to 1 I’m getting more info… my issue now (my fault 'cause I’m not skilled enough), is to “translate” something:

I should transform this string(what I’m getting via MQTT):

emonpi {“power1”: 182, “power2”: 0, “power1pluspower2”: 182, “vrms”: 228.4, “t1”: 0, “t2”: 0, “t3”: 0, “t4”: 0, “t5”: 0, “t6”: 0, “pulsecount”: 0, “time”: 1666778083.3173008}

to this:

{“command”: “udevice”, “idx”: 160, “svalue”: “182”}
(to update watts with power1 value)

and
{“command”: “udevice”, “idx”: 156, “svalue”: “228.4”}
(to update volts with vrms value)

this is what Domoticz expects… and it expects two separate commands like:

mosquitto_pub -h 172.16.3.10 -u users -P password -t ‘domoticz/in’ -m ‘{“command”: “udevice”, “idx”: 160, “svalue”: “182”}’

and

mosquitto_pub -h 172.16.3.10 -u users -P password -t ‘domoticz/in’ -m ‘{“command”: “udevice”, “idx”: 156, “svalue”: “228.4”}’

borpin · 2022-10-26T10:17:13.118Z

Ah, OK. No that format isn’t supported by EmonHub (currently).

Potentially emonhub could be modified to provide a suitable format for Domoticz.

In addition, the bridge solution will not work either as the data is never on the Broker in the right format for you. I suspect your script or using NODE-Red would be the solution.

I think you said time was the issue with your script and having to publish so many different topics, I can see why.

You could create a Python script that monitors for a value to be published to the Broker and then reformats the data and sends it on. It would be quicker than a bash script.

Leonardo · 2022-10-26T10:31:25.616Z

borpin:

It would be quicker than a bash script.

infact here in Italy we call it “a bath of blood”
for this my first idea was reading directly from the ttyAMA0 device and use these value to update Domoticz… at the end, thinking about it, should be the same as how it works right now:
on crontab on Domoticz I put this:

VOLTIDX=156
WATTIDX=160
VOLT=`mosquitto_sub -C 1 -h 172.16.3.39 -u emonpi -P emonpimqtt2016 -t emon/emonpi/vrms`
WATT=`mosquitto_sub -C 1 -h 172.16.3.39 -u emonpi -P emonpimqtt2016 -t emon/emonpi/power1`
OUTPUT=`curl -s "http://172.16.3.10:8080/json.htm?type=command&param=udevice&idx=$VOLTIDX&nvalue=0&svalue=$VOLT"`
OUTPUT=`curl -s "http://172.16.3.10:8080/json.htm?type=command&param=udevice&idx=$WATTIDX&nvalue=0&svalue=$WATT"`

(obviously I can use mosquitto_pub to update Domoticz instead of curl)

[edit - mod - formatting] Please use the ‘Preformatted’ button to post code etc.

borpin · 2022-10-26T10:47:57.409Z

Yes, your issue is triggering it.

With Python, the script could just sit and wait for the single JSON message. When a new one is published, get it with an on_message trigger, parse it (easy enough to convert string to an object), and publish the new values.

With cron you don’t know if a new message has actually arrived (10s gap is not absolute and can fluctuate).

Leonardo · 2022-10-26T12:32:23.228Z

Other way, being able, I could set another port on the section MQTT2 (for domoticz) and, on domoticz raspberry use python (or something else) to open a socket, get data from the Emonhub MQTT, parse it and send to the domoticz MQTT standard port…
this could get and forward data quickly

borpin · 2022-10-27T08:37:29.099Z

Leonardo:

python (or something else) to open a socket, get data from the Emonhub MQTT

Just use the MQTT Python library to subscribe to the topic and use the on_message callback.

Leonardo · 2022-10-28T07:16:14.833Z

Ok Brian, I’ll try, my coding skills are poor, I’m a Linux Sysdamin and Forensic, not a developer, but I can try

Leonardo · 2022-10-28T15:12:11.624Z

ok something moves…
I did this script, I’m reading via MQTT the value of topic emon/emonpi/power1 BUT… I wasn’t able to publish on Domoticz the value via MQTT so I used a requests call.

import paho.mqtt.client as mqtt
import requests

def on_connect(client, userdata, flags, rc):
    print("Connected with result code "+str(rc))
    client.subscribe("emon/emonpi/power1")

def on_message(client, userdata, msg):
    W_decode=str(msg.payload.decode("utf-8", "ignore"))
    #print(W_decode)
    r =requests.get('http://172.16.3.10:8080/json.htm?type=command&param=udevice&idx=160&nvalue=0&svalue='+W_decode+'}')
    #print(r)

client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message
client.username_pw_set("emonpi", "emonpimqtt2016")

client.connect("172.16.3.39", 1883, 60)

client.loop_forever()

On this way it loops and gets value every seconds, the update is in realtime

borpin · 2022-10-29T12:32:15.001Z

Leonardo:

BUT… I wasn’t able to publish on Domoticz the value via MQTT

Because Domoticz uses a different MQTT Broker.

I think you need…

emonpi_client = mqtt.Client()
domoticz_client = mqtt.Client()

borpin · 2022-10-29T12:35:15.075Z

I created a new topic for this.

Leonardo · 2022-10-29T16:34:27.644Z

Ok thanks I’ll work on it and I’ll try to fix it
PS great for new topic