Anyone?
The truth is that lately I haven’t had much time to deal with this, but I’m considering going back to an older SD image to get this working.
Cheers.
Short answer is no.
I had a quick go at trying to replicate the issue and I cannot.
All I can suggest is you try and put some debug code around the line that is failing and see what is actually being received.
Have a Google.
I did and found this which may be related
and this
It could be Python3 is more fussy that the old Python.
[edit]
Looking at the docs, paho-mqtt · PyPI note this example explicitly converts the message.payload to a string. Try that.
Hi there.
I will try to mess with the script and understand the reason why it is crashing, but I would like confirm in the first place that the script is really working…
Anyway, that seems very strange to me, because any message that I throw at mqtt, independently of the destination node I set, causes it to crash.
A fairly simple message like this:
mosquitto_pub -h 127.0.0.1 -u emonpi -P emonpimqtt2016 -t emonhub/tx/6/values -n -d
should be delivered to the hub, but it isn’t, it crashes the mqtt server.
Cheers.
Not strange at all. All those messages are being handled by the script as the topic is emonhub.
Is is the point when it tries to evaluate the message it is failing.
I think you need
payload = str(msg.payload)
Hi.
Finally I had some minutes to try.
The result is the same:
File "/opt/openenergymonitor/emonhub/src/interfacers/EmonHubMqttInterfacer.py", line 264, in on_message
payload = str(msg.payload)
TypeError: a bytes-like object is required, not 'str'
It seems that the object is already a string and it wants a “bytes-like”, whatever it means.
I will check the original script from my working EmonPi.
Cheers.
Ok, I think I know the problem.
Try
payload2 = str(msg.payload)
realdata = payload2.split(",")
self._log.debug("Nodeid: %s values: %s", nodeid, msg.payload)
You could try it without the str - that is belt and braces.
I think this is a Python3 thing and your old system may still have been Python.
Hi there.
This is how the code presently looks like:
if topic_parts[0] == self._settings["node_format_basetopic"][:-1] and topic_parts[1] == "tx" and topic_parts[3] == "values":
nodeid = int(topic_parts[2])
payload2 = str(msg.payload)
realdata = payload2.split(",")
self._log.debug("Nodeid: %s values: %s", nodeid, msg.payload)
# payload = msg.payload
# realdata = payload.split(",")
# self._log.debug("Nodeid: %s values: %s", nodeid, msg.payload)
rxc = Cargo.new_cargo(realdata=realdata, nodeid=nodeid)
for channel in self._settings["pubchannels"]:
# Add cargo item to channel
self._pub_channels.setdefault(channel, []).append(rxc)
self._log.debug("%d Sent to channel' : %s", rxc.uri, channel)
This is the result log when I run “mosquitto_pub -h 127.0.0.1 -u emonpi -P emonpimqtt2016 -t emonhub/tx/6/values -n -d” or “mosquitto_pub -h 127.0.0.1 -u emonpi -P emonpimqtt2016 -t emonhub/tx/5/values -m 10,111,33 -d”:
2022-06-06 21:18:12,374 WARNING MQTT Exception caught in MQTT thread. Traceback (most recent call last):
File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 31, in wrapper
return func(*args)
File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 126, in run
self.action()
File "/opt/openenergymonitor/emonhub/src/interfacers/EmonHubMqttInterfacer.py", line 214, in action
self._mqttc.loop(0)
File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 1181, in loop
rc = self.loop_read(max_packets)
File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 1572, in loop_read
rc = self._packet_read()
File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 2310, in _packet_read
rc = self._packet_handle()
File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 2936, in _packet_handle
return self._handle_publish()
File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 3216, in _handle_publish
self._handle_on_message(message)
File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 3444, in _handle_on_message
self.on_message(self, self._userdata, message)
File "/opt/openenergymonitor/emonhub/src/interfacers/EmonHubMqttInterfacer.py", line 264, in on_message
payload2 = str(msg.payload)
TypeError: a bytes-like object is required, not 'str'
2022-06-06 21:18:12,442 WARNING MainThread MQTT thread is dead.
2022-06-06 21:18:12,442 WARNING MainThread Attempting to restart thread MQTT (thread has been restarted 10 times...)
Hi, sorry for the delay. New volunteer role taking up a lot of time. The problem is that this code has never been touched under Python 3. As others have mentioned, what you’re doing is unique, so the code has never been tested.
This patch should fix it:
Please can you test it?
Bruce
1 Like
Hi there.
Thank you all for your efforts in trying to help me and sorry for taking so long to test.
Here is how the code presently looks, please check it is ok:
if topic_parts[0] == self._settings["node_format_basetopic"][:-1] and topic_parts[1] == "tx" and topic_parts[3] == "values":
nodeid = int(topic_parts[2])
# payload = msg.payload
payload = msg.payload.decode()
realdata = payload.split(",")
self._log.debug("Nodeid: %s values: %s", nodeid, msg.payload)
rxc = Cargo.new_cargo(realdata=realdata, nodeid=nodeid)
This is the command I issued to test:
mosquitto_pub -h 127.0.0.1 -u emonpi -P emonpimqtt2016 -t emonhub/tx/5/values -m 10,111,33 -d
This is the result in the log:
2022-06-20 19:56:45,451 WARNING MQTT Exception caught in MQTT thread. Traceback (most recent call last):
File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 31, in wrapper
return func(*args)
File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 126, in run
self.action()
File "/opt/openenergymonitor/emonhub/src/interfacers/EmonHubMqttInterfacer.py", line 214, in action
self._mqttc.loop(0)
File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 1181, in loop
rc = self.loop_read(max_packets)
File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 1572, in loop_read
rc = self._packet_read()
File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 2310, in _packet_read
rc = self._packet_handle()
File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 2936, in _packet_handle
return self._handle_publish()
File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 3216, in _handle_publish
self._handle_on_message(message)
File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 3444, in _handle_on_message
self.on_message(self, self._userdata, message)
File "/opt/openenergymonitor/emonhub/src/interfacers/EmonHubMqttInterfacer.py", line 264, in on_message
# payload = msg.payload
TypeError: a bytes-like object is required, not 'str'
2022-06-20 19:56:45,650 WARNING MainThread MQTT thread is dead.
2022-06-20 19:56:45,651 WARNING MainThread Attempting to restart thread MQTT (thread has been restarted 16 times...)
2022-06-20 19:56:45,652 INFO MainThread Setting RFM2Pi quiet: 1 (1q)
2022-06-20 19:56:45,728 DEBUG RFM2Pi device settings updated: E i5 g210 @ 433 MHz
Am I doing anything wrong?
Cheers.
I am not sure why this is failing on a comment.
File "/opt/openenergymonitor/emonhub/src/interfacers/EmonHubMqttInterfacer.py", line 264, in on_message
# payload = msg.payload
Can you stop the service and restart it, please?
To be sure you have an up to date file
cd /opt/openenergymonitor/emonhub/
sudo systemctl stop emonhub.service
and do
git status
It should warn you that you have local changes in that file.
Do
git checkout -- /opt/openenergymonitor/emonhub/src/interfacers/EmonHubMqttInterfacer.py
git pull
And you should be back with a clean set of files.
Then edit the file as you did - just add the .decode() on (don’t comment out existing line)
payload = msg.payload.decode()
and restart emonhub
sudo systemctl restart emonhub.service
I didn’t imagine that I had to restart the service after editing the scripts.
Anyway, I did all the instructions and this is the result:
2022-06-21 19:41:24,915 DEBUG MQTT Nodeid: 5 values: b'10,111,33'
2022-06-21 19:41:24,916 DEBUG MQTT 30 Sent to channel' : ToRFM12
2022-06-21 19:41:25,152 WARNING RFM2Pi 30 Scales 1 for RX data : ['10', '111', '33'] not suitable
2022-06-21 19:41:25,154 WARNING RFM2Pi Exception caught in RFM2Pi thread. Traceback (most recent call last):
File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 31, in wrapper
return func(*args)
File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 121, in run
self.add(frame)
File "/opt/openenergymonitor/emonhub/src/interfacers/EmonHubJeeInterfacer.py", line 74, in add
self.send(txc)
File "/opt/openenergymonitor/emonhub/src/interfacers/EmonHubJeeInterfacer.py", line 240, in send
if self.getName() in f.encoded:
AttributeError: 'bool' object has no attribute 'encoded'
2022-06-21 19:41:25,275 WARNING MainThread RFM2Pi thread is dead.
2022-06-21 19:41:25,276 WARNING MainThread Attempting to restart thread RFM2Pi (thread has been restarted 1 times...)
2022-06-21 19:41:25,276 INFO MainThread Creating EmonHubJeeInterfacer 'RFM2Pi'
2022-06-21 19:41:25,278 DEBUG MainThread Opening serial port: /dev/ttyAMA0 @ 38400 bits/s
Yes, it gets loaded to run at that point - systemctl does not read the script from file again.
Something not quite right with the end bit of the message (the b) - b'10,111,33'
[edit]
I can see why
Take out the msg. so it is just payload from -
self._log.debug("Nodeid: %s values: %s", nodeid, msg.payload)
That does not explain the error.
@bwduncan does it need converting from a string?
[edit2]
I suspect the file EmonHubJeeInterfacer.py needs some Python3 love at this point 
First time some data has been sent I guess.
Hi there.
I believe that the main problem is solved, because I can now send an integer.
Now I need to adjust the original code (like I did before) to accept a sequence of bytes instead of just one integer.
My example message was 10,111,33 (3 bytes) that in fact are the variable number and a 16 bit integer, as in: set variable 10 to 8559 (111+33*256).
My personalized emonPi firmware has a bunch of variables that allow me to change some settings on-the-fly, like CT calibration values and much other stuff.
If you think that what I want is easy to implement in the original code and would be also useful for other users, I can wait a bit for it to be implemented, otherwise, I will investigate how I did it before and do it again in the new script.
Cheers.
Hi there.
I found that I made changes in emonhub_interfacer.py, but this file no longer exists in the new version.
Can someone point me to the new file that replaces this one?
I’m attaching my version of the file that I have on my old EmonPi if someone wants to analyse my changes.
Cheers.
emonhub_interfacer.py.txt (21.1 KB)
It is in my version.
and on github
I think you need to pull a new version down.
What is the output of git status?
Hummm, I see, it is in the parent directory.
I will analyse it.
Cheers.
Hi there.
I’m not very interested in sending just one value, but when I do it I receive this in the log
2022-06-24 20:03:12,310 DEBUG MQTT Nodeid: 5 values: b'65535'
2022-06-24 20:03:12,311 DEBUG MQTT 21 Sent to channel' : ToRFM12
2022-06-24 20:03:12,506 WARNING RFM2Pi RFM2Pi discarding Tx packet: values out of scope
What does exacly mean values out of scope?
I have this section in the node configuration, so it should be able to receive one long int
[[[tx]]]
names = O
units = X
scales = 1
datacodes = L
Cheers.