emonPi new SD image - emonHub failing

emonHub
21

ok great but also strange:

sudo service emonhub restart

should bring it back up in a way that allows it to continue once you exit.

Given that it has stopped before there is likely still an issue.

22

Ok Iā€™ve been able to replicate this here on a new system. Emonhub appears to work fine from the command line but the service fails after the first second or two. Im looking into it.

1 Like
23

Alas no, does not come back up.

24

Ok the quick fix is, similar to @robinrottier thanks! It needs the sudo:

sudo pip3 install paho-mqtt requests
sudo service emonhub restart

We will fix up the update process to do this automatically next

25

Thank you Trystan. Those two lines appear to have sorted my emonbase out. The emonhub service is now running again and the inputs are updating.

1 Like
26

Running emonPI/base update from the admin page will also now automatically fix this issue.

27

Were there any other changes to do with the emonhub socket listener? I have emonhub config entry like this:

[[mysocketlistener]]
        Type = EmonHubSocketInterfacer
        [[[init_settings]]]
                port_nb = 8080
        [[[runtimesettings]]]
                pubchannels = ToEmonCMS,

And now on startup it logs this the first time it receives something from the socket:

2020-06-22 11:57:21,546 WARNING  mysocketlistener Exception caught in mysocketlistener thread. Traceback (most recent call last):
  File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 32, in wrapper
    return f(*args)
  File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 99, in run
    rxc = self.read()
  File "/opt/openenergymonitor/emonhub/src/interfacers/EmonHubSocketInterfacer.py", line 78, in read
    self._sock_rx_buf = self._sock_rx_buf + conn.recv(1024)
TypeError: can only concatenate str (not "bytes") to str

The socket thread is then deadā€¦ it tries to restart that listener but its now stuck ina loop saying the socket already exists.

Before the update this listener was working fine receiving content posted from a python script on my victron/venus controller

To fix it I edited EmonHubSocketInterfacer.py line 78 as per the error and changed to:

        # Read data
        self._sock_rx_buf = self._sock_rx_buf + conn.recv(1024).decode("utf-8")

adding the decode to turn bytes into string. Restarted and its collecting values from the listener again.

28

Sorted, many thanks.

29

Could you open a PR on GitHub for this? Just edit the file in GitHub itself is the easiest way.

30

Thanks @borpin @robinrottier Iā€™ve applied the decode fix.

1 Like
31

All the similar decodes do not specify the character set - perhaps this should not either to be consistent.

32

youā€™re probably right ā€“ I only used that it of code because I saw it somewhere on google and it worked for me, without having to think too much and I wanted my system to be back working asap on this nice sunny day.

I just changed my edit to :

self._sock_rx_buf = self._sock_rx_buf + conn.recv(1024).decode()

and it still parses my output in the socket listener, so great!

Iā€™d suggest for a generic implementation you would read that string parameter value from the config of that instance of each socket listener, so different senders could write in whatever encoding they wanted to the socket listener; and have a missing or blank config to pick up the default, whatever that is (Id guess utf8 is default but I dont know). Maybe there might be something somewhere sending strings in utf16 or some other encoding and that would let you handle it with little extra effort

my real wonder would be why did it stop working ā€“ what changed in your update that meant the deocde() was now necessary and are there going to be other places that did not have a decode() that also now dont work?

33

itā€™s probably something python3 related, thatā€™s been the main and large change? @bwduncan do you know what it might be?

34

It was Bruce who told me to include the decode() statement in the Serial TX3e interfacer when that didnā€™t play. I added it to another decoder as well at some point I think.

35

Yes it could be that some bytes slipped through where text was expected. There really isnā€™t a reliable automatic way to detect all the places where this happens. You just have to look through all the code and know which interfaces are dealing with bytes and which are dealing with strings. Or you let them fail in production, which sadly seems to be the way to find them in this case :frowning_face: Sorry about that.

Iā€™ll have another human look at some of the interfacers and see if I can spot any suspect code.

UTF-8 is the default encoding in Python since like 3.5 I think. Almost everything weā€™ll be dealing with is using UTF-8, or ASCII which is a subset. I believe windows uses UTF-16 internally still, but Iā€™d be surprised if you can show me a device which is sending something other than UTF-8!

2 Likes
36

Confirmed back and working through update.

2 Likes