Python3 for emonhub

borpin · 2020-05-26T13:13:13.890Z

@TrystanLea, Is this a good opportunity to move to a ‘stable’ branch and do releases from there? The Python 2 emonpi branch could then be left alone.

TrystanLea · 2020-05-26T13:18:02.408Z

Good point, yes that would be good

TrystanLea · 2020-05-26T14:44:11.348Z

I think this should be all that is needed to both install the python3 version of emonhub on new installs and update to python3 on existing systems via emonPi/base update: Install and update python3 version of emonhub by TrystanLea · Pull Request #87 · openenergymonitor/EmonScripts · GitHub

Leaving that as an open pull request now for discussion and testing.

Would you also like me to merge Python3 fixes by TrystanLea · Pull Request #99 · openenergymonitor/emonhub · GitHub @bwduncan?

borpin · 2020-05-26T16:11:36.264Z

Have you removed the python3 branch @TrystanLea? Glad I noticed else that would have really mucked me up!

TrystanLea · 2020-06-05T11:05:54.259Z

emonPi/base update now includes updating emonhub to python3!

The update process includes the installation of Python3 and dependencies as well as a call to apt-get update which can take a little time on slower connections.

The emonPiLCD script and service-runner are still ran with Python 2.7 which remains on the system.

Thanks @bwduncan and @borpin for all of your help with this!

Robert.Wall · 2020-06-05T16:59:22.783Z

@TrystanLea
Something missing?

Emonhub fails to start after update emonHub

ok With more googleing it seems I need python3-configobj python3-serial I wonder why they weren’t there? If anyone else has this issue, you need to run sudo apt-get install python3-serial python3-configobj and a reboot brings it back to life…So far, anyway!

borpin · 2020-06-05T18:41:20.415Z

It looks like the install script fails for some reason.

borpin · 2020-06-05T19:04:24.962Z

@TrystanLea

The install script will fail as it wants user input. Manual run below.

pi@emonpi:/opt/openenergymonitor/emonhub $ ./install.sh
Apply raspberrypi serial configuration? 1=yes, 0=no:

The key press is not echoed.

TrystanLea · 2020-06-05T21:13:48.484Z

borpin:

The install script will fail as it wants user input. Manual run below.

Strange, It ran through fine in my tests?

TrystanLea · 2020-06-05T21:20:34.089Z

To be better safe than sorry I’ve pushed a fix that passed the $emonSD_pi_env variable to the install script as argument 1, I realise the install script is actually looking for this and then skips the user entry skip user entry · openenergymonitor/EmonScripts@6e3d2b7 · GitHub

merged to stable

borpin · 2020-06-06T11:25:34.771Z

TrystanLea:

merged to stable

Patch release?

borpin · 2020-06-08T06:08:46.940Z

Hi Trystan, something odd happening with updates Disk Space for Logs - #7 by Pukka

borpin · 2020-06-10T16:18:47.320Z

@bwduncan @TrystanLea

Attempted to use the python3 version of emonhub and the EmonHubSocketInterfacer.py.

I get this error on packet receipt.

2020-06-10 17:10:39,183 WARNING  Pulse      Exception caught in Pulse thread. Traceback (most recent call last):
  File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 32, in wrapper
    return f(*args)
  File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 99, in run
    rxc = self.read()
  File "/opt/openenergymonitor/emonhub/src/interfacers/EmonHubSocketInterfacer.py", line 69, in read
    select.select([self._socket], [], [], 0)
TypeError: argument must be an int, or have a fileno() method.

I am using this script to send the packets - Directly connecting to Optical Pulse Counter with RPi? - #61 by danbates

I note that there is a print f which I corrected but I am getting this error with Python3

Traceback (most recent call last):
  File "/home/pi/pulses.py", line 31, in eventHandler2
    processpulse(2,GPIO.input(channel))
  File "/home/pi/pulses.py", line 51, in processpulse
    send(f)
  File "/home/pi/pulses.py", line 58, in send
    s.send(f)
TypeError: a bytes-like object is required, not 'str'

This all relates to trying to get a pulse interfacer going Emonhub Pulse Interfacer

bwduncan · 2020-06-10T16:38:31.805Z

HI,

I’m not sure I fully understand the architecture here. Is emonhub running on a pi with a pulse counter (well, photodiode) connected to its own GPIO pins?

If that’s the case then this should be quite simple, since emonhub will handle all the network stuff, we just need to give it the appropriate packets at the right time.

Happy to chuck something together if someone can explain the hardware setup.

Bruce

borpin · 2020-06-10T16:47:04.849Z

bwduncan:

I’m not sure I fully understand the architecture here.

Sorry. Where I’d like to end up is that the Pi with emonhub running has the pulse counter connected to the GPIO pins and there is an emonhub Pulse interfacer to read that data and load it into a frame for the rest of emonhub to do it’s stuff (other thread really).

The socket thing will probably not be needed for this, it is just the reading of the pins.

However, in trying to test this, I used the linked script on a Pi without emonhub - so effectively I’m testing the socket interfacer on Python3 which possibly has not been tested.

I have 2 problems with this, one the script didn’t want to play for Python3 and secondly the socket interfacer on the other Pi with emonhub didn’t want to play.

2 issues on the emonhub side,

it seems the nb_port works when the default is used, but not if a port is specified in the [[initsettings]]
line 78 needs a .decode()

Other than that I seem to have got that working.

bwduncan · 2020-06-10T17:05:14.494Z

Are there any log messages associated with that socket error? If the initialisation of the socket fails, the code continues as if nothing is wrong, except for logging an error, so then it explodes later when it tries to use it.

Nice work finding those bugs, I agree that a .decode() is needed there.

It looks like the set() method of doesn’t understand the port_nb setting. Not sure if that’s the cause of your first problem.

borpin · 2020-06-10T17:07:59.871Z

bwduncan:

Are there any log messages associated with that socket error?

Just what I posted. Results in TypeError: argument must be an int, or have a fileno() method. as though the open_socket call did not return an integer. I did think a debug message saying socket opened might help!

borpin · 2020-06-10T17:21:55.949Z

Odd it now seems to work! I’ll do a PR for that.

bwduncan · 2020-06-10T17:23:30.185Z

Congratulations on your Heisenbug.

bwduncan · 2020-06-15T00:11:29.253Z

This topic is getting a bit diverse, maybe @borpin can work some thread-splitting magic?

With encouragement and help from @alexandrecuer I have had a go at upgrading emonPiLCD to python3. To encourage people to pull it, I added some new features, like being able to gather the temperature and other things from the new MQTT topics, rather than the legacy “values” topic. Also, it now displays a neat little degree symbol!

You will need to install python3-xmltodict, and perhaps other things, but the install/update scripts scare me so I’ll leave the upgrade to someone who understands that better.

Please test and report back!

github.com/openenergymonitor/emonpi

Port LCD to python3

openenergymonitor:master ← bwduncan:master

opened 12:11AM - 15 Jun 20 UTC

bwduncan

+443 -565

This isn't ready for release yet, I should have used a different branch name :sh…rug: but this PR could be a handy way to consolidate comments.

Bruce