Python3 for emonhub

emonHub
62

Have you removed the python3 branch @TrystanLea? Glad I noticed else that would have really mucked me up!

63

emonPi/base update now includes updating emonhub to python3! :tada:

The update process includes the installation of Python3 and dependencies as well as a call to apt-get update which can take a little time on slower connections.

The emonPiLCD script and service-runner are still ran with Python 2.7 which remains on the system.

Thanks @bwduncan and @borpin for all of your help with this!

1 Like
64

@TrystanLea
Something missing?

65

It looks like the install script fails for some reason.

66

@TrystanLea

The install script will fail as it wants user input. Manual run below.

pi@emonpi:/opt/openenergymonitor/emonhub $ ./install.sh
Apply raspberrypi serial configuration? 1=yes, 0=no:

The key press is not echoed.

67

Strange, It ran through fine in my tests?

68

To be better safe than sorry I’ve pushed a fix that passed the $emonSD_pi_env variable to the install script as argument 1, I realise the install script is actually looking for this and then skips the user entry skip user entry · openenergymonitor/EmonScripts@6e3d2b7 · GitHub

merged to stable

1 Like
69

Patch release?

70

Hi Trystan, something odd happening with updates Disk Space for Logs - #7 by Pukka

71

@bwduncan @TrystanLea

Attempted to use the python3 version of emonhub and the EmonHubSocketInterfacer.py.

I get this error on packet receipt.

2020-06-10 17:10:39,183 WARNING  Pulse      Exception caught in Pulse thread. Traceback (most recent call last):
  File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 32, in wrapper
    return f(*args)
  File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 99, in run
    rxc = self.read()
  File "/opt/openenergymonitor/emonhub/src/interfacers/EmonHubSocketInterfacer.py", line 69, in read
    select.select([self._socket], [], [], 0)
TypeError: argument must be an int, or have a fileno() method.

I am using this script to send the packets - Directly connecting to Optical Pulse Counter with RPi? - #61 by danbates

I note that there is a print f which I corrected but I am getting this error with Python3

Traceback (most recent call last):
  File "/home/pi/pulses.py", line 31, in eventHandler2
    processpulse(2,GPIO.input(channel))
  File "/home/pi/pulses.py", line 51, in processpulse
    send(f)
  File "/home/pi/pulses.py", line 58, in send
    s.send(f)
TypeError: a bytes-like object is required, not 'str'

This all relates to trying to get a pulse interfacer going Emonhub Pulse Interfacer

72

HI,

I’m not sure I fully understand the architecture here. Is emonhub running on a pi with a pulse counter (well, photodiode) connected to its own GPIO pins?

If that’s the case then this should be quite simple, since emonhub will handle all the network stuff, we just need to give it the appropriate packets at the right time.

Happy to chuck something together if someone can explain the hardware setup.

Bruce

73

Sorry. Where I’d like to end up is that the Pi with emonhub running has the pulse counter connected to the GPIO pins and there is an emonhub Pulse interfacer to read that data and load it into a frame for the rest of emonhub to do it’s stuff (other thread really).

The socket thing will probably not be needed for this, it is just the reading of the pins.

However, in trying to test this, I used the linked script on a Pi without emonhub - so effectively I’m testing the socket interfacer on Python3 which possibly has not been tested.

I have 2 problems with this, one the script didn’t want to play for Python3 and secondly the socket interfacer on the other Pi with emonhub didn’t want to play.

2 issues on the emonhub side,

  1. it seems the nb_port works when the default is used, but not if a port is specified in the [[initsettings]]
  2. line 78 needs a .decode()

Other than that I seem to have got that working.

74

Are there any log messages associated with that socket error? If the initialisation of the socket fails, the code continues as if nothing is wrong, except for logging an error, so then it explodes later when it tries to use it.

Nice work finding those bugs, I agree that a .decode() is needed there.

It looks like the set() method of doesn’t understand the port_nb setting. Not sure if that’s the cause of your first problem.

75

Just what I posted. Results in TypeError: argument must be an int, or have a fileno() method. as though the open_socket call did not return an integer. I did think a debug message saying socket opened might help!

76

Odd it now seems to work! I’ll do a PR for that.

77

Congratulations on your Heisenbug.

2 Likes
78

This topic is getting a bit diverse, maybe @borpin can work some thread-splitting magic?

With encouragement and help from @alexandrecuer I have had a go at upgrading emonPiLCD to python3. To encourage people to pull it, I added some new features, like being able to gather the temperature and other things from the new MQTT topics, rather than the legacy “values” topic. Also, it now displays a neat little degree symbol!

You will need to install python3-xmltodict, and perhaps other things, but the install/update scripts scare me so I’ll leave the upgrade to someone who understands that better.

Please test and report back!

Bruce

2 Likes
79

@bwdcuncan : made some tests :slight_smile:

on the emonPi I was using to test your fork…did a git pull and a sudo reboot : the LCD did not want to reboot by itself…I had to poweroff manually the Pi…Since that it seems to work fine…

EDIT : after a reboot, the LCD detects the sensors, boots the PI, but remains illuminated on a blue screen without displaying anything
Anyway, it appears the service file is running correctly :

pi@emonpi:~ $ sudo systemctl status emonPiLCD.service
● emonPiLCD.service - emonPi LCD driver
   Loaded: loaded (/lib/systemd/system/emonPiLCD.service; enabled; vendor preset
   Active: active (running) since Thu 2020-06-18 11:25:03 CEST; 4min 42s ago
 Main PID: 935 (python3)
    Tasks: 5 (limit: 2319)
   Memory: 11.5M
   CGroup: /system.slice/emonPiLCD.service
           └─935 /usr/bin/python3 /opt/openenergymonitor/emonpi/lcd/emonPiLCD.py

Jun 18 11:25:03 emonpi systemd[1]: Started emonPi LCD driver.

In addition to the actions related to the scripts, I have installed the following dependencies:

sudo apt-get install python3-smbus i2c-tools python3-rpi.gpio python3-pip redis-server  python3-gpiozero -y
pip3 install redis paho-mqtt xmltodict requests

This could be included in the LCD install script on the EmonScript repo :

on the same script, line 42, the sed command overwrites the modified service file and so bypass the use of python3 to launch the emonPiLCD service…

sudo sed -i "s~ExecStart=.*~ExecStart=/usr/bin/python $openenergymonitor_dir/emonpi/lcd/emonPiLCD.py~" /lib/systemd/system/$service.service

@bwduncan also tried :

sudo apt-get install -y python3-xmltodict

But still the blue screen…

Alex

1 Like
81

The general advice is the sudo should not be used for pip installs.

1 Like
82

There is a fundamental flaw in the install that has been fixed elsewhere.

First it should be a link to the service file.
Second the install script should be in the lcd folder so it relates to the code in the folder.

When the emonpi repo is updated with a Python3 script, the install and the service file would then be updated as appropriate.

This is what has been done elsewhere but most of those are independent repos.

@TrystanLea,

  1. Update the emonScripts repo to point the install to the emonpi/lcd folder
  2. Create an install file in emonpi/lcd ‘as is’.
  3. Publish both repos as an update.

Once done, as part of the PR for Python3

  1. update install file for python3 and modify how user, log etc are created.
  2. update the service file for Python3

Publishing the Python3 updated emonpi repo will then pull in the right install script and service file for Python3.

[edit] I have raised an issue on EmonScripts and commented on the PR.