Victron Energy Battery PYMODBUS error

Hi,

I’ve got a battery and solar and been using the MySolarBattery app for some time with no issues. I’ve been pestering my installer (PureDrive) to help me get State of Charge info as I know they are using Victron Energy within the system. I’ve finally got the info I need and configured EmonModbusTcpInterfacer but I keep getting a “PYMODBUS NOT PRESENT BUT NEEDED !!” error. I’ve done a system update and also a “pip install -U pymodbus” and everything says pymodbus is installed but I still get the same error! Does it need to be in a certain directory and if so how do I move/install it there?

Any help gratefully received as I’ve been trying for several hours now to resolve the issue!

Thanks.

2023-06-09 15:26:43,606 INFO     MainThread Creating EmonModbusTcpInterfacer 'ModbusTCP'
2023-06-09 15:26:43,607 ERROR    MainThread PYMODBUS NOT PRESENT BUT NEEDED !!
2023-06-09 15:26:43,608 DEBUG    MainThread Setting ModbusTCP register: 400267
2023-06-09 15:26:43,608 DEBUG    MainThread Setting ModbusTCP nodeId: 12
2023-06-09 15:26:43,609 DEBUG    MainThread Setting ModbusTCP pubchannels: ['ToEmonCMS']
2023-06-09 15:26:43,610 DEBUG    MainThread Setting ModbusTCP interval: 10
2023-06-09 15:26:43,615 DEBUG    MainThread Automatic configuration of nodes enabled

pip install -U pymodbus installs pymodbus in a directory that’s not on the search path.

Give this a try…

First, uninstall pymodbus. pip uninstall pymodbus

Then, re-install it like this: sudo pip install -U pymodbus

Hi Bill. Thanks for the reply. I tried that but still getting the same error. The uninstall worked correctly, but the messages I got for the install were as follows…

pi@emonpi:/ $ sudo pip install -U pymodbus
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Requirement already satisfied: pymodbus in /usr/lib/python3/dist-packages (2.1.0)
Collecting pymodbus
  Using cached https://www.piwheels.org/simple/pymodbus/pymodbus-3.3.1-py3-none-any.whl (210 kB)
Installing collected packages: pymodbus
  Attempting uninstall: pymodbus
    Found existing installation: pymodbus 2.1.0
    Not uninstalling pymodbus at /usr/lib/python3/dist-packages, outside environment /usr
    Can't uninstall 'pymodbus'. No files were found to uninstall.
Successfully installed pymodbus-3.3.1

The last line says it sucessfully installed pymodbus-3.3.1. The “error” is complaining about
not being able to find pymodbus 2.1.0 to uninstall. (because it was uninstalled when you ran
the pip uninstall command)

If it’s the “PYMODBUS NOT PRESENT BUT NEEDED !!” error you’re still getting,
then I’ve run out of ideas. :frowning_face:

Yes it is the “PYMODBUS NOT PRESENT BUT NEEDED !!” error I’m still getting. I did a full reboot just to make sure it picked it up.

Thanks for your help. Hopefully someone else might be able to point me in the right direction. I don’t really know what to do next in terms troubleshooting!

1 Like

At the top of EmonModbusTcpInterfacer.py there is a section as follows…

import time
import Cargo

try:
    from pymodbus.constants import Endian
    from pymodbus.payload import BinaryPayloadDecoder
    from pymodbus.client.sync import ModbusTcpClient as ModbusClient
    pymodbus_found = True
except ImportError:
    pymodbus_found = False

Then later on it says…

if not pymodbus_found:
            self._log.error("PYMODBUS NOT PRESENT BUT NEEDED !!")

So it is these import functions that are going wrong! Not sure if that helps…

An easy way to verify if the modules are indeed present, is to start python at a command prompt
then import them. If the module exists, you’ll see something similar to this:

bt@trailer:~ $ python
Python 2.7.18 (default, Jul 14 2021, 08:11:37)
[GCC 10.2.1 20210110] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import time
>>>

If a module isn’t present on your system, you’ll get an error:

>>> import cargo
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named cargo
>>>

Thanks Bill. I’ll give that a go tomorrow and see how I get on.

thumbsup thumbs_up

Well no, they are not going wrong, it is telling you you have not got the package installed (so it is doing what it needs to do).

@Bill.Thomson - this is not the right module.

@BenWarwIck Is this an old install of emoncms (not if it has not been updated, but was the original install a while ago)? There was an issue with all possible modules not being installed.

To fix -

cd /opt/openenergymonitor/emonhub/
git pull

Check there are no errors here (such as cannot update due to local changes)

run the install script for emonhub

./install.sh

This should fix the issue.

Hi @borpin . Thanks for you comments. It isn’t an old install. I re-did my install from a vanilla EmonSD install using the Nov22 version.

Do you think I need to try the GitHub commands you mentioned as well first? It looks like this is updating Python and all the modules, or will the git pull do all that for me?

Many thanks.

The git pull will ensure the files are all up to date and then run the install script.

Odd it doesn’t work.

@borpin still no joy. Still getting the PYMODBUS NEEDED BUT NOT PRESENT !!error which still seems to be triggered by those missing modules. I’ve included the return from the git pull below

hint: Pulling without specifying how to reconcile divergent branches is
hint: discouraged. You can squelch this message by running one of the following
hint: commands sometime before your next pull:
hint: 
hint:   git config pull.rebase false  # merge (the default strategy)
hint:   git config pull.rebase true   # rebase
hint:   git config pull.ff only       # fast-forward only
hint: 
hint: You can replace "git config" with "git config --global" to set a default
hint: preference for all repositories. You can also pass --rebase, --no-rebase,
hint: or --ff-only on the command line to override the configured default per
hint: invocation.
Already up to date.

Then also an extract from the ./install.sh command as well as the full extract below. As far as I can see it seems to think everything is up to date!

python3-configobj is already the newest version (5.0.6-4).
python3-pymodbus is already the newest version (2.1.0+dfsg-2).
python3-serial is already the newest version (3.5~b0-1).
EmonHub directory: /opt/openenergymonitor/emonhub
Apply raspberrypi serial configuration? (y/n): y
Would you like to install emonhub under the pi user? (y/n): y
Running apt update
Get:1 http://raspbian.raspberrypi.org/raspbian bullseye InRelease [15.0 kB]
Get:2 https://download.docker.com/linux/raspbian bullseye InRelease [26.7 kB]                                                       
Hit:3 http://archive.raspberrypi.org/debian bullseye InRelease                                                                      
Get:4 https://packages.sury.org/php bullseye InRelease [6,841 B]            
Get:5 https://packages.sury.org/php bullseye/main armhf Packages [376 kB]                                                           
Fetched 424 kB in 7s (61.5 kB/s)                                                                                                    
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
117 packages can be upgraded. Run 'apt list --upgradable' to see them.
installing or updating emonhub dependencies
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
python3-configobj is already the newest version (5.0.6-4).
python3-pymodbus is already the newest version (2.1.0+dfsg-2).
python3-serial is already the newest version (3.5~b0-1).
bluetooth is already the newest version (5.55-3.1+rpt2).
libbluetooth-dev is already the newest version (5.55-3.1+rpt2).
python3-pip is already the newest version (20.3.4-4+rpt1+deb11u1).
python3-spidev is already the newest version (20200602~200721-1).
0 upgraded, 0 newly installed, 0 to remove and 117 not upgraded.
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Requirement already satisfied: paho-mqtt in /home/pi/.local/lib/python3.9/site-packages (1.6.1)
Requirement already satisfied: requests in /usr/lib/python3/dist-packages (2.25.1)
Requirement already satisfied: pybluez in /home/pi/.local/lib/python3.9/site-packages (0.23)
Requirement already satisfied: py-sds011 in /home/pi/.local/lib/python3.9/site-packages (0.9)
Requirement already satisfied: sdm_modbus in /home/pi/.local/lib/python3.9/site-packages (0.6.1)
Requirement already satisfied: minimalmodbus in /home/pi/.local/lib/python3.9/site-packages (2.0.1)
Requirement already satisfied: pyserial>=3.0 in /usr/lib/python3/dist-packages (from minimalmodbus) (3.5b0)
Requirement already satisfied: pyserial-asyncio>=0.6.0 in /home/pi/.local/lib/python3.9/site-packages (from sdm_modbus) (0.6)
Requirement already satisfied: pymodbus>=3.0.0 in /usr/local/lib/python3.9/dist-packages (from sdm_modbus) (3.3.1)
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting https://github.com/openenergymonitor/rpi-rfm69/archive/refs/tags/v0.3.0-oem-4.zip
  Using cached https://github.com/openenergymonitor/rpi-rfm69/archive/refs/tags/v0.3.0-oem-4.zip
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Requirement already satisfied: RPI.GPIO in /usr/lib/python3/dist-packages (from rpi-rfm69==0.3.4) (0.7.0)
Requirement already satisfied: spidev in /usr/lib/python3/dist-packages (from rpi-rfm69==0.3.4) (3.5)
installing or updating raspberry pi related dependencies
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Requirement already satisfied: RPi.GPIO in /usr/lib/python3/dist-packages (0.7.0)
Disabling Bluetooth
Stop Bluetooth modem
Remove console from /boot/cmdline.txt
Stop and disable serial service
Unit /etc/systemd/system/[email protected] is masked, ignoring.
/etc/emonhub directory already exists
Setting ownership of /var/log/emonhub to pi
Setting ownership of /var/log/emonhub/emonhub.log to pi and permissions to 644
Installing /usr/local/bin/emonhub symlink
emonhub.service already installed
- Service ActiveState=active

That is odd. No time for me to test but @TrystanLea or @glyn.hudson mught chip in. @bwduncan is a Python man as well.

I’ll have a look later.

Hello @BenWarwIck Looks like there was a change in the pymodbus library 3.3 onwards. I’ve made a couple of changes to the interfacer that at least get past the error here, though I am not able to test it end to end here. EmonHub v2.5.8

Hi Tristan.

Thanks for your help. Is that something I can update using the normal update within emonpi.local interface?

Or should I do the git pull and ./install.sh commands?

@TrystanLea - from GitHub

image

Yes a full update should pull it through.

OK, so that has worked! Thanks for your help @TrystanLea and @borpin. However getting new error, which I don’t understand. It is saying ModbusTCP You have define and equal number of registers and of names. I don’t understand as I though I had. My config looks like this…

    [[ModbusTCP]]
        # this interfacer retrieves register information from modbusTCP clients
        # retrieve register information from modbus TCP documentation for your inverter.
        Type = EmonModbusTcpInterfacer
        [[[init_settings]]]
            modbus_IP = 192.168.0.141   # ip address of client to retrieve data from
            modbus_port = 502          # Portclient listens on
        [[[runtimesettings]]]
            # List of starting registers for items listed above
            register = 400267
            # nodeid used to match with node definition in nodes section below. Can be set to any integer value not previously used.
            nodeId = 12
            # Channel to publish data to should leave as ToEmonCMS
            pubchannels = ToEmonCMS,
            # time in seconds between checks, This is in addition to emonhub_interfacer.run() sleep time of .01
            # use this value to set the frequency of data retrieval from modbus client
            interval = 10

    [[12]]
        nodename = Batery_SOC
        [[[rx]]]
            # list of names of items being retrieved
            names = batterysoc
            datacodes = H
            scales = 1
            units = %

I’m told by my provide that this register is for the Battery Soc (the only thing I’m missing for the MySolayBattery app. Is there any way to check? Should I just add other names and see what I get?

So the register number is usually one more than the Modbus address (register number starts at 1, address starts at 0). Try 400266. This really does want the register number. -1 done in the code.

Next thing to check is if it is Holding register (RW) or an Input register (RO). 16bit or 32 bit. Int, UInt or float. All makes a difference when decoding it!

Do you have Node-RED or HomeAssistant running anywhere?

[edit]
emonhub only reads Holding registers.