Reading modbus data from Eltako 3-phase meter

Lawrie_Witham · 17 August 2024 17:14

I recently bought a Level 3 heatpump monitoring kit (order #33521) and have connected it up to a existing Kamstrup 403 heat meter and an Eltako DSZ15DZMOD 3-phase meter, which my electrician installed. At first glance the Modbus protocol looks very similar to that of the SDM630, but the values are all 4-byte integers, mostly unsigned. Here is the extract from my emonhub.conf:


    [[modbus]]
        Type = EmonHubMinimalModbusInterfacer
        [[[init_settings]]]
            #device = /dev/serial/by-id/usb-FTDI_FT232R_USB_UART_B001AG05-if00-port0
            device = /dev/ttyACM0
            baud = 9600
            #datatype = integer
        [[[runtimesettings]]]
            pubchannels = ToEmonCMS,
            read_interval = 10
            nodename = heatpump
            [[[[meters]]]]
                [[[[[electric]]]]]
                    address = 1
                    registers =  52, 72
                    names =  P_total, EI_total
                    datacodes = l, L
		    scales = 1, 0.01
		    units = kW, kWh

Unfortunately, “datacodes” and “units” are ignored and the values are read in as floats. Here a log extract after restart:

2024-08-17 16:46:19,945 INFO     MainThread Creating EmonHubMinimalModbusInterfacer 'modbus'
2024-08-17 16:46:19,946 INFO     MainThread Connecting to Modbus device=/dev/ttyACM0 baud=9600 parity=none datatype=float
2024-08-17 16:46:19,947 INFO     MainThread Setting modbus read_interval: 10
2024-08-17 16:46:19,948 INFO     MainThread Setting modbus nodename: heatpump
2024-08-17 16:46:19,948 INFO     MainThread Setting modbus prefix: 
2024-08-17 16:46:19,949 INFO     MainThread Setting modbus meters electric address 1
2024-08-17 16:46:19,949 INFO     MainThread Setting modbus meters electric registers [52, 72]
2024-08-17 16:46:19,950 INFO     MainThread Setting modbus meters electric names ["P_total", "EI_total"]
2024-08-17 16:46:19,950 INFO     MainThread Setting modbus meters electric scales [1.0, 0.01]
2024-08-17 16:46:19,951 DEBUG    MainThread Setting modbus pubchannels: ['ToEmonCMS']
2024-08-17 16:46:19,952 INFO     MainThread Setting MBUS read_interval: 10
2024-08-17 16:46:19,952 INFO     MainThread Setting MBUS validate_checksum: False
2024-08-17 16:46:19,953 INFO     MainThread Setting MBUS meters: {"heatmeter": {"address": "1", "type": "kamstrup403"}}
2024-08-17 16:46:20,419 DEBUG    modbus     [2.6624670822171524e-44, 1.807675018979014e-45]
2024-08-17 16:46:20,420 DEBUG    modbus     833 NEW FRAME : 
2024-08-17 16:46:20,421 DEBUG    modbus     833 Timestamp : 1723909580.052268
2024-08-17 16:46:20,421 DEBUG    modbus     833 From Node : heatpump
2024-08-17 16:46:20,422 DEBUG    modbus     833    Values : [2.6624670822171524e-44, 1.807675018979014e-45]
2024-08-17 16:46:20,423 DEBUG    modbus     833 Sent to channel(start)' : ToEmonCMS
2024-08-17 16:46:20,423 DEBUG    modbus     833 Sent to channel(end)' : ToEmonCMS

Entering “datatype=int” in the init_settings creates a different error - the register addresses no longer fit (2-byte data?). Is there another magic word I can enter here - “long” is ignored?

How can I get emonhub to read the datacodes? I would welcome any suggestions.

Lawrie_Witham · 20 August 2024 13:49

UPDATE:
If all else fails, read the documentation!
From the Minimal Modbus API documentation it is clear that read_long is needed. However, the interfacer EmonHubMinimalModbusInterfacer.py only implements read_register (datatype = int) and read_float. So I added an extra elif for read_long that I can access with datatype = long.

elif self.datatype == 'long':
    time.sleep(0.1)	
    value = self._rs485.read_long(int(self._settings['meters'][meter]['registers'][i]), functioncode=4, signed=True, number_of_registers=2)

This works for me, but note that all registers are read as longs, you can’t have mixed longs and floats.
I hope this helps anyone else who bought the “wrong” electricity meter.

TrystanLea · 21 August 2024 10:43

Thanks for sharing @Lawrie_Witham

gadgetbazza · 3 October 2024 11:19

Hi,

Im having trouble with a long time Modbus device on my emonCMS instance. Not sure when it started causing issues, but I’ve had a lot of missed data in the feeds.

I updated emonCMS to the latest version today 11.6.5, but now I’m seeing the following error in the emonhub log

2024-10-03 12:13:40,057 INFO     ModbusTCP  Not connected, retrying connect {'modbus_IP': '192.168.4.161', 'modbus_port': '502'}
2024-10-03 12:13:40,068 INFO     ModbusTCP  Opening modbusTCP connection: 502 @ 192.168.4.161
2024-10-03 12:13:40,069 DEBUG    ModbusTCP  expected bytes number after encoding: 132
2024-10-03 12:13:40,070 DEBUG    ModbusTCP  datacode I
2024-10-03 12:13:40,070 DEBUG    ModbusTCP  reading register #: 30058, qty #: 2, unit #: 3
2024-10-03 12:13:40,242 WARNING  ModbusTCP  Exception caught in ModbusTCP thread. Traceback (most recent call last):
  File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 31, in wrapper
    return func(*args)
  File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 105, in run
    rxc = self.read()
  File "/opt/openenergymonitor/emonhub/src/interfacers/EmonModbusTcpInterfacer.py", line 185, in read
    decoder = self.BinaryPayloadDecoder.fromRegisters(self.rVal.registers, byteorder=self.Endian.BIG, wordorder=self.Endian.BIG)
AttributeError: type object 'Endian' has no attribute 'BIG'

Do you think this is related to the long data type you mentioned in your issue?

Lawrie_Witham · 3 October 2024 13:04

Hi Barry,
You are using Modbus TCP; I use Modbus RTU, so my experience is not directly related. You have specified “datacode I”, which is a 2 byte integer, not a long. Is that correct?
The error chain is ending up at line 185 of EmonModbusTcpInterfacer.py, but this may be a red herring, as it is in an else-statement that may not usually run…
My suggestion would be to first check the parameters in emon.config against the datasheet of your device.

gadgetbazza · 4 October 2024 16:22

Thanks Lawrie,

All that you say is true, however, I have had my inverter running for 6 years with emoncms (only re-installed once about 18-24 months ago with data restored) and it seems that I have confused a couple of issues…

The intermittent data was caused by a change to my DHCP setup. That is now resolved.
I updated emonCMS as part of my attempts at resolution from 11.3 (I think), to 11.6.5. (Update log attached).

Since then I have errors reading the MODBUS data. I’ve removed all the registers from the Emonhub configuration and added a single register back of type I and also another that is type I, both result in the same error to do with BIG Endian as below:

Example of data type I

2024-10-04 17:00:07,096 DEBUG    ModbusTCP  expected bytes number after encoding: 8
2024-10-04 17:00:07,097 DEBUG    ModbusTCP  datacode I
2024-10-04 17:00:07,098 DEBUG    ModbusTCP  reading register #: 30202, qty #: 2, unit #: 3
2024-10-04 17:00:07,297 WARNING  ModbusTCP  Exception caught in ModbusTCP thread. Traceback (most recent call last):
  File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 31, in wrapper
    return func(*args)
  File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 105, in run
    rxc = self.read()
  File "/opt/openenergymonitor/emonhub/src/interfacers/EmonModbusTcpInterfacer.py", line 185, in read
    decoder = self.BinaryPayloadDecoder.fromRegisters(self.rVal.registers, byteorder=self.Endian.BIG, wordorder=self.Endian.BIG)
AttributeError: type object 'Endian' has no attribute 'BIG'

2024-10-04 17:00:07,311 WARNING  MainThread ModbusTCP thread is dead.

Example of data type I

2024-10-04 17:18:48,783 INFO     ModbusTCP  Not connected, retrying connect {'modbus_IP': '192.168.4.161', 'modbus_port': '502'}
2024-10-04 17:18:48,795 INFO     ModbusTCP  Opening modbusTCP connection: 502 @ 192.168.4.161
2024-10-04 17:18:48,796 DEBUG    ModbusTCP  expected bytes number after encoding: 8
2024-10-04 17:18:48,797 DEBUG    ModbusTCP  datacode i
2024-10-04 17:18:48,797 DEBUG    ModbusTCP  reading register #: 30980, qty #: 2, unit #: 3
2024-10-04 17:18:48,971 WARNING  ModbusTCP  Exception caught in ModbusTCP thread. Traceback (most recent call last):
  File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 31, in wrapper
    return func(*args)
  File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 105, in run
    rxc = self.read()
  File "/opt/openenergymonitor/emonhub/src/interfacers/EmonModbusTcpInterfacer.py", line 185, in read
    decoder = self.BinaryPayloadDecoder.fromRegisters(self.rVal.registers, byteorder=self.Endian.BIG, wordorder=self.Endian.BIG)
AttributeError: type object 'Endian' has no attribute 'BIG'

2024-10-04 17:18:49,076 WARNING  MainThread ModbusTCP thread is dead.

I would look to roll back to an earlier version to prove / disprove the issue, but I can’t see a way to do that easily from the UI as per the updates.

My inverter only has registers of type I and i, so I can’t check any others.

Guess my question now is ModBus broken in this release or has my installation broken somehow during the update?

Thanks for any thoughts.

update.log.txt (54.3 KB)

Lawrie_Witham · 4 October 2024 20:28

Hi Barry,

Maybe you do have a problem from the update. The error is being flagged up for the attribute Endian.BIG, not Endian.Big as I would have expected. According to the Pymodbus documentation, there was a change in version 3.5.0 where Endian.Big was changed to Endian.BIG. Maybe your interfacer and your Pymodbus.constants file are from different versions?
Best contact [email protected] for help.

TrystanLea · 7 October 2024 08:36

@jkjaer @bwduncan any ideas how we might fix this? @jkjaer looks like your recent commit may be related ? Change Endian.Big to Endian.BIG and unit to slave as per pymodbus change · openenergymonitor/emonhub@4cc7818 · GitHub

TrystanLea · 7 October 2024 08:42

A quick way to roll back in the mean time:

cd /opt/openenergymonitor/emonhub
git checkout 3241c0529b79a35e2b0be5c03b1127858b44ebc8

That would take you to just before @jkjaer’s change in May… but the issue may be not specifically related to that, e.g python core or library version changes interacting with the code.

jkjaer · 7 October 2024 09:13

Correct. I reported an issue in May where I found that the use of Endian.Big broke the EmonModbusTcpInterfacer class (see more here Change Endian.Big to Endian.BIG as per pymodbus change · Issue #213 · openenergymonitor/emonhub · GitHub). This happened on a new install of the most recent EmonSD image from February 24. I also fixed the problem by changing Endian.Big to Endian.BIG. It seems like @gadgetbazza has exactly the opposite problem which makes me wonder which version of Python is installed on his system. If we want a more robust solution, then we could check the Python version in src/interfacers/EmonModbusTcpInterfacer.py and then use either .Big or .BIG depending on that.

To see the exact changes made back in May, please see

github.com/openenergymonitor/emonhub

Change Endian.Big to Endian.BIG and unit to slave as per pymodbus change

Commit by

jkjaer - Change Endian.Big to Endian.BIG and unit to slave as per pymodbus change

openenergymonitor:master ← jkjaer:modbusInterfacerFix

In https://github.com/pymodbus-dev/pymodbus/pull/1743, pymodbus changed constant…s.Endian into being an enumeration, resulting in that Endian.Big was changed into Endian.BIG. Moreover, the read_holding_registers methods changed the unit parameter naming into slave with pymodbus 3.0 (see https://github.com/openenergymonitor/emonhub/issues/182 and https://github.com/pymodbus-dev/pymodbus/issues/1083). These changes have broken the EmonModbusTcpInterfacer.py class which uses Endian.Big for both the byte and word order and the unit parameter name. This commit fixes these problems. branches ModbusInterfacerFix

Lawrie_Witham · 7 October 2024 10:14

Excuse me butting in, but the problem is that the update is only requiring pymodbus >3.0, but the change to Endian.BIG was first made in pymodbus 3.5.0. A robust solution would therefore require checking the pymodbus version, not the Python version.

borpin · 7 October 2024 10:24

I think it is more likely that there is nothing in the update script that forces the update of any pip installed modules @TrystanLea.

python - Find outdated/updatable pip packages - Super User

borpin · 7 October 2024 10:27

Check

pip list --outdated

AFAIK, the update script never updates any packages installed via pip.

gadgetbazza · 7 October 2024 16:25

This is the output:

pi@emonpi:~ $ pip list --outdated
Package                         Version   Latest    Type
------------------------------- --------- --------- -----
Adafruit-Blinka                 8.47.0    8.48.0    wheel
adafruit-circuitpython-requests 4.1.6     4.1.7     wheel
Adafruit-PlatformDetect         3.74.0    3.75.0    wheel
async-timeout                   4.0.2     4.0.3     wheel
attrs                           20.3.0    24.2.0    wheel
Automat                         20.2.0    24.8.1    wheel
bcrypt                          3.1.7     4.2.0     wheel
certifi                         2020.6.20 2024.8.30 wheel
chardet                         4.0.0     5.2.0     wheel
click                           7.1.2     8.1.7     wheel
colorama                        0.4.4     0.4.6     wheel
colorzero                       1.1       2.0       wheel
configobj                       5.0.6     5.0.9     wheel
constantly                      15.1.0    23.10.4   wheel
cryptography                    3.3.2     43.0.1    wheel
Deprecated                      1.2.13    1.2.14    wheel
distro                          1.5.0     1.9.0     wheel
gpiozero                        1.6.2     2.0.1     wheel
hyperlink                       19.0.0    21.0.0    wheel
idna                            2.10      3.10      wheel
incremental                     17.5.0    24.7.2    wheel
minimalmodbus                   2.0.1     2.1.1     wheel
numpy                           1.19.5    2.0.2     wheel
packaging                       21.3      24.1      wheel
paho-mqtt                       1.6.1     2.1.0     wheel
picamera2                       0.3.6     0.3.22    wheel
Pillow                          8.1.2     10.4.0    wheel
pip                             20.3.4    24.2      wheel
pyasn1                          0.4.8     0.6.1     wheel
pyasn1-modules                  0.2.1     0.4.1     wheel
PyHamcrest                      1.9.0     2.1.0     wheel
pymodbus                        3.0.2     3.7.2     wheel
pyOpenSSL                       20.0.1    24.2.1    wheel
pyparsing                       3.0.9     3.1.4     wheel
pyserial                        3.5b0     3.5       wheel
python-prctl                    1.7       1.8.1     sdist
redis                           4.3.4     5.1.1     wheel
requests                        2.25.1    2.32.3    wheel
rpi-rfm69                       0.3.4     0.7.0     wheel
RPi.GPIO                        0.7.0     0.7.1     wheel
sdm-modbus                      0.6.1     0.7.0     wheel
service-identity                18.1.0    24.1.0    wheel
setuptools                      52.0.0    75.1.0    wheel
simplejpeg                      1.6.4     1.7.6     sdist
spidev                          3.5       3.6       wheel
ssh-import-id                   5.10      5.11      wheel
toml                            0.10.1    0.10.2    wheel
Twisted                         20.3.0    24.7.0    wheel
urllib3                         1.26.5    2.2.3     wheel
v4l2-python3                    0.3.1     0.3.5     wheel
wheel                           0.34.2    0.44.0    wheel
wrapt                           1.14.1    1.16.0    wheel
zope.interface                  5.2.0     7.0.3     wheel

I see pymodbus in there, do I just update that one? If so, would you be able to share the command syntax please?

Thanks
Barry

alexandrecuer · 7 October 2024 19:25

@TrystanLea : this is why you have to go for containers
@gadgetbazza : if you have docker, try this : https://hub.docker.com/r/alexjunk/emonhub

gadgetbazza · 7 October 2024 19:39

Thanks Trystan,

I’ve gone with your suggested roll back for the minute. Not sure what the impact of doing other updates it. My hardware is a simple emonBase for reference. Not running anything else on it and used the emonSD-10Nov22 image if that’s of any use.

I can confirm the modbus data is reading again now.

Thanks
Barry

gadgetbazza · 7 October 2024 19:42

I don’t run docker on my EmonBase device, I do have it elsewhere on a bigger machine. I tried setting up emoncms there once, but it became too convoluted with the location of the EmonTX’s etc, so I stuck with the emonBase as it’s been a solid unit.

borpin · 8 October 2024 06:30

Yes an old version of pymodbus is the issue @TrystanLea

Google is your friend

pip install pymodbus -U

I think. note don’t run as sudo.

The other solution would be to get a new SDImage and transfer the data/settings to that.