Odd error in single sensor config in emonhub

alandpearson · 12 February 2023 13:54

Seems I spoke too soon.
I added some config for decoding to emonhub.conf (taken from my working emonpi).
While emonhub started ok, it crashes every time it wants to decode things

2023-02-12 13:40:46,694 DEBUG    RFM2Pi     23 NEW FRAME : OK 20 91 08 (0)
2023-02-12 13:40:46,702 WARNING  RFM2Pi     Exception caught in RFM2Pi thread. Traceback (most recent call last):
  File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 31, in wrapper
    return func(*args)
  File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 105, in run
    rxc = self._process_rx(rxc)
  File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 301, in _process_rx
    datacodes = ehc.nodelist[node]['rx']['datacodes'].copy()
AttributeError: 'str' object has no attribute 'copy'

Looks like this could be php version thing.

Offending line 301:

            datacodes = ehc.nodelist[node]['rx']['datacodes'].copy

Removing the .copy fixes the problem, but I wonder does this issue exist in other emon code that is maybe built for php 8.1

borpin · 12 February 2023 14:06

No, this an error from Python not PHP/Apache.

github.com

openenergymonitor/emonhub/blob/4539b9e928e3f7b5bc3dd1922440cf7e725ac6b9/src/emonhub_interfacer.py#L301-L301


      
          datacodes = ehc.nodelist[node]['rx']['datacodes'].copy()

Is there a valid datacodes entry in the config file?

alandpearson · 12 February 2023 14:07

Yes, there is.

[[20]]
    nodename = emonglcd
    firmware =V1_1
    hardware = emonglcd
    [[[rx]]]
     names = temperature,
     datacodes = h
     scales = 0.01,1
     units = c

As i said removing the .copy suffix fixes the issue and the packet is decoded correctly and onward sent to MQTT as expected.

borpin · 12 February 2023 14:09

Except for the trailing comma

alandpearson · 12 February 2023 14:12

Nothing wrong with your eyesight !

I’ll remove that and see if it still occurs (I suspect it will).

Comma removed, crash still occurs.

borpin · 12 February 2023 14:16

Oh you have one datacode and 2 scales I think. And only one name.

All 3 4 arrays need to be the same size I suspect.

[edit]
datacode or datacodes (singular/plural).

https://docs.openenergymonitor.org/emonhub/configuration.html#datacodes

alandpearson · 12 February 2023 14:27

Another good spot.
Removed the spurious entry, still crashing.

Singular vs Plural … my understanding is singular seems to be only used if you have a load of datacodes the same type, saves putting an entry for each one.

It seems to be the version of python installed doesn’t like the .copy as it isn’t part of the string object ?

I assume your emonhub on your piW works. Can you tell me the version of python installed ?

root@heatmeter:~# python3 --version
Python 3.9.2

root@heatmeter:~# python --version
Python 3.9.2

borpin · 12 February 2023 14:28

Can you post your current config?

alandpearson · 12 February 2023 14:31

root@heatmeter:~# cat /etc/emonhub/emonhub.conf 
#######################################################################
#######################      emonhub.conf     #########################
#######################################################################
### emonHub configuration file, for info see documentation:
### https://github.com/openenergymonitor/emonhub/blob/emon-pi/configuration.md
#######################################################################
#######################    emonHub  settings    #######################
#######################################################################

[hub]
### loglevel must be one of DEBUG, INFO, WARNING, ERROR, and CRITICAL
loglevel = DEBUG
### Uncomment this to also send to syslog
# use_syslog = yes
#######################################################################
#######################       Interfacers       #######################
#######################################################################

[interfacers]
### This interfacer manages the RFM12Pi/RFM69Pi/emonPi module
[[RFM2Pi]]
    Type = EmonHubJeeInterfacer
    [[[init_settings]]]
        com_port = /dev/ttyUSB0
        com_baud = 57600                        # 9600 for old RFM12Pi
    [[[runtimesettings]]]
        pubchannels = ToEmonCMS,
        subchannels = ToRFM12,
#
        group = 210
        frequency = 433
        baseid = 5                              # emonPi / emonBase nodeID
        calibration = 230V                      # (UK/EU: 230V, US: 110V)
        quiet = true                            # Disable quite mode (default enabled) to enable RF packet debugging, show packets which fail crc
        # interval =  300                         # Interval to transmit time to emonGLCD (seconds)


[[MQTT]]

    Type = EmonHubMqttInterfacer
    [[[init_settings]]]
        mqtt_host = emonpi
        mqtt_port = 1883
        mqtt_user = emonpi
        mqtt_passwd = emonpimqtt2016

    [[[runtimesettings]]]
        pubchannels = ToRFM12,
        subchannels = ToEmonCMS,

        # emonhub/rx/10/values format
        # Use with emoncms Nodes module
        node_format_enable = 1
        node_format_basetopic = emonhub_test/

        # emon/emontx/power1 format - use with Emoncms MQTT input
        # http://github.com/emoncms/emoncms/blob/master/docs/RaspberryPi/MQTT.md
        nodevar_format_enable = 1
        nodevar_format_basetopic = emon_test/

        # Single JSON payload published  - use with Emoncms MQTT
        node_JSON_enable = 0
        node_JSON_basetopic = emon/


#######################################################################
#######################          Nodes          #######################
#######################################################################

[nodes]

## See config user guide: https://github.com/openenergymonitor/emonhub/blob/emon-pi/conf/emonhub.conf


[[20]]
    nodename = emonglcd
    firmware =V1_1
    hardware = emonglcd
    [[[rx]]]
     names = temperature
     datacodes = h
     scales = 0.01
     units = c

borpin · 12 February 2023 15:53

Try datacode and scale?

It has to be a configuration error else everyone’s system would be failing!

Robert.Wall · 12 February 2023 16:14

Definitely not scale - it is always scales, names & units. The questionable one is only datacode, and I think I agree with @alandpearson - it should be datacodes as one is specified and there’s only one item; it would be datacode if the same one datacode applied to all the items in the message. What is not clear is whether that rule applies with only one item present.

alandpearson · 12 February 2023 16:15

I’d tend to agree with you, but what is weird is that exact config works on my production emonpi

Regardless, I tried:

[nodes]

## See config user guide: https://github.com/openenergymonitor/emonhub/blob/emon-pi/conf/emonhub.conf

[[20]]
    nodename = emonglcd
    firmware =V1_1
    hardware = emonglcd
    [[[rx]]]
     names = temperature
     datacode = h
     scale = 0.01
     units = c

Same error.

2023-02-12 16:12:26,930 DEBUG    RFM2Pi     16 NEW FRAME : OK 20 91 08 (0)
2023-02-12 16:12:26,940 WARNING  RFM2Pi     Exception caught in RFM2Pi thread. Traceback (most recent call last):
  File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 31, in wrapper
    return func(*args)
  File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 105, in run
    rxc = self._process_rx(rxc)
  File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 410, in _process_rx
    names = ehc.nodelist[node]['rx']['names'].copy()
AttributeError: 'str' object has no attribute 'copy'

I retract my assertion that removing the .copy() fixes it. It stops the error, but then later fails on other .copy() statements in emonhub_interfacer.py
Removing the other .copy() statements stops the error but no publishing that I can see.

This is a very weird one, config is as simple as it can get on a freshly installed (emonhub only) box.

borpin · 12 February 2023 16:18

Not according to the docs link I posted;

scale definitely allowed

github.com

openenergymonitor/emonhub/blob/4539b9e928e3f7b5bc3dd1922440cf7e725ac6b9/src/emonhub_interfacer.py#L45-L56


      
          # Initialise settings
          self._defaults = {'pause': 'off',
                            'interval': 0,
                            'datacode': '0',
                            'scale': '1',
                            'timestamped': False,
                            'targeted': False,
                            'nodeoffset': '0',
                            'pubchannels': [],
                            'subchannels': [],
                            'batchsize': '1'}

Robert.Wall · 12 February 2023 16:23

Is scale there the internal name of the variable, or the text tag, or both?

borpin · 12 February 2023 16:26

github.com

openenergymonitor/emonhub/blob/4539b9e928e3f7b5bc3dd1922440cf7e725ac6b9/src/emonhub_interfacer.py#L365-L381


      
          # check if node is listed and has individual scales for each value
          if node in ehc.nodelist and 'rx' in ehc.nodelist[node] and 'scales' in ehc.nodelist[node]['rx']:
              scales = ehc.nodelist[node]['rx']['scales'].copy()
              # === Removed check for scales length so that failure mode is more gracious ===
              # Discard the frame & return 'False' if it doesn't match the number of scales
              # if len(decoded) != len(scales):
              #     self._log.warning("%d Scales %s for RX data : %s not suitable", rxc.uri, scales, rxc.realdata)
              #     return False
              # else:
                    # Determine the expected number of values to be decoded
                    # Set decoder to "Per value" scaling using scale 'False' as flag
              #     scale = False
              if len(scales) > 1:
                  scale = False
              else:
                  scale = "1"
          else:

borpin · 12 February 2023 16:28

Can you try 2 registers?

Might be an obscure bug with only one value to decode.

@bwduncan - can you have a look, please?

[edit]
My suspicion is that Python doesn’t like reading it as it isn’t an array. Probably something it tolderated in the past, but these sorts of things have been tightened up as the language has progressed.

alandpearson · 12 February 2023 16:35

In an effort to further debug, I copied my entire emonhub.conf over from my prod emonpi. This contains a lot of devices of varying complexity

Oddly, some other devices work, with emonhub on piZ relaying correctly via MQTT.

The single value device (id 20) doesn’t work and crashes emonhub with the .copy() error when packets are received from it. When this entry is removed from emonhub.conf, I no longer get the .copy() error.

I’ll keep trying to hone in on what is causing this.

alandpearson · 12 February 2023 16:56

I am now pretty sure it is the single value device causing this issue.

This device (actually pair of them) is an emonGLCD transmitting it’s temperature.

Every other device that has ‘checked in’ so far to piZ has worked without issue and a lot of them are non-trivial packet streams.

Is it seems @borpin hypothesis may be correct, this looks to be an issue with a single value.

Sorry to rain on this thread with this, it seems it’s not related at all to the Raspberry Pi Zero (except to say, it works on mine with this exception).

borpin · 12 February 2023 16:58

@alandpearson note I have split this discussion to its own thread.

alandpearson · 12 February 2023 18:11

@borpin is correct.
When the names/scales/datacodes (plural) has a singular value, python does not seem to hand back an object, but a string. Therefore .copy() fails in the places where this config is processed and crashes.

A dirty workaround that I found seems to work
Use singular for scale and datacode, plural for names with a dummy value.
There is no singular for names.

i.e.

[[20]]
    nodename = emonglcd
    firmware =V1_1
    hardware = emonglcd
    [[[rx]]]
     names = temperature, dummy
     datacode = h
     scale = 0.01
     units = c

I personally think it’s very confusing to have singular and plural config items.
It may be a better approach if the system would automatically apply logic that if only a single value is specified for scales or datacodes it will use that for all received packets (as the singular does today).

Sadly, my python isn’t good enough to fix it.

Hope this helps someone else, and huge thanks to Brian.