Emonhub Pulse Interfacer

[bwduncan (Bruce)]

I pushed some changes. Please try now.

[borpin (Brian Orpin)]

Nearly there - thanks for your help. .

1. The value pushed in via the pulse_pins parameter is a string.
2. As pulse_pins & bouncetime are init setting I don’t think they don’t need to be setup as a self._pulse_settings - these are runtime settings - but it didn’t like it when I removed them!
3. There needs to be something in the read that checks for a pulse being received otherwise returns false. I added a counter to the process_pulse callback but I’m not sure if that is the right way to do that or if there is an event that can be used.

I think it will be better if the interfacer simply takes a single value as the pin. If more than one pulse monitor is needed, simply start another one with a separate config in emonhub. Will that work from a Python and GPIO interface front?

TODO

1. Change pulse_pins to pulse_pin
2. Can the Dummy-3 that appears in the log be set to something sensible - seems to be the callback process name.

So this works!

from emonhub_interfacer import EmonHubInterfacer
from collections import defaultdict
import time
import atexit
import RPi.GPIO as GPIO

import Cargo

"""class EmonhubPulseCounterInterfacer

Monitors GPIO pins for pulses

"""

class EmonHubPulseCounterInterfacer(EmonHubInterfacer):

    def __init__(self, name, pulse_pins=None, bouncetime=1):
        """Initialize interfacer

        """

        # Initialization
        super().__init__(name)

        self.gpio_pin = int(pulse_pins)

        self._pulse_settings = {
            'pulse_pins': pulse_pins,
            'bouncetime' : bouncetime,
        }

        self._settings.update(self._pulse_settings)
        self.pulse_count = defaultdict(int)

        self.pulse_received = 0

        self.init_gpio()

    def init_gpio(self):
        """Register GPIO callbacks

        """

        atexit.register(GPIO.cleanup)
        GPIO.setmode(GPIO.BOARD)
        self._log.debug('Pulse pin set to: %d', self.gpio_pin)
        GPIO.setup(self.gpio_pin, GPIO.IN, pull_up_down=GPIO.PUD_DOWN)
        GPIO.add_event_detect(self.gpio_pin, GPIO.FALLING, callback=self.process_pulse, bouncetime=int(self._settings['bouncetime']))

    def process_pulse(self, channel):
        self.pulse_count[channel] += 1
        self._log.debug('Pulse Channel %d pulse: %d', channel, self.pulse_count[channel])
        self.pulse_received += 1

    def read(self):

        if self.pulse_received == 0:
            return False

        self.pulse_received = 0
        t = time.time()
        f = '{t} {nodeid}'.format(t=t, nodeid=self._settings['nodeoffset'])
        f += ' {}'.format(self.pulse_count[self.gpio_pin])

        # Create a Payload object
        c = Cargo.new_cargo(rawdata=f)

        f = f.split()

        if int(self._settings['nodeoffset']):
            c.nodeid = int(self._settings['nodeoffset'])
            c.realdata = f
        else:
            c.nodeid = int(f[0])
            c.realdata = f[1:]

        return c


    def set(self, **kwargs):
        super().set(**kwargs)

        for key, setting in self._pulse_settings.items():

            if key not in kwargs:
                setting = self._pulse_settings[key]
            else:
                setting = kwargs[key]

            if key in self._settings and self._settings[key] == setting:
                continue
            else:
                self._log.warning("'%s' is not valid for %s: %s", setting, self.name, key)

As a note I needed to add the emonhub user to the gpio group. sudo adduser emonhub gpio. That will at some point need to go into the installer.

emonhub log

2020-06-11 09:41:00,929 INFO     MainThread EmonHub emonHub emon-pi variant v3-beta
2020-06-11 09:41:00,932 INFO     MainThread Opening hub...
2020-06-11 09:41:00,936 INFO     MainThread Logging level set to DEBUG
2020-06-11 09:41:00,940 INFO     MainThread Creating EmonHubPulseCounterInterfacer 'pulse2'
2020-06-11 09:41:00,951 DEBUG    MainThread Pulse pin set to: 15
2020-06-11 09:41:00,957 DEBUG    MainThread Setting pulse2 nodeoffset: 3
2020-06-11 09:41:00,960 DEBUG    MainThread Setting pulse2 pubchannels: ['ToEmonCMS']
2020-06-11 09:41:00,979 INFO     MainThread Creating EmonHubMqttInterfacer 'MQTT'
2020-06-11 09:41:00,988 DEBUG    MainThread Setting MQTT pubchannels: ['ToRFM12']
2020-06-11 09:41:00,991 DEBUG    MainThread Setting MQTT subchannels: ['ToEmonCMS']
2020-06-11 09:41:00,994 INFO     MainThread Setting MQTT node_format_enable: 1
2020-06-11 09:41:00,997 INFO     MainThread Setting MQTT nodevar_format_enable: 1
2020-06-11 09:41:01,000 INFO     MainThread Setting MQTT nodevar_format_basetopic: emon/
2020-06-11 09:41:01,848 DEBUG    Dummy-3    Pulse Channel 15 pulse: 1
2020-06-11 09:41:01,890 DEBUG    pulse2     1 NEW FRAME : 1591864861.8900208 3 1
2020-06-11 09:41:01,894 DEBUG    pulse2     1 Timestamp : 1591864861.890199
2020-06-11 09:41:01,898 DEBUG    pulse2     1 From Node : 3
2020-06-11 09:41:01,902 DEBUG    pulse2     1    Values : [1591864861.8900208, 3, 1]
2020-06-11 09:41:01,905 DEBUG    pulse2     1 Sent to channel(start)' : ToEmonCMS
2020-06-11 09:41:01,908 DEBUG    pulse2     1 Sent to channel(end)' : ToEmonCMS

emonhub.conf

[[pulse2]]
    Type = EmonHubPulseCounterInterfacer
    [[[init_settings]]]
        pulse_pins = 15
    [[[runtimesettings]]]
        pubchannels = ToEmonCMS,

        nodeoffset = 3

[bwduncan (Bruce)]

1. Yes the string is split into a list of ints later on.

2. Again, not sure what the difference is, but you’re right that those settings aren’t needed once everything is set up.

3. Is that necessary? If read is called and returns the current pulse count, that should be enough. emonhub has rate limiting and buffering layered on top. But if it doesn’t work…! Perhaps the author of emonhub could weigh in and say what the intended interface is in this case?

TODO:

1. Yeah ok, I’m not invested in either implementation. Normally one would avoid using a module like this in multiple threads, but as far as I can see the biggest risk is calling GPIO.cleanup will affect other threads, but since we only do that at exit, it’s not a problem. In principle if an interfacer is removed we should call cleanup, but there’s no nice interface to do that, and I don’t feel like overriding run just to do that…

2. Not easily. The thread is created deep in the GPIO module to trigger the callback. There’s no sensible interface I can see to rename it. We don’t really need to log every single pulse, though! That’s just for debugging right now. We shouldn’t be doing anything which takes a considerable amount of time in this callback anyway. If the logging blocks waiting to write to disk, will we miss pulses? I don’t know…

[borpin (Brian Orpin)]

@bwduncan

Is that necessary? If read is called and returns the current pulse count, that should be enough.

It is replying every 0.1 cycle. Look at the Tx3 serial interfacer, it checks to see if there is data and if not exits. Each read results in data to emoncms.

@bwduncan

Yeah ok, I’m not invested in either implementation.

I think the data coming out will make more sense else one number changes and the other doesn’t and really we want data when it increments.

@bwduncan

Not easily. The thread is created deep in the GPIO module to trigger the callback.

OK, that is fine just wondered. Dummy seemed an odd name!

[borpin (Brian Orpin)]

I have submitted a PR if anyone is interested. Many thanks to @bwduncan for his assistance.

Pulse Interfacer

[bwduncan (Bruce)]

Some minor points:

Bug in the very first (missing) character!

pulse_pins should be singular

pulse_received can just be a boolean, set to True in the callback, set to False in read

set has what look like debugging log statements with unhelpful messages! I suppose in an ideal world this method would be refactored and instead of this code being copied around every module it should be done in the superclass.

If pulse_pins is going to be removed from settings, bouncetime should be too. Either way there’s an inconsistency there.

Bruce

[borpin (Brian Orpin)]

@bwduncan

pulse_pins should be singular

It should be - I had changed it but it must have got lost.

@bwduncan

pulse_received can just be a boolean, set to True in the callback, set to False in read

When I did it I wondered if there was a possibility of getting 2 callbacks before the read, but I suspect that is highly unlikely and shouldn’t matter. Changed to boolean.

@bwduncan

If pulse_pins is going to be removed from settings, bouncetime should be too. Either way there’s an inconsistency there.

No it needs to be in the setting, it is just one pin instead of suggesting more than one.

@bwduncan

set has what look like debugging log statements with unhelpful messages!

Gone - just me trying to work out what path was being taken through the logic.

Changes pushed - thanks.

[bwduncan (Bruce)]

Actually, if we’re only targeting a single pin with each instance, we don’t need pulse_count to be a dictionary. That will simplify the code a lot, just initialise it to 0.

The nodeid can be set in one line with

c.nodeid = int(self._settings.get('nodeoffset', 0))

which won’t throw an exception if nodeoffset isn’t set.

The set method doesn’t do anything now, so it can be deleted. The _pulse_settings attribute can be deleted from __init__