Installation Emonpi and emonTx

Getting Started
,
62

I haven’t looked at the source code yet but that shouldn’t happen and I don’t think it would be dependent on any payload content or config so it might be a bug. Can you post the node definition from emonhub.conf for node 5, please copy and paste exactly as minor typo’s matter.

Also can you post the RFM2Pi interfacer config too please.

Can you confirm you are running the latest emonhub, the version is printed to the log when emonhub starts, so use sudo systemctl restart emonhub and check the emonhub.log for a version number.

can you reboot for the changes to take effect and update us on any changes, or just post some emonhub.log again.

63

Firmware version: [emonpi.29.20]

[[5]]
    nodename = emonpi
    [[[rx]]]
        names = power1,power2,power1pluspower2,vrms,t1,t2,t3,t4,t5,t6,pulsecount
        datacode = h, h, h, h, h, h, h, h, h, h, L
        scales = 1,1,1,0.01,0.1,0.1,0.1,0.1,0.1,0.1,1
        units = W,W,W,V,C,C,C,C,C,C,p

Logs

2021-02-10 11:14:36,766 INFO     MainThread Setting RFM2Pi frequency: 433 (4b)
2021-02-10 11:14:37,768 INFO     MainThread Setting RFM2Pi group: 210 (210g)
2021-02-10 11:14:38,770 INFO     MainThread Setting RFM2Pi quiet: 1 (1q)
2021-02-10 11:14:39,772 INFO     MainThread Setting RFM2Pi calibration: 230V (1p)
2021-02-10 11:14:40,775 DEBUG    MainThread Setting RFM2Pi pubchannels: ['ToEmonCMS']
2021-02-10 11:14:40,776 DEBUG    MainThread Setting RFM2Pi subchannels: ['ToRFM12']
2021-02-10 11:14:40,778 WARNING  MainThread Setting emoncmsorg apikey: invalid format
2021-02-10 11:14:40,784 DEBUG    RFM2Pi     18 NEW FRAME : OK 5 0 0 0 0 0 0 246 89 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 (-0)
2021-02-10 11:14:40,786 WARNING  RFM2Pi     Exception caught in RFM2Pi thread. Traceback (most recent call last):
  File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 32, in wrapper
    return func(*args)
  File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 101, in run
    rxc = self._process_rx(rxc)
  File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 347, in _process_rx
    elif len(rxc.realdata) % ehc.check_datacode(datacode) != 0:
  File "/opt/openenergymonitor/emonhub/src/emonhub_coder.py", line 10, in check_datacode
    return struct.calcsize(datacode)
TypeError: Struct() argument 1 must be a str or bytes object, not list

2021-02-10 11:14:41,013 WARNING  MainThread RFM2Pi thread is dead.
2021-02-10 11:14:41,014 WARNING  MainThread Attempting to restart thread RFM2Pi (thread has been restarted 17 times...)
2021-02-10 11:14:41,016 INFO     MainThread Creating EmonHubJeeInterfacer 'RFM2Pi'
2021-02-10 11:14:41,017 DEBUG    MainThread Opening serial port: /dev/serial0 @ 38400 bits/s
2021-02-10 11:14:43,027 INFO     MainThread RFM2Pi device firmware version: [emonPi.29.20]
2021-02-10 11:14:43,028 INFO     MainThread RFM2Pi device current settings:  E i5 g210 @ 433 MHz q1 USA 0 
2021-02-10 11:14:43,029 INFO     MainThread Setting RFM2Pi calibration: 230V (1p)
2021-02-10 11:14:44,031 DEBUG    MainThread Setting RFM2Pi pubchannels: ['ToEmonCMS']
2021-02-10 11:14:44,032 DEBUG    MainThread Setting RFM2Pi subchannels: ['ToRFM12']
2021-02-10 11:14:44,034 WARNING  MainThread Setting emoncmsorg apikey: invalid format
2021-02-10 11:14:44,038 DEBUG    RFM2Pi     device settings updated: E i5 g210 @ 433 MHz q1 USA 0
64

should be

datacodes = h, h, h, h, h, h, h, h, h, h, L

plural datacodeS

1 Like
65

Thanks its solved. I got the readings back

Capture
Capture1626×509 7.5 KB

66

Tagging in @bwduncan as I believe this exception is a biproduct of the change to use return struct.calcsize(datacode) (in PR Simplify code. · openenergymonitor/emonhub@e003356 · GitHub)

Previously the datacode was tested against a list and just absorbed the error if it wasn’t a valid code,

whereas now an invalid code results in an exception

I do not know where the list is slipping in exactly but it is a common mistake for users to use “datacode” and “datacodes” in the wrong way so the user error must be handled without generating an exception.

Perhaps a quick test to confirm the datacode is a single char before struct.calcsize(datacode) and maybe even a log message to tell users about the discrepancy would be nice, but we cannot be sure what the user wants so the message would have to be on a “check settings” theme rather than a “correct the typo” type message.

Or maybe just splitting the line in the try: so it doesn’t attempt to return the exception might work

def check_datacode(datacode):
    try:
        x= struct.calcsize(datacode)
        return x
    except struct.error:
        return False
67

. . and the LCD?

68

Its back as well. I can see the readings on the LCD too.

Thanks to you and @borpin. you both are LEGEND and did your best.

Thanks Again. Cheers

1 Like
69

Hi Paul,

Feels like you’ve got it in for me! (Just kidding)

Unfortunately I can’t take the blame for this one since the code behaves the same as before. If you tell it that datacode is a list, it will try to use that list as an index, which is obviously nonsense. As people used to say: Garbage in, garbage out.

From the point of view of check_datacode I think raising the exception is absolutely the right thing to do here. It is definitely an exceptional state and there isn’t anything we can do about it. The program should stop until the input is fixed.

However it would be easy for the higher level code to check that, for example, datacode is in fact a single character. It could give a helpful message at that point because it has enough context to say something useful.

Unfortunately splitting the line does nothing to help. You don’t return exceptions, they represent a separate exit path from functions, and your version is functionally identical to the existing code (with the exception that it creates a local variable x)

To be honest, return False has always bothered me here. False isn’t a size, it has no business being returned from this function. If check_datacode fails, it should raise an exception and the caller should know what to do (log an error and quit, probably, which is exactly what happens in this case).

Bruce

70
1 Like
71

I certainly do not mean to if that’s how it appears. I tagged you in as yes you changed the code at the pivotal point, but also because you have recently been through most of the code and are probably a more proficient coder than I am. I sorta’ hoped you would look, say “ahh yes!” and know what or where to tackle this. I wrote the original code back in 2014 and it was changed and taken out of my hands in 2015, since then many hands have been at it. I can’t rely on memory of what I originally coded/intended nor read the code to an “absolute” level, I need to digest and go over it.

However, nothing was aimed at you specifically, just hoped you might help with a remedy perhaps. But don’t worry I will look at it as we do seem to have differing opinions on how emonhub should function. It was originally intended to run 24/7 continuously without needing restarts etc and it certainly under no circumstance should raise and exception and crash the thread unless there is a bug to be fixed to ensure it doesn’t happen again.

Whether by fluke or not, up til now the datacodes v datacode user error has not caused an issue other than the questionable node not being processed, which is right, if any part of the node definition is wrong it is better that node doesn’t reach the emoncms server. Now we are in a position where the thread will crash out and all data via that interfacer is stopped. I dislike the way emonhub has now been coded to keep restarting when there is a standing exception, but I have no control over that.

I will come back to this and try for a better solution when I can. Right now I have a very sick ram that needs my attention, never a dull moment round here.

[edit] sorry didn’t notice you had posted a PR whilst I was (slowly) writing my post, I’ll take a look later as I really need to go now. :smile: :+1:

72

Not at all, I am grateful for you tagging me in important posts like this. I hope my input is helpful.

I haven’t forgotten your other post about logging. I hope to get back to that this week. Unfortunately the day job is not the only thing demanding my time! I think we basically agree and we (and others) have good ideas which could improve emonhub, make it more usable and maintainable into the future.

This is the curse of programming. When writing the code it seems blatantly obvious what you mean, yet when you read your own code years later it makes no sense! Writing comments to explain what you mean as you write it seems totally pointless, until you come back to it!

There does seem to be a mishmash of error paths in the code. Some things return False, or None to signal an error. Some things raise an exception. Sometimes those exceptions are handled. The code could make much better use of exceptions to help debugging (you get a traceback for free, if nothing else!) but with a top-level exception handler which will attempt to restart if something goes wrong. Restarting indefinitely seems a bit pointless, though. You don’t want the system to hide failing hardware from you.

However, I do believe that config errors should be fatal. There is no use in blindly carrying on when we know the user has configured something wrongly. If there is any chance that the user is still sitting there near the config file, it’s much better to stop immediately and alert them.

Good luck with the RAM!

73

Don’t I know it. I was once told by a senior engineer (I think I might even have been a student apprentice still) “If you’re called out to a breakdown at 3 am and you can’t understand what you did, it was too complicated to begin with.”

Or sometimes better still: writing comments to explain what you intend… and what you don’t, can be extremely useful.

1 Like
74

Since the OP’s main issue has been resolved…

Quite true.

The DS3231 has an integrated temperature compensated crystal oscillator that gives it
a drift spec of ±2 minutes per year.

Ref: DS3231.pdf (824.1 KB)

That’s not to say it won’t drift, because it will. But, the drift will be fairly small.

75

It is that the system clock will drift from the RTC not the RTC drifting from real time that is the issue.

Under normal circumstances, the system clock is regularly updated by NTP (which updates the RTC).

With no network, so no NTP the inbuilt systems do not default to updating the system clock from the RTC (it seems). This needs to be done manually.

It is pretty hard to search for as most results assume system → RTC rather than RTC → system (other than at boot).

76

Gotcha.

On a Pi with an RTC, running hwclock --hctosys will set the system clock from the RTC.
A cron job running 1 or more times a day (depending on how bad the drift is) should do it.
(hwclock -s is equivalent to hwclock --hctosys, but the latter is a good mnemonic)

Ref:

2 Likes
77

I note that tutorial fiddles with boot. Actually that is not required as timedatectl will do that on boot anyway.

It also does not set it up to subsequently use the RTC.

As I say, I’ve not found a good tutorial to do this.

I was getting issues with one Pi where the fakehwclock was always wrong so nothing really worked on reboot until NTP had kicked in. I fixed it by changing the setting so the saved time was updated more regularly.

78

I should have mentioned that the main point for referencing the page was to show that updating
the system clock from the RTC can be done.

As you’ve already mentioned, with no network connection, NTP isn’t going to work.

If the RTC is set manually, then hwclock --hctosys runs via a cron job at regular intervals,
given the fairly minimal drift of the DS3231 RTC, he should be OK.

1 Like