NULL inputs after replacement SD card and Archive Import

Hardware emonPi
,
1

My system stopped on 14 Oct 2023 and today I had some time to look at fixing it.

Overview (as at time of issue occurring)

  • Raspberry Pi 2B v1.1 with a RFM69Pi (SKU: RFM69Pi_433), software up to date as of 13/10/2023
  • Pi Power: PoE splitter from a POE+ TP Link switch
  • 1 x emonTx (SKU: emonTxV3) with 2 x 100A max clip-on current sensor CT (SKU: SCT013-000), Optical Utility Meter LED Pulse Sensor, Power Supply & RJ45 DS18B20
  • 2 x emonTH (SKU: emonTH_V2), one with 2 x DS18B20 (SKU: ERTSK)

After resolving several issues, some of my own making (not really relevant to my ultimate issue but might help others in the future):

  • Using a 60W mobile phone PSU which was swapped out for a genuine Raspberry Pi PSU due to power supply issues (i.e. crashed soon after boot).
  • Running the latest Raspberry Pi OS (Legacy) with the RFM69Pi attached which failed to completely boot on 2 x Pi 2B v1.1’s, removing the RFM69Pi resolved this.

My actual issue:
On watching the boot-up of the original SD card, it stopped at 70% of disk 1 progress check (after a message of ‘random: 7 urandom warning(s) missed due to ratelimiting’, this is after a start failure of the Uncomplicated Firewall.

I have created a new SD using the latest emonSD image (emonSD-10Nov22 (Stable)) by following the instructions here:

Then I restored a the latest backup archive that I had (13/10/2023) using the instructions here:

The import appears to have been successful with the original user/password login and the input definitions looking correct.
However there is no current data received on the ‘Inputs’ GUI.
I have attached the EmonHub debug log file.

I have seen the recent topic of:

As a result I have looked at the post:

Before I start to spend any more time looking into whether I need to perform any firmware updates to accommodate the CM operational mode, please could someone assist in the easiest way forward for me to restore my set-up.

Thanks in advance.

2

Sorry I cannot upload the emonhub.txt file because I get the error:
Sorry, the file you are trying to upload is not authorised (authorised extensions: jpg, jpeg, png, gif, zip, tar, ino, ico, txt, pdf, sch, brd, odt, gz, mp4, mov, ods, csv, doc, webm, bin, hex, webp, xlsx, xls).

3

Here is that last few lines of the emonhub file, which indicates an issue with the MQTT Server.

2023-11-05 21:11:00,921 DEBUG    MainThread Signal 15 received.
2023-11-05 21:11:00,988 INFO     MainThread Exiting hub...
2023-11-05 21:11:01,241 INFO     MainThread Exit completed
2023-11-05 21:11:09,258 INFO     MainThread EmonHub v2.5.8
2023-11-05 21:11:09,260 INFO     MainThread Opening hub...
2023-11-05 21:11:09,260 INFO     MainThread Running as user: pi
2023-11-05 21:11:09,261 INFO     MainThread Creating EmonHubJeeInterfacer 'RFM2Pi'
2023-11-05 21:11:11,271 INFO     MainThread RFM2Pi device firmware version: [RF12demo.14]
2023-11-05 21:11:11,273 INFO     MainThread RFM2Pi device current settings:  E i5 g0 @ 433 MHz
2023-11-05 21:11:11,274 INFO     MainThread Setting RFM2Pi group: 210 (210g)
2023-11-05 21:11:12,277 INFO     MainThread Setting RFM2Pi quiet: 1 (1q)
2023-11-05 21:11:13,279 INFO     MainThread Setting RFM2Pi calibration: 230V (1p)
2023-11-05 21:11:14,284 INFO     MainThread Creating EmonHubMqttInterfacer 'MQTT'
2023-11-05 21:11:14,289 INFO     MainThread Setting MQTT node_format_enable: 1
2023-11-05 21:11:14,290 INFO     MainThread Setting MQTT nodevar_format_enable: 1
2023-11-05 21:11:14,291 INFO     MainThread Setting MQTT nodevar_format_basetopic: emon/
2023-11-05 21:11:14,293 INFO     MainThread Creating EmonHubEmoncmsHTTPInterfacer 'emoncmsorg'
2023-11-05 21:11:14,297 INFO     MainThread Setting emoncmsorg apikey: set
2023-11-05 21:11:14,298 INFO     MainThread Setting emoncmsorg url: https://emoncms.org
2023-11-05 21:11:14,299 INFO     MainThread Setting emoncmsorg senddata: 1
2023-11-05 21:11:14,300 INFO     MainThread Setting emoncmsorg sendstatus: 1
2023-11-05 21:11:14,300 INFO     MainThread Setting emoncmsorg sendnames: 0
2023-11-05 21:11:14,301 INFO     MainThread Setting emoncmsorg compress: 0
2023-11-05 21:12:18,025 INFO     MQTT       Connecting to MQTT Server
2023-11-05 21:12:18,127 INFO     emoncmsorg sending: https://emoncms.org/input/bulk.json?sentat=1699218738 (104 bytes of data, 1 frames, uncompressed)
2023-11-05 21:12:18,132 WARNING  MQTT       Connection refused - not authorised
2023-11-05 21:12:18,133 INFO     MQTT       Unexpected disconnection
2023-11-05 21:12:28,091 INFO     MQTT       Connecting to MQTT Server
2023-11-05 21:12:28,162 INFO     emoncmsorg sending: https://emoncms.org/input/bulk.json?sentat=1699218748 (106 bytes of data, 1 frames, uncompressed)
2023-11-05 21:12:28,196 WARNING  MQTT       Connection refused - not authorised
2023-11-05 21:12:28,198 INFO     MQTT       Unexpected disconnection
4

Did you mean emonhub.conf? Failing that, zip it and upload the zipped file.

I’m afraid I can’t help with MQTT - I know nothing about it.
“WARNING MQTT Connection refused - not authorised” would imply to me a password is missing, or has been overwritten when you restored the backup.

If you have mixed radio protocols, that’s a big problem. You must consistently use either JeeLib ‘Classic’ (the original) OR JeeLib ‘Native’ OR LPL with each radio module in your installation. There is no interoperability between any of the three.

  • JeeLib ‘Classic’ and JeeLib ‘Native’ can be used with both the RFM12B and the RFM69CW radios.

  • LPL can only be used with RFM69 radios.

  • The emonPi can only do CM when using either JeeLib ‘Native’ or LPL.

5

Oddly even though I followed the instructions not to change the default username of ‘pi’ when I created the SD card image, there seems to be an issue with that user as well as the ‘emon’ username. Also I know the ‘emon’ username was active on 1st power up because I struggled to find the default password for it.

Emoncms Log

2023-11-05 20:46:12.057|ERROR|user_model.php|Login: Username does not exist username:pi ip:192.168.0.20
2023-11-05 20:49:13.123|ERROR|user_model.php|Login: Username does not exist username:emon ip:192.168.0.20

6

It’s the emonhub debug file, zipped attaches, but default txt doesn’t:
emonhub.zip (143.1 KB)

7

It looks as if you might have the wrong baud rate for the RFM2Pi / RFM69Pi

2023-11-05 21:03:15,089 ERROR MainThread Could not open serial port: /dev/ttyUSB0 @ 115200 bits/s (retry every 10s)
…
2023-11-05 21:03:15,223 ERROR MainThread Could not connect to RFM69 module

Try 115200 or 9600 for /dev/ttyAMA0 (just restart emonhub after editing in emonhub.conf)

8

@Robert.Wall I agree that it looks like users could have been overwritten, hopefully someone can advise on how to re-add them because it will only re-occur if I redo the SD card & import.

As for the radio protocols, originally I would have been running JeeLib ‘Classic’ because I have a emonTxV3, do you know how I can tell what is now set on the Pi’s RFM69Pi?
Looking at the emonhub log, it appears to be using LPL, please can you confirm either way. Also is it then just a case of updating the firmware via the GUI if this is the case?

I might switch to LPL later once I get this issue sorted.

I was replying and saw your additional response about the baud rate, therefore I had a look at the logs and noticed that error occurred twice and then no longer appeared. I have changed it from 38400 to 115200 but it has not fixed the issue even after a reboot (yes I initially tried a emonhub restart).

I won’t now get an opportunity to look at this again until tomorrow evening at the earliest.

9

I can’t confirm for certain, but I believe there’s a push to move to LPL, therefore if it appears to be LPL, it probably is LPL. There’s no way LPL will receive anything other than data from another LPL radio.
You should be able to ‘downgrade’ your Pi to using JeeLib Classic, but you won’t be able to have CM on the Pi, that will need to downgrade to the DS version.
It’s probably best long-term to stay with LPL on your Pi and change the emonTx V3 and anything else to LPL.

10

I’ve just pushed out JeeLib ‘Classic’ to the RFM69Pi and no change to inputs.
I did notice that in the update log there was the following:

Overriding Baud Rate : 38400

I left the system a few minutes to before reverting the baud back to 38400 in the emonhub.conf but it made difference either way.

11

Here’s the update log output if anyone’s interested:

-------------------------------------------------------------
emonBase_jeelib Firmware Upload
-------------------------------------------------------------
Downloading firmware from: 
https://github.com/openenergymonitor/RFM2Pi/releases/download/1.4.0/firmware.hex

Downloaded file: 
-rw-r--r-- 1 pi pi 22K Dec  7  2021 /opt/openenergymonitor/data/firmware/emonBase_jeelib.hex

EmonHub is running, stopping EmonHub

Uploading emonBase_jeelib on serial port ttyAMA0
Attempt 1...


avrdude-original: Version 6.3-20171130
                  Copyright (c) 2000-2005 Brian Dean, http://www.bdmicro.com/
                  Copyright (c) 2007-2014 Joerg Wunsch

                  System wide configuration file is "avrdude.conf"
                  User configuration file is "/root/.avrduderc"
                  User configuration file does not exist or is not a regular file, skipping

                  Using Port                    : /dev/ttyAMA0
                  Using Programmer              : arduino
                  Overriding Baud Rate          : 38400
                  AVR Part                      : ATmega328P
                  Chip Erase delay              : 9000 us
                  PAGEL                         : PD7
                  BS2                           : PC2
                  RESET disposition             : dedicated
                  RETRY pulse                   : SCK
                  serial program mode           : yes
                  parallel program mode         : yes
                  Timeout                       : 200
                  StabDelay                     : 100
                  CmdexeDelay                   : 25
                  SyncLoops                     : 32
                  ByteDelay                     : 0
                  PollIndex                     : 3
                  PollValue                     : 0x53
                  Memory Detail                 :

                                           Block Poll               Page                       Polled
                    Memory Type Mode Delay Size  Indx Paged  Size   Size #Pages MinW  MaxW   ReadBack
                    ----------- ---- ----- ----- ---- ------ ------ ---- ------ ----- ----- ---------
                    eeprom        65    20     4    0 no       1024    4      0  3600  3600 0xff 0xff
                    flash         65     6   128    0 yes     32768  128    256  4500  4500 0xff 0xff
                    lfuse          0     0     0    0 no          1    0      0  4500  4500 0x00 0x00
                    hfuse          0     0     0    0 no          1    0      0  4500  4500 0x00 0x00
                    efuse          0     0     0    0 no          1    0      0  4500  4500 0x00 0x00
                    lock           0     0     0    0 no          1    0      0  4500  4500 0x00 0x00
                    calibration    0     0     0    0 no          1    0      0     0     0 0x00 0x00
                    signature      0     0     0    0 no          3    0      0     0     0 0x00 0x00

                  Programmer Type : Arduino
                  Description     : Arduino
                  Hardware Version: 3
                  Firmware Version: 4.4
                  Vtarget         : 0.3 V
                  Varef           : 0.3 V
                  Oscillator      : 28.800 kHz
                  SCK period      : 3.3 us

avrdude-original: AVR device initialized and ready to accept instructions

Reading | ################################################## | 100% 0.00s

avrdude-original: Device signature = 0x1e950f (probably m328p)
avrdude-original: reading input file "/opt/openenergymonitor/data/firmware/emonBase_jeelib.hex"
avrdude-original: writing flash (7814 bytes):

Writing | ################################################## | 100% 2.77s

avrdude-original: 7814 bytes of flash written
avrdude-original: verifying flash memory against /opt/openenergymonitor/data/firmware/emonBase_jeelib.hex:
avrdude-original: load data flash data from input file /opt/openenergymonitor/data/firmware/emonBase_jeelib.hex:
avrdude-original: input file /opt/openenergymonitor/data/firmware/emonBase_jeelib.hex contains 7814 bytes
avrdude-original: reading on-chip flash data:

Reading | ################################################## | 100% 2.56s

avrdude-original: verifying ...
avrdude-original: 7814 bytes of flash verified

avrdude-original done.  Thank you.

avrdude-original: Using autoreset DTR on GPIO Pin 7
avrdude-original: Using autoreset DTR on GPIO Pin 7
avrdude-original: Using autoreset DTR on GPIO Pin 7
SUCCESS: flash verifed

Restarting EmonHub
12

Have you got your old NodeIDs and specifications for your emonTx(s) / emonTH(s) in your emonhub.conf, or new ones?

What does emonhub log tell you?

13

I’ve thought for a while that the backup and restore doesn’t use the old emonhub configuration file.

Solve this issue.

Not sure why it might not be correct. Did you change the MQTT credentials on the old system for some reason?

What is the MQTT section of the emonhub.conf file?

14

Have you got your old NodeIDs and specifications for your emonTx(s) / emonTH(s) in your emonhub.conf, or new ones?
What does emonhub log tell you?

The nodes are correct as per my previous setup because I did an archive import.

15

Issue resolved by replacing the MQTT password with the default (emonpimqtt2016).

I hadn’t changed it previously and it looks like the ‘Archive Export’ encrypts the MQTT credentials (as it looks to be in that format), and then on import it is not decrypted.

Also after an Emonhub restart, the inputs were not reliably being read, a full reboot seems to have resolved this additional issue.

I have raised an issue in GitHub: