Update failing - confused about the "safe-update" list

So the command line curl (on an SSH session) works fine

pi@emonpi:~ $ curl -s https://raw.githubusercontent.com/openenergymonitor/EmonScripts/master/safe-update
emonSD-02Oct19
emonSD-17Oct19
pi@emonpi:~ 

The first df -h

pi@emonpi:~ $ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/root       4.0G  2.1G  1.7G  55% /
devtmpfs        484M     0  484M   0% /dev
tmpfs           488M     0  488M   0% /dev/shm
tmpfs           488M   19M  470M   4% /run
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           488M     0  488M   0% /sys/fs/cgroup
tmpfs           1.0M     0  1.0M   0% /var/lib/php/sessions
tmpfs            30M   30M     0 100% /tmp
tmpfs           1.0M     0  1.0M   0% /var/tmp
/dev/mmcblk0p3   10G   41M  9.5G   1% /var/opt/emoncms
/dev/mmcblk0p1  253M   52M  201M  21% /boot
log2ram          50M   37M   14M  74% /var/log
tmpfs            98M     0   98M   0% /run/user/1000
pi@emonpi:~ $

Reboot [see below], and second df -h

pi@emonpi:~ $ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/root       4.0G  2.1G  1.7G  55% /
devtmpfs        484M     0  484M   0% /dev
tmpfs           488M     0  488M   0% /dev/shm
tmpfs           488M  6.5M  482M   2% /run
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           488M     0  488M   0% /sys/fs/cgroup
tmpfs           1.0M     0  1.0M   0% /var/lib/php/sessions
tmpfs            30M     0   30M   0% /tmp
tmpfs           1.0M     0  1.0M   0% /var/tmp
/dev/mmcblk0p3   10G   41M  9.5G   1% /var/opt/emoncms
/dev/mmcblk0p1  253M   52M  201M  21% /boot
log2ram          50M   38M   13M  75% /var/log
tmpfs            98M     0   98M   0% /run/user/1000
pi@emonpi:~ $

Now about this rebooting thing.

Maybe related to my other issues, could be nothing but I am find rebooting troublesome. Today when trying the reboot sandwich in between a pair of df -h I struggled. I first asked the emonPi to reboot from the button at the bottom of Setup / Admin from a browser login to it’s IP address.

Certainly yes the separate PuTTY session in to port 22 died immediately, and after 3 minutes (measured by a timer on my phone) I tried PuTTY again - timed out. Went to the emonPi (inconveniently located but aren’t all utility meters), no LCD backlight and no response to the button.

Removed 5V power, waited around 2 minutes minutes, repowered. Went through splash screen, sensor detection, and gets to Raspberry Pi booting … message. After another 5 minutes no progress on LCD, backlight and booting message still there (that message needs to be animated so you can tell when it sticks). Removed 5V power and AC voltage sensor power, left for 15 minutes.

Reapplied AC voltage then 5V power. Went through normal boot sequence to WiFi Yes xx% & DHCP’d IP address. Happy, or so I think.

I extract the second df -h and write it here. Then try browsing to the usual web page at the Pi’s IP. Very strange result - rather than a login landing page I get

Fatal error : Uncaught Error: Call to a member function fetch_array() on bool in /var/www/emoncms/core.php:43 Stack trace: #0 /var/www/emoncms/index.php(82): db_check(Object(mysqli), ‘emoncms’) #1 {main} thrown in /var/www/emoncms/core.php on line 43

Tried on two different browsers (edge & chrome) and to the bare IP or at IP:80. All the same result.

So is my SDcard gradually disintegrating? Do I have a hardware issue? What is the issue with not powering down gracefully?

ConfusionLevel++

Apologies, now I look I see that the /tmp mount was full prior to the reboot, hence maybe the original second failure to update.

But I’m definitely in a worse place now. How do I get my Pi back?

This might be related /tmp being full. One alternative is from the command line sudo reboot now.

Should be OK.

SSH in and reboot from the command line. May take a few (wait 10) minutes before you can reconnect to the Pi via Putty (tip - right click on banner and select restart session). Hard reboot when there are updates going on can be a very bad thing! Patience is a necessity.

1 Like

The /tmp file being full is interesting, it may be caused by the apache2 sessions folder filling up, do you have any other OEM hardware running EmonESP by any chance?

@borpin Now completely borked. I did sudo reboot now from PuTTY and that was the last my network has seen of the Pi. LCD got as far as Raspberry Pi rebooting… Where it has sat for some hours.

About to download the image and burn a new card.

@TrystanLea I was running the vanilla image as supplied from the shop, with the only addition being an install of node-red last weekend. That’s obviously the prime suspect, but I think I was having issues before anyways.

Today has been a bad day for technology in my day job as well.

Happiness–

That sounds like the best course of action at this point. Sorry that this has failed.
If you want to try and recover any data on the image you could try Update & Upgrade — OpenEnergyMonitor 0.0.1 documentation you may need to use fsck to recover the card see Import / Backup / Restore / Update — OpenEnergyMonitor 0.0.1 documentation

Ah, well, that could easily explain it. I’d run NR on a separate Pi.

Today is a different day.

Downloaded image and flashed to virgin 16GB SDcard using etcher. Installed in emonPi, powered up and walked away for a few hours. Did initial set up on WiFi - will come back to wired Ethernet issue later. Ran Full Update, but I think it had already done so, as nothing was pulled in.

Enabled SSH from button by LCD. Put old SDcard into USB reader and plugged into emonPi. Tried Setup | Backup | Import USB - this was the output.

=== USB Emoncms import start ===
2020-04-29-12:59:27
Backup module version:
    "version"      : "2.2.2"
EUID: 1000
Reading /opt/emoncms/modules/backup/config.cfg....
Location of data databases: /var/opt/emoncms
Location of emonhub.conf: /etc/emonhub
Location of Emoncms: /var/www/emoncms

USB drive not found

Assumed SDcard corrupted. From SSH session ran

pi@emonpi:~ $ sudo fsck.ext4 /dev/sda2
e2fsck 1.44.5 (15-Dec-2018)
rootfs contains a file system with errors, check forced.
Pass 1: Checking inodes, blocks, and sizes
Inode 98182 is in use, but has dtime set.  Fix<y>? yes
Inodes that were part of a corrupted orphan linked list found.  Fix<y>? yes
Inode 98184 was part of the orphaned inode list.  FIXED.
Inode 98182 has a bad extended attribute block 32768.  Clear<y>? yes
Inode 98182, i_size is 576460752303423488, should be 0.  Fix<y>? yes
Pass 2: Checking directory structure
Directory inode 42052, block #0, offset 520: directory corrupted
Salvage<y>? yes
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Unattached zero-length inode 8197.  Clear<y>? yes
Inode 98182 (...) has invalid mode (01000).
Clear<y>? yes
Inode 98184 (...) has invalid mode (00).
Clear<y>? yes
Pass 5: Checking group summary information
Block bitmap differences:  -(155136--155378)
Fix<y>? yes
Free inodes count wrong for group #12 (8050, counted=8048).
Fix ('a' enables 'yes' to all) <y>? yes
Free inodes count wrong (190683, counted=190681).
Fix ('a' enables 'yes' to all) <y>? yes

rootfs: ***** FILE SYSTEM WAS MODIFIED *****
rootfs: 74903/265584 files (0.1% non-contiguous), 559467/1053440 blocks

And then

pi@emonpi:~ $ sudo fsck.ext2 /dev/sda3
e2fsck 1.44.5 (15-Dec-2018)
/dev/sda3: clean, 40/665088 files, 210847/10634240 blocks
pi@emonpi:~ $ 

Tried Import USB again, same message. Unplugged USB card reader, plugged into different USB port and tried again - still nothing found.

No linux guru so not understanding that Import USB can’t see the card in the reader, but fsck has seen and fixed some errors. Were the errors that fsck fixed found on the SDcard, or some other storage on the Pi motherboard? Are /dev/sda2 and /dev/sda3 on the SDcard or somewhere else?

What is my next step to attempt to restore my feeds and data? Painful and disappointing if I can’t but not disastrous.

Now to the Wired Ethernet. Before Corruption (BC) I had tried and failed to move the emonPi onto my wired network. I used a Powerline plug pair to present 10/100Mb LAN at the utility meter. A normal laptop happily DHCP’d there to my 10.5.1.0/24 home network and connected to all network resources. The emonPi did not, it came up on the LCD as 169.254.204.239, and still does now After Corruption (AC).

This makes me think that either (a) there is be some other stored state in the emonPi - like an MMC, Flash or EEPROM chip that is corrupted, or (b) there’s a fault in my emonPi’s Ethernet port.

Can anyone comment.

PS Port lights on the emonPi and Powerline plug showed Link Up and Active.

PPS I had also directly linked a laptop to the emonPi using a straight wired RJ45 - good link LEDs at both ends, and the laptop assigned itself a compatible 169.254.0.0/16 address. I could not web browse to the emonPi’s IP address, with or without a :80 suffix, or SSH to its port 22.

I see 2 different IP addresses (when I interrogate the network) - one for wired (eth0) one for Wi-Fi (wlan0).

try ip a for the IP addresses.

The 2 commands have been successful when run after a hard power down due to a power cut.

Sometimes the card is just too badly corrupted.

Sorry about this @Simonsse I think this is an issue with the import script, I’ve been trying to scan for likely USB card reader names and dont seem to be able to catch all the variations, see the lines here backup/usb-import.sh at master · emoncms/backup · GitHub

Could you run:

find /dev/disk/by-id/

Really need to list the drives found (if not auto found) and allow that as a parameter to the script.

Good idea I will add that in now

Your post indicates you checked /dev/sda2 which indicates an external device. i.e. your
card and adapter vice the “internal” storage.

Did you reboot after the filesystem error were fixed but before you tried importing again?

169.254.x.x: This is what’s called an Automatic Private IP address. An IP in this range means that the computer cannot see the network. A computer using DHCP needs to have an external server tell it what IP address to use. Unfortunately, if there’s no network connectivity, the computer is unable to talk to the server. In those cases, the computer will actually give itself an IP starting with 169.254, since it must assign itself some sort of number. When you see a 169.254.x.x address, you definitely have a problem. It could be as simple as an unplugged network cable, or it could be as complex as the network being down. A fair amount of troubleshooting is involved at this point, but the bottom line is that your computer doesn’t even see the network.

Ref: What are some IP addresses that might indicate I have a network problem?

A RasPi has no EEPROM, so no danger of that being corrupted.
Have you tried a direct Ethernet connection without your powerline adapters?
We’ve got a pair of them at my workplace, and we’ve observed that while normally fairly reliable,
they can be picky about the hardware connected to them as well as the particular power circuit
they’re used on.

The script now prints out the usb drives that its finds, I’ve updated both the stable and master branches with this change. @Simonsse could you try updating your emonpi and trying the import again?

This is the first part of the result I see:

=== USB Emoncms import start ===
2020-04-29-14:54:33
Backup module version:
    "version"      : "2.2.2"
EUID: 1000
Reading /opt/emoncms/modules/backup/config.cfg....
Location of data databases: /var/opt/emoncms
Location of emonhub.conf: /etc/emonhub
Location of Emoncms: /var/www/emoncms

Scanning for USB card reader:
Found: /dev/disk/by-id/usb-Generic-_SD_MMC_20090815198100000-0:0 at /dev/sda
Using: /dev/disk/by-id/usb-Generic-_SD_MMC_20090815198100000-0:0 at /dev/sda
- No card reader found on sdb
- No card reader found on sdc

Trying the find command

pi@emonpi:~ $ find /dev/disk/by-id/
/dev/disk/by-id/
/dev/disk/by-id/usb-Generic_MassStorageClass_000000001536-0:0-part2
/dev/disk/by-id/usb-Generic_MassStorageClass_000000001536-0:0-part3
/dev/disk/by-id/usb-Generic_MassStorageClass_000000001536-0:0-part1
/dev/disk/by-id/usb-Generic_MassStorageClass_000000001536-0:0
/dev/disk/by-id/mmc-SC16G_0xb4aaceb9-part3
/dev/disk/by-id/mmc-SC16G_0xb4aaceb9-part2
/dev/disk/by-id/mmc-SC16G_0xb4aaceb9-part1
/dev/disk/by-id/mmc-SC16G_0xb4aaceb9
pi@emonpi:~ $

Did an emonPi update and got your new import script. Update log is

Starting update via service-runner-update.sh (v3.0) >
- emonSD version: emonSD-17Oct19
emonSD base image check passed...continue update
git pull /opt/openenergymonitor/EmonScripts
  master
* stable
On branch stable
Your branch is up to date with 'origin/stable'.

nothing to commit, working tree clean
Already up to date.
-------------------------------------------------------------
Main Update Script
-------------------------------------------------------------
Date: Wed 29 Apr 15:19:16 BST 2020
EUID: 1000
openenergymonitor_dir: /opt/openenergymonitor
type: all
firmware: none
Hardware detected: EmonPi
Stopping emonPiLCD service
Display update message on LCD
I2C LCD DETECTED Ox27
git pull /opt/openenergymonitor/emonpi
* master
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean
Fetching origin
Already up to date.
git pull /opt/openenergymonitor/RFM2Pi
* master
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean
Fetching origin
Already up to date.
Start emonhub update script:
-------------------------------------------------------------
emonHub update
-------------------------------------------------------------
git pull /opt/openenergymonitor/emonhub
* emon-pi
On branch emon-pi
Your branch is up to date with 'origin/emon-pi'.

nothing to commit, working tree clean
Already up to date.
Failed to restart .service.service: Unit .service.service not found.
- Service 
Running emonhub automatic node addition script
[[5]]
Node 5 already present
[[6]]
Node 6 already present
[[7]]
Node 7 already present
[[8]]
Node 8 already present
[[9]]
Node 9 already present
[[10]]
Node 10 already present
[[11]]
Node 11 already present
[[12]]
Node 12 already present
[[13]]
Node 13 already present
[[14]]
Node 14 already present
[[15]]
Node 15 already present
[[16]]
Node 16 already present
[[19]]
Node 19 already present
[[20]]
Node 20 already present
[[21]]
Node 21 already present
[[22]]
Node 22 already present
[[23]]
Node 23 already present
[[24]]
Node 24 already present
[[25]]
Node 25 already present
[[26]]
Node 26 already present

Start emoncms update:
-------------------------------------------------------------
Update Emoncms Core
-------------------------------------------------------------


Checking status of /var/www/emoncms git repository
- git branch: stable
- no local changes
- running: git pull origin stable

Fetching origin
From https://github.com/emoncms/emoncms
 * branch              stable     -> FETCH_HEAD
Already up to date.

Update Emoncms database
[]

-------------------------------------------------------------
Update Emoncms Services
-------------------------------------------------------------
emoncms_mqtt.service already installed
feedwriter.service already installed
service-runner.service already installed

Reloading systemctl deamon
Restarting Services...
- sudo systemctl restart feedwriter.service
--- ActiveState=active ---
- sudo systemctl restart emoncms_mqtt.service
--- ActiveState=active ---
- sudo systemctl restart emonhub.service
--- ActiveState=active ---
/opt/openenergymonitor/EmonScripts/sudoers.d/emoncms-rebootbutton: parsed OK
emonPi emoncms admin reboot button sudoers updated

-------------------------------------------------------------
Update Emoncms Modules
-------------------------------------------------------------
------------------------------------------
Updating /var/www/emoncms/Modules/app module
------------------------------------------
- git branch: stable
- git tags: 2.1.1
- no local changes
- running: git pull origin stable

Fetching origin
Already on 'stable'
Your branch is up to date with 'origin/stable'.
Already up to date.

------------------------------------------
Updating /var/www/emoncms/Modules/config module
------------------------------------------
- git branch: stable
- git tags: 2.0.5
- no local changes
- running: git pull origin stable

Fetching origin
Already on 'stable'
Your branch is up to date with 'origin/stable'.
Already up to date.

------------------------------------------
Updating /var/www/emoncms/Modules/dashboard module
------------------------------------------
- git branch: stable
- git tags: 2.0.7
- no local changes
- running: git pull origin stable

Fetching origin
Already on 'stable'
Your branch is up to date with 'origin/stable'.
Already up to date.

------------------------------------------
Updating /var/www/emoncms/Modules/device module
------------------------------------------
- git branch: stable
- git tags: 2.0.5
- no local changes
- running: git pull origin stable

Fetching origin
Already on 'stable'
Your branch is up to date with 'origin/stable'.
Already up to date.

------------------------------------------
Updating /var/www/emoncms/Modules/graph module
------------------------------------------
- git branch: stable
- git tags: 2.0.8-2-g7df046d
- no local changes
- running: git pull origin stable

Fetching origin
Already on 'stable'
Your branch is up to date with 'origin/stable'.
Already up to date.

------------------------------------------
Updating /var/www/emoncms/Modules/setup module
------------------------------------------
- git branch: stable
fatal: No names found, cannot describe anything.
- git tags: 
- no local changes
- running: git pull origin stable

Fetching origin
Already on 'stable'
Your branch is up to date with 'origin/stable'.
Already up to date.

------------------------------------------
Updating /var/www/emoncms/Modules/wifi module
------------------------------------------
- git branch: stable
- git tags: 2.0.0-29-g274bcd5
- no local changes
- running: git pull origin stable

Fetching origin
Already on 'stable'
Your branch is up to date with 'origin/stable'.
Already up to date.

/opt/openenergymonitor/EmonScripts/sudoers.d/wifi-sudoers: parsed OK
wifi sudoers entry updated
------------------------------------------
Updating /opt/emoncms/modules/backup module
------------------------------------------
- git branch: stable
- git tags: 2.2.2
- no local changes
- running: git pull origin stable

From https://github.com/emoncms/backup
   8b670a0..70894e1  stable     -> origin/stable
   dfaca67..3f2beaa  master     -> origin/master
Updating 8b670a0..70894e1
Fast-forward
 .travis.yml               |  14 ++
 LICENSE.txt               | 619 ++++++++++++++++++++++++++++++++++++++++++++++
 backup-module/module.json |   2 +-
 composer.json             |  17 ++
 usb-import.sh             |  15 +-
 5 files changed, 658 insertions(+), 9 deletions(-)
 create mode 100644 .travis.yml
 create mode 100644 LICENSE.txt
 create mode 100644 composer.json
Already on 'stable'
Your branch is up to date with 'origin/stable'.

------------------------------------------
Updating /opt/emoncms/modules/demandshaper module
------------------------------------------
- git branch: stable
- git tags: 1.2.4-5-gd689aae
- no local changes
- running: git pull origin stable

Already up to date.
Already on 'stable'
Your branch is up to date with 'origin/stable'.

------------------------------------------
Updating /opt/emoncms/modules/postprocess module
------------------------------------------
- git branch: stable
- git tags: 2.1.0-21-g0e2d78f
- no local changes
- running: git pull origin stable

Already up to date.
Already on 'stable'
Your branch is up to date with 'origin/stable'.

------------------------------------------
Updating /opt/emoncms/modules/sync module
------------------------------------------
- git branch: stable
- git tags: 2.0.5
- no local changes
- running: git pull origin stable

Already up to date.
Already on 'stable'
Your branch is up to date with 'origin/stable'.

------------------------------------------
Updating /opt/emoncms/modules/usefulscripts module
------------------------------------------
- git branch: stable
fatal: No names found, cannot describe anything.
- git tags: 
- no local changes
- running: git pull origin stable

Already up to date.
Already on 'stable'
Your branch is up to date with 'origin/stable'.

--------------------------------------------
Backup module installation and update script
--------------------------------------------
- Loading EmonScripts config.ini
- Copying default.config.cfg to config.cfg
- Setting config.cfg settings
- Backup module symlink already exists
- PHP Version: 7.3
- Creating /etc/php/7.3/mods-available/emoncmsbackup.ini
post_max_size = 3G
upload_max_filesize = 3G
upload_tmp_dir = /var/opt/emoncms/backup/uploads
- phpenmod emoncmsbackup
- /var/opt/emoncms/backup already exists
- /var/opt/emoncms/backup/uploads already exists
- restarting apache

Update Emoncms database
[]


Starting emonPi LCD service..


-------------------------------------------------------------
emonPi update done: Wed 29 Apr 15:20:03 BST 2020
-------------------------------------------------------------
restarting service-runner

Ran it and it did lots of work

=== USB Emoncms import start ===
2020-04-29-15:20:13
Backup module version:
    "version"      : "2.2.3"
EUID: 1000
Reading /opt/emoncms/modules/backup/config.cfg....
Location of data databases: /var/opt/emoncms
Location of emonhub.conf: /etc/emonhub
Location of Emoncms: /var/www/emoncms

Scanning for USB card reader:
- Found: /dev/disk/by-id/usb-Generic_MassStorageClass_000000001536-0:0 at /dev/sda
**Using: /dev/disk/by-id/usb-Generic_MassStorageClass_000000001536-0:0 at /dev/sda**
- No card reader found on sdb
- No card reader found on sdc
creating mount point /media/old_sd_boot
creating mount point /media/old_sd_root
creating mount point /media/old_sd_data
Mounting old SD card boot partition
Mounting old SD card root partition
Mounting old SD card data partition

Stopping services..
Read MYSQL authentication details from settings.php
stopping mysql
Manually deleting old mysql emoncms database
Manual install of emoncms database
'/media/old_sd_root/var/lib/mysql/emoncms' -> '/var/lib/mysql/emoncms'
'/media/old_sd_root/var/lib/mysql/emoncms/db.opt' -> '/var/lib/mysql/emoncms/db.opt'
'/media/old_sd_root/var/lib/mysql/emoncms/app_config.frm' -> '/var/lib/mysql/emoncms/app_config.frm'
'/media/old_sd_root/var/lib/mysql/emoncms/app_config.MYI' -> '/var/lib/mysql/emoncms/app_config.MYI'
'/media/old_sd_root/var/lib/mysql/emoncms/app_config.MYD' -> '/var/lib/mysql/emoncms/app_config.MYD'
'/media/old_sd_root/var/lib/mysql/emoncms/demandshaper.frm' -> '/var/lib/mysql/emoncms/demandshaper.frm'
'/media/old_sd_root/var/lib/mysql/emoncms/demandshaper.MYI' -> '/var/lib/mysql/emoncms/demandshaper.MYI'
'/media/old_sd_root/var/lib/mysql/emoncms/demandshaper.MYD' -> '/var/lib/mysql/emoncms/demandshaper.MYD'
'/media/old_sd_root/var/lib/mysql/emoncms/device.frm' -> '/var/lib/mysql/emoncms/device.frm'
'/media/old_sd_root/var/lib/mysql/emoncms/device.MYI' -> '/var/lib/mysql/emoncms/device.MYI'
'/media/old_sd_root/var/lib/mysql/emoncms/device.MYD' -> '/var/lib/mysql/emoncms/device.MYD'
'/media/old_sd_root/var/lib/mysql/emoncms/feeds.frm' -> '/var/lib/mysql/emoncms/feeds.frm'
'/media/old_sd_root/var/lib/mysql/emoncms/feeds.MYI' -> '/var/lib/mysql/emoncms/feeds.MYI'
'/media/old_sd_root/var/lib/mysql/emoncms/feeds.MYD' -> '/var/lib/mysql/emoncms/feeds.MYD'
'/media/old_sd_root/var/lib/mysql/emoncms/graph.frm' -> '/var/lib/mysql/emoncms/graph.frm'
'/media/old_sd_root/var/lib/mysql/emoncms/graph.MYI' -> '/var/lib/mysql/emoncms/graph.MYI'
'/media/old_sd_root/var/lib/mysql/emoncms/graph.MYD' -> '/var/lib/mysql/emoncms/graph.MYD'
'/media/old_sd_root/var/lib/mysql/emoncms/input.frm' -> '/var/lib/mysql/emoncms/input.frm'
'/media/old_sd_root/var/lib/mysql/emoncms/input.MYI' -> '/var/lib/mysql/emoncms/input.MYI'
'/media/old_sd_root/var/lib/mysql/emoncms/input.MYD' -> '/var/lib/mysql/emoncms/input.MYD'
'/media/old_sd_root/var/lib/mysql/emoncms/postprocess.frm' -> '/var/lib/mysql/emoncms/postprocess.frm'
'/media/old_sd_root/var/lib/mysql/emoncms/postprocess.MYI' -> '/var/lib/mysql/emoncms/postprocess.MYI'
'/media/old_sd_root/var/lib/mysql/emoncms/postprocess.MYD' -> '/var/lib/mysql/emoncms/postprocess.MYD'
'/media/old_sd_root/var/lib/mysql/emoncms/schedule.frm' -> '/var/lib/mysql/emoncms/schedule.frm'
'/media/old_sd_root/var/lib/mysql/emoncms/schedule.MYI' -> '/var/lib/mysql/emoncms/schedule.MYI'
'/media/old_sd_root/var/lib/mysql/emoncms/schedule.MYD' -> '/var/lib/mysql/emoncms/schedule.MYD'
'/media/old_sd_root/var/lib/mysql/emoncms/setup.frm' -> '/var/lib/mysql/emoncms/setup.frm'
'/media/old_sd_root/var/lib/mysql/emoncms/setup.MYI' -> '/var/lib/mysql/emoncms/setup.MYI'
'/media/old_sd_root/var/lib/mysql/emoncms/setup.MYD' -> '/var/lib/mysql/emoncms/setup.MYD'
'/media/old_sd_root/var/lib/mysql/emoncms/sync.frm' -> '/var/lib/mysql/emoncms/sync.frm'
'/media/old_sd_root/var/lib/mysql/emoncms/sync.MYI' -> '/var/lib/mysql/emoncms/sync.MYI'
'/media/old_sd_root/var/lib/mysql/emoncms/sync.MYD' -> '/var/lib/mysql/emoncms/sync.MYD'
'/media/old_sd_root/var/lib/mysql/emoncms/users.frm' -> '/var/lib/mysql/emoncms/users.frm'
'/media/old_sd_root/var/lib/mysql/emoncms/users.MYI' -> '/var/lib/mysql/emoncms/users.MYI'
'/media/old_sd_root/var/lib/mysql/emoncms/users.MYD' -> '/var/lib/mysql/emoncms/users.MYD'
'/media/old_sd_root/var/lib/mysql/emoncms/rememberme.frm' -> '/var/lib/mysql/emoncms/rememberme.frm'
'/media/old_sd_root/var/lib/mysql/emoncms/rememberme.MYI' -> '/var/lib/mysql/emoncms/rememberme.MYI'
'/media/old_sd_root/var/lib/mysql/emoncms/rememberme.MYD' -> '/var/lib/mysql/emoncms/rememberme.MYD'
'/media/old_sd_root/var/lib/mysql/emoncms/multigraph.frm' -> '/var/lib/mysql/emoncms/multigraph.frm'
'/media/old_sd_root/var/lib/mysql/emoncms/multigraph.MYI' -> '/var/lib/mysql/emoncms/multigraph.MYI'
'/media/old_sd_root/var/lib/mysql/emoncms/multigraph.MYD' -> '/var/lib/mysql/emoncms/multigraph.MYD'
'/media/old_sd_root/var/lib/mysql/emoncms/dashboard.frm' -> '/var/lib/mysql/emoncms/dashboard.frm'
'/media/old_sd_root/var/lib/mysql/emoncms/dashboard.MYI' -> '/var/lib/mysql/emoncms/dashboard.MYI'
'/media/old_sd_root/var/lib/mysql/emoncms/dashboard.MYD' -> '/var/lib/mysql/emoncms/dashboard.MYD'
Setting database ownership
starting mysql
checking database
emoncms.app_config                                 OK
emoncms.dashboard                                  OK
emoncms.demandshaper                               OK
emoncms.device                                     OK
emoncms.feeds                                      OK
emoncms.graph                                      OK
emoncms.input                                      OK
emoncms.multigraph                                 OK
emoncms.postprocess                                OK
emoncms.rememberme                                 OK
emoncms.schedule                                   OK
emoncms.setup                                      OK
emoncms.sync                                       OK
emoncms.users                                      OK
Updating Emoncms Database..
[]
Archive old data folders
Copying PHPFina feed data
'/media/old_sd_data/phpfina' -> '/var/opt/emoncms/phpfina'
'/media/old_sd_data/phpfina/1.meta' -> '/var/opt/emoncms/phpfina/1.meta'
'/media/old_sd_data/phpfina/1.dat' -> '/var/opt/emoncms/phpfina/1.dat'
'/media/old_sd_data/phpfina/4.meta' -> '/var/opt/emoncms/phpfina/4.meta'
'/media/old_sd_data/phpfina/4.dat' -> '/var/opt/emoncms/phpfina/4.dat'
'/media/old_sd_data/phpfina/3.meta' -> '/var/opt/emoncms/phpfina/3.meta'
'/media/old_sd_data/phpfina/3.dat' -> '/var/opt/emoncms/phpfina/3.dat'
'/media/old_sd_data/phpfina/5.meta' -> '/var/opt/emoncms/phpfina/5.meta'
'/media/old_sd_data/phpfina/5.dat' -> '/var/opt/emoncms/phpfina/5.dat'
'/media/old_sd_data/phpfina/6.meta' -> '/var/opt/emoncms/phpfina/6.meta'
'/media/old_sd_data/phpfina/6.dat' -> '/var/opt/emoncms/phpfina/6.dat'
'/media/old_sd_data/phpfina/7.meta' -> '/var/opt/emoncms/phpfina/7.meta'
'/media/old_sd_data/phpfina/7.dat' -> '/var/opt/emoncms/phpfina/7.dat'
'/media/old_sd_data/phpfina/8.meta' -> '/var/opt/emoncms/phpfina/8.meta'
'/media/old_sd_data/phpfina/8.dat' -> '/var/opt/emoncms/phpfina/8.dat'
'/media/old_sd_data/phpfina/9.meta' -> '/var/opt/emoncms/phpfina/9.meta'
'/media/old_sd_data/phpfina/9.dat' -> '/var/opt/emoncms/phpfina/9.dat'
'/media/old_sd_data/phpfina/14.meta' -> '/var/opt/emoncms/phpfina/14.meta'
'/media/old_sd_data/phpfina/15.meta' -> '/var/opt/emoncms/phpfina/15.meta'
'/media/old_sd_data/phpfina/14.dat' -> '/var/opt/emoncms/phpfina/14.dat'
'/media/old_sd_data/phpfina/15.dat' -> '/var/opt/emoncms/phpfina/15.dat'
'/media/old_sd_data/phpfina/16.meta' -> '/var/opt/emoncms/phpfina/16.meta'
'/media/old_sd_data/phpfina/16.dat' -> '/var/opt/emoncms/phpfina/16.dat'
Copying PHPTimeSeries feed data
'/media/old_sd_data/phptimeseries' -> '/var/opt/emoncms/phptimeseries'
'/media/old_sd_root/etc/emonhub/emonhub.conf' -> '/etc/emonhub/emonhub.conf'
Flushing redis
OK
Restarting emonhub...
Restarting feedwriter...
Restarting emoncms MQTT...
2020-04-29-15:21:03
=== Emoncms import complete! ===

So I’ve rebooted. Deep breath, refresh the browser on the emonPi and …
I’m back! All previously recorded data restored.

Thank you Trystan, much appreciate the instant responses today, and glad to have been able to provide material to improve the product.


Apologies for the off topic comments below, I’ll happily repost appropriately if you recommend so.

  1. There is a ca. 2 day data gap which the My Solar App shows as constant use rather than My Electric showing zero, which IMHO would be better.

  2. I suspect that the old SDcard will be absolutely fine for reuse - or for the few pounds cost should I just ditch it as suspect and tainted?

  3. I will repost on the wired ethernet in another forum section - clearly nothing to do with emonSD.

@borpin I also have eth0 and wlan0 IPs. And had a play with dhcpcd.conf to try and assign a valid local wired network IP, to no avail. Efforts to sort were pushed to the back of the queue when all started falling apart.

@Bill.Thomson I will try without the powerline adapters - I too have found them flaky. Had a big win when I realised the out of the box setup with default credentials was locking on to somebody else’s adapter on the same power phase (a neighbour somewhere). Changed the authorisation and its been much more stable for many months.

1 Like

@borpin So what was the history here of emonCMS earlier with, and now without node-red? What happened to take it out of the default SDcard build for emonPi, and why would you not run NR on the same box as emonPi. Also why do you say elsewhere that Home Assistant beats EmonCMS in some corners of the automation market? Would you also suggest HA should have its own Pi?

My interest in NR is first to record and display number of my SolaX X1-Hybrid PV inverter and battery storage system at finer granularity than I can get from the SolaX Cloud (ca. 5 min update periods). I see traffic on this forum about SolaX connectivity through the X1-Hybrid’s WiFi port which can report all the inverter stats, and there is a NR flow to parse these into MQTT to feed into emonCMS, where I plan to record data and build some dashboard(s).

In the fullness of time I would also like to add in charging control - ideally enabling the export of battery power to the grid, this preferably being in late afternoon as I am on Octopus Agile Outgoing, where export rates up to 30p/kWh are on offer in the 1600-1900 afternoon slots.

I am also interested in automating the house’s heating, taking in to account near future expectations of which rooms will be in use, external temperatures, costs of energy (in CO2e and £), etc, etc.

A pointer to a emonPi, EmonCMS, Home Assistant, Node-Red, … comparison would be helpful.

Many thanks

I would second Brian’s suggestion to run Node-Red on a second Pi. I didn’t used to and it caused a lot of grief when things went wrong (like a failed SD card… sound familiar?)
Since using a second Pi for mosquitto and NR any problems have been much easier to deal with.

An early release of the emonCMS image included Node-RED. Node-RED is a little notorious when trying to keep it up to date (though it is better now) and having somethign else on the emonCMS system can cause issues as the layout of the partitions etc is optimised to prolong the life of the SD Card. Adding things to that, can have unexpected consequences ( as you may or may not have discovered).

HA has a mass of automation options and you can build very complex relationships between things. emonCMS will never be able to compete on that front. However, emonCMS does have some specialised automation systems (like demandshaper). HA does have a fairly steep learning curve! I use HA to control my heating and more recently my hot water (4 separate temp sensors rather than a single ‘thermostat’).

Personally, I run a supervised install of HA on a Pi4-4Gb. There is an add-on to run Node-RED inside HA effectively and I use that for my main Node-RED instance. If you do go down this road, I suggest a HDD or SDD and probably the prebuilt image. In the future I will look to move HA onto an NUC as I feel that will be more reliable and robust.