Manually editing "rogue" data + upgrading from 10.0.1

I give up. Back to the old card.

Flatten and re-do the whole card? Or is there some better way to reset it?

No idea.

I utterly give up.

Right, back where I started. Card rewritten, default config done. Just doing an update.

Let’s try again, lol.

I’m not really sure where you got to with this.

There is a password reset file. (yes needs adding to docs). This should work.

php /opt/emoncms/modules/usefulscripts/resetpassword.php

On import, odd it is showing up as sdd - yes the script I think assumes a,b,c

File is at

/opt/emoncms/modules/backup/usb-import.sh

line 49 add sdd to the list

Once imported you need to revert the change in git otherwise it will not pick up any updates later on.

(once imported we can do that)

1 Like

Be patient on first boot.

1 Like

Ok, no, still no joy. Maybe it is just an unsupported reader?

dmesg output shows it’s now (with the adaptor) on sdb not sdd … so the adaptor “worked” :

[   71.775353] w1_master_driver w1_bus_master1: Attaching one wire slave 00.800000000000 crc 8c
[   71.784668] w1_master_driver w1_bus_master1: Family 0 for 00.800000000000.8c is not registered.
[  135.485093] w1_master_driver w1_bus_master1: Attaching one wire slave 00.400000000000 crc 46
[  135.492393] w1_master_driver w1_bus_master1: Family 0 for 00.400000000000.46 is not registered.
[  186.865976] w1_master_driver w1_bus_master1: Attaching one wire slave 00.c00000000000 crc ca
[  186.874597] w1_master_driver w1_bus_master1: Family 0 for 00.c00000000000.ca is not registered.
[  264.556808] w1_master_driver w1_bus_master1: Attaching one wire slave 00.200000000000 crc 23
[  264.568498] w1_master_driver w1_bus_master1: Family 0 for 00.200000000000.23 is not registered.
[  328.086770] w1_master_driver w1_bus_master1: Attaching one wire slave 00.a00000000000 crc af
[  328.096515] w1_master_driver w1_bus_master1: Family 0 for 00.a00000000000.af is not registered.
[  365.997515] w1_master_driver w1_bus_master1: Attaching one wire slave 00.600000000000 crc 65
[  366.016051] w1_master_driver w1_bus_master1: Family 0 for 00.600000000000.65 is not registered.
[  406.536926] w1_master_driver w1_bus_master1: Attaching one wire slave 00.e00000000000 crc e9
[  406.544321] w1_master_driver w1_bus_master1: Family 0 for 00.e00000000000.e9 is not registered.
[  469.527851] w1_master_driver w1_bus_master1: Attaching one wire slave 00.100000000000 crc 9d
[  469.539244] w1_master_driver w1_bus_master1: Family 0 for 00.100000000000.9d is not registered.
[  508.627213] w1_master_driver w1_bus_master1: Attaching one wire slave 00.900000000000 crc 11
[  508.635100] w1_master_driver w1_bus_master1: Family 0 for 00.900000000000.11 is not registered.
[  570.843420] usb 1-1.1.2: new high-speed USB device number 5 using dwc_otg
[  571.001109] usb 1-1.1.2: New USB device found, idVendor=11b0, idProduct=6368, bcdDevice=12.27
[  571.001136] usb 1-1.1.2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[  571.001145] usb 1-1.1.2: Product: Multi-Reader
[  571.001153] usb 1-1.1.2: Manufacturer: Kingston
[  571.001160] usb 1-1.1.2: SerialNumber: 08738180105837
[  571.007539] usb-storage 1-1.1.2:1.0: USB Mass Storage device detected
[  571.008504] scsi host0: usb-storage 1-1.1.2:1.0
[  571.057880] usbcore: registered new interface driver uas
[  572.083003] scsi 0:0:0:0: Direct-Access              Multi-Reader  -0 1.00 PQ: 0 ANSI: 4
[  572.092502] sd 0:0:0:0: [sda] Media removed, stopped polling
[  572.100710] scsi 0:0:0:1: Direct-Access              Multi-Reader  -1 1.00 PQ: 0 ANSI: 4
[  572.106016] sd 0:0:0:1: [sdb] Media removed, stopped polling
[  572.108469] sd 0:0:0:0: [sda] Attached SCSI removable disk
[  572.115817] sd 0:0:0:0: Attached scsi generic sg0 type 0
[  572.116435] sd 0:0:0:1: Attached scsi generic sg1 type 0
[  572.118747] scsi 0:0:0:2: Direct-Access              Multi-Reader  -2 1.00 PQ: 0 ANSI: 4
[  572.119356] sd 0:0:0:2: Attached scsi generic sg2 type 0
[  572.121550] sd 0:0:0:1: [sdb] Attached SCSI removable disk
[  572.129459] scsi 0:0:0:3: Direct-Access              Multi-Reader  -3 1.00 PQ: 0 ANSI: 4
[  572.130283] sd 0:0:0:3: Attached scsi generic sg3 type 0
[  572.132352] sd 0:0:0:2: [sdc] Media removed, stopped polling
[  572.143626] sd 0:0:0:3: [sdd] Media removed, stopped polling
[  572.153617] sd 0:0:0:2: [sdc] Attached SCSI removable disk
[  572.160731] sd 0:0:0:3: [sdd] Attached SCSI removable disk
[  577.449337] sd 0:0:0:1: [sdb] 31116288 512-byte logical blocks: (15.9 GB/14.8 GiB)
[  577.456249] sdb: detected capacity change from 0 to 31116288
[  577.464552]  sdb: sdb1 sdb2
[  583.848101] w1_master_driver w1_bus_master1: Attaching one wire slave 00.500000000000 crc db
[  583.857324] w1_master_driver w1_bus_master1: Family 0 for 00.500000000000.db is not registered.
[  635.798534] w1_master_driver w1_bus_master1: Attaching one wire slave 00.d00000000000 crc 57
[  635.808808] w1_master_driver w1_bus_master1: Family 0 for 00.d00000000000.57 is not registered.

And yet :

=== USB Emoncms import start ===
2023-03-08-16:42:13
Backup module version:
cat: /opt/emoncms/modules/backup/backup-module/module.json: No such file or directory
EUID: 1000
Reading /opt/emoncms/modules/backup/config.cfg....
Location of data databases: /var/opt/emoncms
Location of emonhub.conf: /etc/emonhub
Location of Emoncms: /var/www/emoncms

Scanning for USB card reader:
- Unmounted disk: /dev/disk/by-id/usb-Kingston_Multi-Reader_-0_08738180105837-0:0 at /dev/sda
Error: Error opening /dev/sda: No medium found
- Error: Unmounted disk has incorrect number of partitions
- Unmounted disk: /dev/disk/by-id/usb-Kingston_Multi-Reader_-1_08738180105837-0:1 at /dev/sdb
- Error: Unmounted disk has incorrect number of partitions
- Unmounted disk: /dev/disk/by-id/usb-Kingston_Multi-Reader_-2_08738180105837-0:2 at /dev/sdc
Error: Error opening /dev/sdc: No medium found
- Error: Unmounted disk has incorrect number of partitions
USB drive not found

Although now I look a bit harder, it seems to be moaning about the incorrect number of partitions! Which, I’m honestly not sure why it would care about! But it is a clue!

I think using the adaptor did move it to sdb, which it feels like should then work…

However it warns about number of partitions.

@TrystanLea - can you help?

This is probably as it is the old partition setup, I thought the import coped with that. Looks like it doesn’t.

So, The route for you now is to put the old card in, reboot and do an ‘export’ to file and then reimport that to the new card.

Sorry :frowning:

1 Like

Ah, my problem (I’m pretty sure, anyway) and why I didn’t do that is I do not think I have any export scripts in my pretty bad/broken/I did it wrong, probably old install!

Could I just delete the errant partition? IIRC it’s just swap.

Perhaps I might be missing something, though.

So… On my shiny new EmonSD installation, I have a menu bar at the left side of my browser window, which contains lots of items. Including one called “Backup”.

On my old installation, I have no bar at all there, on the left. I think it’s always been like that, but perhaps when I installed it years ago, I just didn’t realise something was wrong. Might it be that the backup/restore stuff is still physically there, but just the menu referring to it is missing?

I’ll switch back to the old card and do some screenshots.

Oh yes, I’d forgotten that part of the story!

Get the old SD card up and running and we can take it from there.

1 Like

Hmm sorry about this trouble.

Here’s the point in the usb-import script that tries sda, sdb & sdc - but not sdd.
You could try editing that line in the script to see if it gets any further perhaps? L49

The script is located at /opt/emoncms/modules/backup/usb-import.sh

Beyond that if you get really stuck and your happy to give me remote access I could try and dig into this for you. Dataplicity is relatively easy to install via the command line see: https://www.dataplicity.com, there’s a bit more on it here Remote Access — OpenEnergyMonitor 0.0.1 documentation
Feel free to send me a private message if you’d like to do that.

1 Like

Hi Trystan, we got past this point, but the USB import then complains there are not enough partitions. I think it must be an old partition layout and the ‘backup’ from the admin didn’t work.

1 Like

I have now got the old SD card back up and running; came up without a hitch!

Just so you chaps don’t think I’m going mad here’s what I see :

As you can see, it’s weirdly missing loads of stuff! And when I try manually hitting /emoncms/backup/ or similar, it 406s.

Can you post the output from df -h, please?

Use Ctl-E to format, please.

1 Like

I was mis-remembering. I thought the other partition was swap.

No - it seems it is two partitions

/boot and also /

That does make things a bit trickier I think.

pi@raspberrypi:~ $ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/root        15G  7.7G  6.3G  55% /
devtmpfs        459M     0  459M   0% /dev
tmpfs           464M     0  464M   0% /dev/shm
tmpfs           464M   18M  446M   4% /run
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           464M     0  464M   0% /sys/fs/cgroup
/dev/mmcblk0p1   43M   23M   21M  53% /boot
tmpfs            93M     0   93M   0% /run/user/1000
pi@raspberrypi:~ $

Output from ls -la /var/lib/ please?

1 Like

Sure - here goes!

pi@raspberrypi:~ $ ls -la /var/lib
total 204
drwxr-xr-x 51 root     root    4096 May 21  2019 .
drwxr-xr-x 12 root     root    4096 Feb  5  2019 ..
drwxr-xr-x  2 root     root    4096 Mar  8 15:11 alsa
drwxr-xr-x  5 root     root    4096 Feb  5  2019 apache2
drwxr-xr-x  5 root     root    4096 Mar  6 12:17 apt
drwxr-xr-x  2 root     root    4096 Feb  4  2019 aptitude
drwxr-xr-x  2 root     root    4096 Jun 27  2018 aspell
drwx------  4 root     root    4096 Jun 27  2018 bluetooth
drwxr-xr-x  2 root     root    4096 Jun 27  2018 dbus
drwxr-xr-x  2 root     root    4096 Mar  3  2018 dhcp
drwxr-xr-x  2 root     root    4096 Feb 14 13:13 dhcpcd5
drwxr-xr-x  3 root     root    4096 Jun 27  2018 dictionaries-common
drwxr-xr-x  7 root     root    4096 Mar  6 12:27 dpkg
drwxr-xr-x  3 root     root    4096 Jun 27  2018 emacsen-common
drwxr-xr-x  5 root     root    4096 Jun 27  2018 gconf
drwxr-xr-x  3 root     root    4096 Jun 27  2018 gems
drwxr-xr-x  3 root     root    4096 Jun 27  2018 ghostscript
drwxr-xr-x  2 root     root    4096 May 27  2018 git
drwxr-xr-x  2 root     root    4096 Apr 26  2017 initramfs-tools
drwxr-xr-x  2 root     root    4096 Jun 27  2018 ispell
drwxr-xr-x  3 root     root    4096 Jun 27  2018 libreoffice
drwxr-x---  3 lightdm  lightdm 4096 Jun 27  2018 lightdm
drwxr-xr-x  2 root     root    4096 Mar  8 06:25 logrotate
drwxr-xr-x  2 root     root    4096 Jun 27  2018 man-db
drwxr-xr-x  2 root     root    4096 Aug 21  2018 misc
drwxr-xr-x  5 mysql    mysql   4096 Mar  8 15:11 mysql
drwxr-xr-x  4 root     root    4096 May 26  2021 nfs
drwxr-xr-x  2 root     root    4096 Jun 27  2018 ntp
drwxr-xr-x  2 root     root    4096 Jan 30  2018 openbox
drwxr-xr-x  6 openhab  openhab 4096 Sep  1  2018 openhab2
drwxr-xr-x  2 root     root    4096 Aug 21  2018 PackageKit
drwxr-xr-x  2 root     root    4096 Mar  6 12:27 pam
drwxr-xr-x  4 root     root    4096 May 17  2019 php
drwxr-xr-x  2 www-data root    4096 Sep 30  2019 phpfina
drwxr-xr-x  2 www-data root    4096 May 21  2019 phpfiwa
drwxr-xr-x  2 www-data root    4096 May 21  2019 phptimeseries
drwxr-xr-x  2 root     root    4096 Jun 27  2018 plymouth
drwx------  3 root     root    4096 Jun 27  2018 polkit-1
drwxr-xr-x  2 root     root    4096 Jun 27  2018 python
drwxr-x---  2 redis    redis   4096 Mar  8 17:53 redis
drwxr-xr-x  3 root     root    4096 Jun 27  2018 samba
drwxr-xr-x  2 root     root    4096 Jan 26  2019 sgml-base
drwx--x--x  4 root     root    4096 Jun 27  2018 sudo
drwxr-xr-x  7 root     root    4096 Jun 27  2018 systemd
drwxr-xr-x  3 root     root    4096 May 26  2021 ucf
drwx------  2 root     root    4096 Jun 27  2018 udisks2
drwxr-xr-x  2 root     root    4096 Jan 23  2017 usb_modeswitch
drwxr-xr-x  2 root     root    4096 May 26  2021 usbutils
drwxr-xr-x  3 root     root    4096 Jun 27  2018 vim
drwxr-xr-x  2 root     root    4096 Mar  8 15:11 xkb
drwxr-xr-x  2 root     root    4096 Jan 26  2019 xml-core
pi@raspberrypi:~ $

It looks like you have a fairly new version of emoncms on there! But it’s probably is easier to figure out the import using a the new SD card and a usb reader than to figure out the export on the old image… the backup module will be expecting a particular structure for an export which it might not be successful in finding…

1 Like