Backup v2.1.7 Error restoring an Local EmonBase backed up today. Image emonSD-17Oct19 (stable)

Luiflm2 · 19 November 2019 17:22

I have an EmonBase with new installed image emonSD-17Oct19 (stable).
By full update to version Emoncms version low-write 10.1.13 upgraded.
Created a backup from an EmonBase with Emoncms version low-write 9.9.6 via the backup menu.
This backup into the new EmonBase Buster with Image emonSD-17Oct19 and Emoncms version low-write 10.1.13 installed, reboot, everything is OK and the EmonBase is fully functional.

Today I created a backup of the EmonBase Buster Emoncms version low-write 10.1.13, there were no error messages, the log shows no errors.
When restoring the emoncms-backup-2019-11-19.tar.gz to the same EmonBase Buster Emoncms version low-write 10.1.13 an error occurs:

„ Notice : Undefined index: file in /opt/emoncms/modules/backup/backup-module/backup_controller.php on line 89“ „Error: Import archive not selected“

The importbackup.log is empty, for information: the backup has compressed 56 MB and decompressed approx. 2GB

Server Information

Services

emonhub :- Active Running
emoncms_mqtt :- Active Running
feedwriter :- Active Running - sleep 300s 469 feed points pending write
service-runner :- Active Running
emonPiLCD :- Inactive Dead
redis-server :- Active Running
mosquitto :- Active Running
demandshaper :- Active Running

Emoncms

Version :- low-write 10.1.13
Modules :- Administration | App v2.0.9 | Backup v2.1.7 | EmonHub Config v2.0.4 | Dashboard v2.0.5 | DemandShaper v1.1.0 | Device v2.0.3 | EventProcesses | Feed | Graph v2.0.6 | Input | Postprocess v2.1.2 | CoreProcess | Schedule | Network Setup v1.0.0 | sync | Time | User | Visualisation | WiFi v2.0.2
Git :-
- URL :- GitHub - emoncms/emoncms: Web-app for processing, logging and visualising energy, temperature and other environmental data
- Branch :- * master
- Describe :- 10.1.11-9-g246c17ed

Server

OS :- Linux 4.19.83-v7+
Host :- emonpibu | emonpibu | (10.255.255.26)
Date :- 2019-11-19 18:20:54 CET
Uptime :- 18:20:54 up 1:44, 0 users, load average: 0.27, 0.38, 0.40

Memory

RAM :- Used: 20.12%
- Total :- 975.61 MB
- Used :- 196.26 MB
- Free :- 779.36 MB
Swap :- Used: 86.68%
- Total :- 100 MB
- Used :- 86.68 MB
- Free :- 13.32 MB
  Write Load Period

Disk

/ :- Used: 32.78%
- Total :- 7.86 GB
- Used :- 2.58 GB
- Free :- 4.91 GB
- Write Load :- 378.77 KB/s (1 hours 43 mins)
/var/opt/emoncms :- Used: 9.74%
- Total :- 21.11 GB
- Used :- 2.06 GB
- Free :- 17.98 GB
- Write Load :- 364.84 B/s (1 hours 43 mins)
/boot :- Used: 20.65%
- Total :- 252.05 MB
- Used :- 52.05 MB
- Free :- 200 MB
- Write Load :- 0 B/s (1 hours 43 mins)
/var/log :- Used: 7.31%
- Total :- 50 MB
- Used :- 3.66 MB
- Free :- 46.34 MB
- Write Load :- n/a

HTTP

Server :- Apache/2.4.38 (Raspbian) HTTP/1.1 CGI/1.1 80

MySQL

Version :- 5.5.5-10.3.17-MariaDB-0+deb10u1
Host :- localhost:6379 (127.0.0.1)
Date :- 2019-11-19 18:20:53 (UTC 01:00‌)
Stats :- Uptime: 6281 Threads: 16 Questions: 13769 Slow queries: 0 Opens: 127 Flush tables: 1 Open tables: 121 Queries per second avg: 2.192

Redis

Version :-
- Redis Server :- 5.0.3
- PHP Redis :- 5.0.2
Host :- localhost:6379
Size :- 560 keys (866.73K)
Uptime :- 0 days

MQTT Server

Version :- Mosquitto 1.5.7
Host :- localhost:1883 (127.0.0.1)

PHP

Version :- 7.3.11-1~deb10u1 (Zend Version 3.3.11)
Modules :- apache2handler | calendar v7.3.11-1~deb10u1 | Core v7.3.11-1~deb10u1 | ctype v7.3.11-1~deb10u1 | curl v7.3.11-1~deb10u1 | date v7.3.11-1~deb10u1 | dom v20031129 | exif v7.3.11-1~deb10u1 | fileinfo v7.3.11-1~deb10u1 | filter v7.3.11-1~deb10u1 | ftp v7.3.11-1~deb10u1 | gd v7.3.11-1~deb10u1 | gettext v7.3.11-1~deb10u1 | hash v7.3.11-1~deb10u1 | iconv v7.3.11-1~deb10u1 | json v1.7.0 | libxml v7.3.11-1~deb10u1 | mbstring v7.3.11-1~deb10u1 | mosquitto v0.4.0 | mysqli v7.3.11-1~deb10u1 | mysqlnd vmysqlnd 5.0.12-dev - 20150407 - $Id: 7cc7cc96e675f6d72e5cf0f267f48e167c2abb23 $ | openssl v7.3.11-1~deb10u1 | pcre v7.3.11-1~deb10u1 | PDO v7.3.11-1~deb10u1 | pdo_mysql v7.3.11-1~deb10u1 | Phar v7.3.11-1~deb10u1 | posix v7.3.11-1~deb10u1 | readline v7.3.11-1~deb10u1 | redis v5.0.2 | Reflection v7.3.11-1~deb10u1 | session v7.3.11-1~deb10u1 | shmop v7.3.11-1~deb10u1 | SimpleXML v7.3.11-1~deb10u1 | sockets v7.3.11-1~deb10u1 | sodium v7.3.11-1~deb10u1 | SPL v7.3.11-1~deb10u1 | standard v7.3.11-1~deb10u1 | sysvmsg v7.3.11-1~deb10u1 | sysvsem v7.3.11-1~deb10u1 | sysvshm v7.3.11-1~deb10u1 | tokenizer v7.3.11-1~deb10u1 | wddx v7.3.11-1~deb10u1 | xml v7.3.11-1~deb10u1 | xmlreader v7.3.11-1~deb10u1 | xmlwriter v7.3.11-1~deb10u1 | xsl v7.3.11-1~deb10u1 | Zend OPcache v7.3.11-1~deb10u1 | zlib v7.3.11-1~deb10u1

Pi

Model :- Raspberry Pi 3 Model B Rev 1.2 - 1GB (Sony UK)
Serial num. :- 98B6168C
Temperature :- 49.39°C - 48.9°C
emonpiRelease :- emonSD-17Oct19
File-system :- read-write

Client Information

HTTP

Browser :- Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:70.0) Gecko/20100101 Firefox/70.0
Language :- de,en-US;q=0.7,en;q=0.3

Window

Size :- 1903 x 939

Screen

Resolution :- 1920 x 1080

What am I doing wrong, do I have to configure anything here?

borpin · 19 November 2019 21:28

So I have just tried this (do and export then an import) and it worked fine for me.

You did select a file by clicking on Choose File?

so it looks like this

Luiflm2 · 19 November 2019 22:24

@borpin, Yes, I have selected the file and pressed the Import button,
the error message appears after approx. 8 seconds after pressing the Import button.
The importbackup.log is empty (0byte)

TrystanLea · 21 November 2019 11:57

Hello @Luiflm2 Im not sure why this is happening, it sounds like its getting stuck in the initial file upload stage, the importbackup.log is only written to after the initial upload. It might be worth checking the apache log:

/var/log/emoncms/apache2-error.log

Alternatively you may like to try the direct usb import script. Which may work a bit faster with your 2GB data size, see Backup module usb-import.sh script (beta) for full details.

Luiflm2 · 21 November 2019 17:04

Hello @TrystanLea, in /var/log/emoncms/apache2-error.log there are actually some errors :
[Thu Nov 21 17:43:05.339778 2019] [php7:notice] [pid 1177] [client 10.255.255.2:49409] PHP Notice: Undefined index: hw in /var/www/emoncms/Modules/admin/admin_main_view.php on line 397, referer: http://emonpibu/emoncms/feed/list
[Thu Nov 21 17:43:45.236092 2019] [php7:warn] [pid 1175] [client 10.255.255.2:49416] PHP Warning: POST Content-Length of 58693189 bytes exceeds the limit of 8388608 bytes in Unknown on line 0, referer: http://emonpibu/backup
[Thu Nov 21 17:43:45.248743 2019] [php7:notice] [pid 1175] [client 10.255.255.2:49416] PHP Notice: Undefined index: file in /opt/emoncms/modules/backup/backup-module/backup_controller.php on line 78, referer: http://emonpibu/backup
[Thu Nov 21 17:43:45.248917 2019] [php7:notice] [pid 1175] [client 10.255.255.2:49416] PHP Notice: Undefined index: file in /opt/emoncms/modules/backup/backup-module/backup_controller.php on line 89, referer: http://emonpibu/backup
[Thu Nov 21 17:44:09.282394 2019] [php7:notice] [pid 692] [client 10.255.255.2:49420] PHP Notice: Undefined index: hw in /var/www/emoncms/Modules/admin/admin_main_view.php on line 397, referer: http://emonpibu/backup/upload

borpin · 21 November 2019 17:18

I have raised an issue

github.com/emoncms/backup

Restoring large file fails without usable error message

opened 05:16PM - 21 Nov 19 UTC

borpin

bug

https://community.openenergymonitor.org/t/backup-v2-1-7-error-restoring-an-local…-emonbase-backed-up-today-image-emonsd-17oct19-stable/12522 A large backup file restore failed with an error that did not make sense to the user. It seems it failed because the backup file was too large. Should the size of file that can be backed up match the size of a file that can be restored? If a data partition of 10Gb is being created, it should be possible to backup and restore larger data sets.

TrystanLea · 19 January 2020 09:03

Hello @Luiflm2 apologies for the delay on this, did you find another solution?

I’m unable to replicate the upload issue, I’ve just uploaded a 100mb backup without any issue.

I’m aware however that there is a separate issue caused when a large archive is unzipped (there is not enough space on the root partition for a ~2GB archive to unzip). Im almost there with a fix for this issue, for details see: Moving backup export and uploads location to /var/opt/emoncms/backup · Issue #51 · emoncms/backup · GitHub.

We do however have an alternative import tool now, where you can import straight from the old SD card using an SD card reader, see the guide here Import / Backup - Guide | OpenEnergyMonitor. If you still need to import the backup I would recommend trying this.

Luiflm2 · 19 January 2020 10:35

@TrystanLea, I haven’t been able to import the backup yet, I’ve using 32gb sd cards with customized partitions (GParted) for years. I’ll read and test the guide. Thanks