Community
OpenEnergyMonitor

Community

emonPi Import data failed


(Gerard MARTIN) #1

Hello,

After updating to SD-30Oct18,I can’t import my backup (see error message).
I try to manually import but it does not work too (see terminal script).
My server informations seems to be correct (see capture screens).
My computer is an iMac under OS 10.11.6 and my web browser is Safari 11.1.2

How could I fix it?


Import data failed after SD image migration
(Gerard MARTIN) #2

Hello,

Sorry, I forgot attached files in my previous message.

After updating to SD-30Oct18,I can’t import my backup (see error message).
I try to manually import but it does not work too (see terminal script).
My server informations seems to be correct (see capture screens).
My computer is an iMac under OS 10.11.6 and my web browser is Safari 11.1.2

How could I fix it?


(Trystan Lea) #3

Hello @Gerard1 it looks like the export did not work correctly as the import failed to decompress the compressed archive properly. Do you still have the original image? can you decompress the archive manually on your computer? or is it corrupted?


(Brian Orpin) #4

Hi @Gerard1, if you just use the button on the Admin screen, next to ‘Server Information’ to Copy to Clipboard you can then just do a paste (no formatting required) and the information is inserted like this:

Server Information
Emoncms Version 9.9.6-beta
Modules Administration : App v1.1.1 : Backup v1.1.4 : Dashboard v1.3.0 : Device v1.1.0 : EventProcesses : Feed : Graph v1.2.1 : Input : CoreProcess : Schedule : Time : User : Visualisation
Git URL: https://github.com/emoncms/emoncms.git : Branch: * master : Describe: 9.9.4-17-g97feb998
Server OS Linux 4.9.0-6-amd64
Host DietPi-EmoncmsVM DietPi-EmoncmsVM (192.168.7.47)
Date 2018-11-17 11:57:01 UTC
Uptime 11:57:01 up 21:19, 1 user, load average: 0.00, 0.00, 0.00
HTTP Server lighttpd/1.4.45 HTTP/1.1 CGI/1.1 80
MySQL Version 5.5.5-10.1.26-MariaDB-0+deb9u1
Host localhost (127.0.0.1)
Date 2018-11-17 11:57:01 (UTC 00:00‌​)
Stats Uptime: 76773 Threads: 2 Questions: 124045 Slow queries: 0 Opens: 42 Flush tables: 1 Open tables: 36 Queries per second avg: 1.615
Redis Version 3.2.6
Host localhost:6379 (127.0.0.1)
Size 6032 keys (1.62M)
Uptime 0 days
MQTT Server Version Mosquitto
Host 192.168.7.50:1883 (192.168.7.50)
Memory RAM Used: 30.87% Total: 492.5 MB Used: 152.06 MB Free: 340.45 MB
Swap Used: 0.00% Total: 1.52 GB Used: 0 B Free: 1.52 GB
Disk Mount Stats
/ Used: 38.62% Total: 7.81 GB Used: 3.02 GB Free: 4.7 GB
PHP Version 7.0.30-0+deb9u1 (Zend Version 3.0.0)
Modules apcu v5.1.8 : calendar v7.0.30-0+deb9u1 : cgi-fcgi : Core v7.0.30-0+deb9u1 : ctype v7.0.30-0+deb9u1 : curl v7.0.30-0+deb9u1 : date v7.0.30-0+deb9u1 : dom v20031129 : exif v7.0.30-0+deb9u1 : fileinfo v1.0.5 : filter v7.0.30-0+deb9u1 : ftp v7.0.30-0+deb9u1 : gd v7.0.30-0+deb9u1 : gettext v7.0.30-0+deb9u1 : hash v1.0 : iconv v7.0.30-0+deb9u1 : json v1.4.0 : libxml v7.0.30-0+deb9u1 : mbstring v7.0.30-0+deb9u1 : mcrypt v7.0.30-0+deb9u1 : mosquitto v0.4.0 : mysqli v7.0.30-0+deb9u1 : mysqlnd vmysqlnd 5.0.12-dev - 20150407 - $Id: b5c5906d452ec590732a93b051f3827e02749b83 $ : openssl v7.0.30-0+deb9u1 : pcre v7.0.30-0+deb9u1 : PDO v7.0.30-0+deb9u1 : pdo_mysql v7.0.30-0+deb9u1 : Phar v2.0.2 : posix v7.0.30-0+deb9u1 : readline v7.0.30-0+deb9u1 : redis v3.1.6 : Reflection v7.0.30-0+deb9u1 : session v7.0.30-0+deb9u1 : shmop v7.0.30-0+deb9u1 : SimpleXML v7.0.30-0+deb9u1 : sockets v7.0.30-0+deb9u1 : SPL v7.0.30-0+deb9u1 : standard v7.0.30-0+deb9u1 : sysvmsg v7.0.30-0+deb9u1 : sysvsem v7.0.30-0+deb9u1 : sysvshm v7.0.30-0+deb9u1 : tokenizer v7.0.30-0+deb9u1 : wddx v7.0.30-0+deb9u1 : xml v7.0.30-0+deb9u1 : xmlreader v7.0.30-0+deb9u1 : xmlwriter v7.0.30-0+deb9u1 : xsl v7.0.30-0+deb9u1 : Zend OPcache v7.0.30-0+deb9u1 : zip v1.13.5 : zlib v7.0.30-0+deb9u1
Client Information
HTTP Browser Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36 Vivaldi/2.1.1337.36
Screen Resolution 1920 x 1080
Window Size 1403 x 877

HTH :smile:


(Gerard MARTIN) #5

Hello,

Unfortunately I do not still have the orignal image but I can decompress the archive manually using « utilitaire d’archives » standard application on my iMac.

This is what I get.


(Trystan Lea) #6

I think the attachment is missing @Gerard1?


(Gerard MARTIN) #7

Here are the attachments


(Gerard MARTIN) #8

OK, thank you Brian.


(Robert Wall) #9

Still no attachments. I think you need to log in to the website and not send via email.


(Gerard MARTIN) #10

I am sorry for this malfunction.
Here are the attachements.
12


(Trystan Lea) #11

Good to see that the data is present. Have you tried uploading multiple times? Im not sure what’s going wrong unfortunately. Did you decompress and recompress on your computer?

I tested the decompression command here in isolation on a backup which you could test as a starting point:

cd /home/pi/data/uploads
mkdir test
tar xfzv emoncms-backup-2018-11-16.tar.gz -C test

The error you see is the same as this stackoverflow https://stackoverflow.com/questions/15744023/how-to-extract-filename-tar-gz-file/15744167 but the backup archive should be valid for decompression with the command in the importer…


(Gerard MARTIN) #12

I am a beginner and I am not very familiar with Linux commands.
It seems that I am not allowed to create a test directory. Should I use sudo mkdir instead of mkdir?
However you can see on the attached screenshot that there are five backup files.
Since I was unable to restore my backup after the SD image was migrated, I reconfigured my entries to resume the recordings. That’s why the 2018-11-16-2.tar and the 2018-11-17.tar files are therefore smaller in size.

02


(Trystan Lea) #13

Sorry about the linux commands , Yes you could try running both as sudo.

I wonder if emoncms-backup-2018-11-16.gz is tripping it up, it shouldn’t do as the script does search for tar.gz file extension and the correct filename is listed in your log, could you move the existing uploads out of that folder and try again?

Perhaps @glyn.hudson might have an idea.


(Glyn Hudson) #14

I’m not sure what’s gone wrong, maybe the archive is corrupted in some way. Would you be happy to send us the archive for us to take a look? If so, the best way to do this is upload to dropbox / grdive then send us a link to [email protected].

I will update this thread with the results of my analysis.


(Gerard MARTIN) #15

OK, old data has been imported successfully. It just misses the last 24h.
After that, I erased all old files from the uploads directory, created a new backup, downloaded the associated .tar.gz file, and tried to re-import the backup.
Unfortunately, I encountered the same problem.
I noticed that after downloading, the tar.gz file was transformed into a .tar file.
Could this explain the problem?

09


(Gerard MARTIN) #16

I finally found a way to import a backup.
After performing a backup and downloading the archive to my computer, I use the application “keka” to compress the downloaded file. tar in .tar.gz file and then I import this .tar.gz file.
It’s a bit complicated, but it works.
Thanks for your help.


(Trystan Lea) #17

Great to hear you got it working @Gerard1. To check I understand correctly, the emonpi is offering the backup to download with the .tar extension only?


(Gerard MARTIN) #18

It seems that the backup generated by emonPi is indeed in .tar.gz but when I download this file on my computer, it appears in .tar only. So I have to use “Keka” to compress the downloaded file in .tar.gz
It may be my operating system (OS El Capitan version 10.11.6 with Safari version 11.1.2) that forces the download in .tar

33


(Robert Wall) #19

I do not know the Mac OS, but I think I have read somewhere that it is the Mac OS that “helpfully” decompresses any compressed file automatically.

In this case, it got it wrong.


(Peter Middleton) #20

Computers very rarely make mistakes … they just do as they are told.

Safari has a setting in Preferences -> General -> Open “safe” files after downloading.

Turn this off and the .tar.gz file won’t be decompressed.

HTH

Peter