emonPi Import data failed

TrystanLea · 2018-11-17T18:20:49.476Z

I think the attachment is missing @Gerard1?

Gerard1 · 2018-11-17T18:30:05.000Z

Here are the attachments

Gerard1 · 2018-11-17T19:23:38.000Z

OK, thank you Brian.

Robert.Wall · 2018-11-17T19:34:15.177Z

Still no attachments. I think you need to log in to the website and not send via email.

Gerard1 · 2018-11-17T19:45:37.519Z

I am sorry for this malfunction.
Here are the attachements.

49955×442 68.6 KB

TrystanLea · 2018-11-17T20:25:28.215Z

Good to see that the data is present. Have you tried uploading multiple times? Im not sure what’s going wrong unfortunately. Did you decompress and recompress on your computer?

I tested the decompression command here in isolation on a backup which you could test as a starting point:

cd /home/pi/data/uploads
mkdir test
tar xfzv emoncms-backup-2018-11-16.tar.gz -C test

The error you see is the same as this stackoverflow linux - How to extract filename.tar.gz file - Stack Overflow but the backup archive should be valid for decompression with the command in the importer…

Gerard1 · 2018-11-17T22:38:39.327Z

I am a beginner and I am not very familiar with Linux commands.
It seems that I am not allowed to create a test directory. Should I use sudo mkdir instead of mkdir?
However you can see on the attached screenshot that there are five backup files.
Since I was unable to restore my backup after the SD image was migrated, I reconfigured my entries to resume the recordings. That’s why the 2018-11-16-2.tar and the 2018-11-17.tar files are therefore smaller in size.

TrystanLea · 2018-11-17T23:39:34.678Z

Sorry about the linux commands , Yes you could try running both as sudo.

I wonder if emoncms-backup-2018-11-16.gz is tripping it up, it shouldn’t do as the script does search for tar.gz file extension and the correct filename is listed in your log, could you move the existing uploads out of that folder and try again?

Perhaps @glyn.hudson might have an idea.

glyn.hudson · 2018-11-18T01:57:35.062Z

I’m not sure what’s gone wrong, maybe the archive is corrupted in some way. Would you be happy to send us the archive for us to take a look? If so, the best way to do this is upload to dropbox / grdive then send us a link to support@openenergymonitor.zendesk.com.

I will update this thread with the results of my analysis.

Gerard1 · 2018-11-18T02:05:57.894Z

OK, old data has been imported successfully. It just misses the last 24h.
After that, I erased all old files from the uploads directory, created a new backup, downloaded the associated .tar.gz file, and tried to re-import the backup.
Unfortunately, I encountered the same problem.
I noticed that after downloading, the tar.gz file was transformed into a .tar file.
Could this explain the problem?

51942×446 46.5 KB

Gerard1 · 2018-11-18T08:02:42.576Z

I finally found a way to import a backup.
After performing a backup and downloading the archive to my computer, I use the application “keka” to compress the downloaded file. tar in .tar.gz file and then I import this .tar.gz file.
It’s a bit complicated, but it works.
Thanks for your help.

TrystanLea · 2018-11-18T20:03:21.591Z

Great to hear you got it working @Gerard1. To check I understand correctly, the emonpi is offering the backup to download with the .tar extension only?

Gerard1 · 2018-11-18T21:36:11.406Z

It seems that the backup generated by emonPi is indeed in .tar.gz but when I download this file on my computer, it appears in .tar only. So I have to use “Keka” to compress the downloaded file in .tar.gz
It may be my operating system (OS El Capitan version 10.11.6 with Safari version 11.1.2) that forces the download in .tar

05980×512 55.9 KB

Robert.Wall · 2018-11-18T21:45:32.518Z

Gerard1:

when I download this file on my computer, it appears in .tar only.

I do not know the Mac OS, but I think I have read somewhere that it is the Mac OS that “helpfully” decompresses any compressed file automatically.

In this case, it got it wrong.

pem · 2018-11-22T03:54:48.061Z

Robert.Wall:

In this case, it got it wrong.

Computers very rarely make mistakes … they just do as they are told.

Safari has a setting in Preferences → General → Open “safe” files after downloading.

Turn this off and the .tar.gz file won’t be decompressed.

HTH

Peter

Gerard1 · 2018-11-22T07:00:24.282Z

Thank you Peter for your useful help.

Gerard

TrystanLea · 2018-11-22T09:00:40.050Z

Great, glad the reason was found, I was really scratching my head there!

Robert.Wall · 2018-11-22T09:37:00.610Z

pem:

Computers very rarely make mistakes … they just do as they are told.

If you want to be a pedant, @pem, then whoever decided that the user always wants a compressed file decompressing got it wrong.

pem · 2018-11-22T11:37:11.362Z

Robert.Wall:

If you want to be a pedant

Well I wasn’t Sorry to be of help.

Robert.Wall · 2018-11-22T14:34:04.831Z

pem:

Safari has a setting in Preferences -> General -> Open “safe” files after downloading.

was indeed being helpful.

pem:

Computers very rarely make mistakes … they just do as they are told.

You obviously understood the meaning. It reminds me of a former colleague who often said “I want the computer to do what I want it to do, not what I tell it to do”