Backup feed emoncms 8.4.0

hello everyone, on a site that offers the hosting service, for some years now I have configured emoncms in version 8.4.0. It is still functional today.
Now, on another hosting site I have installed Emoncms 10.8.5 and configured this as well.
The problem is that I would like to move the history of some feeds stored in the old version to the new one, so that if I look at the dashboard on emoncms I see the graph of the values received with the old version …
I tried with the backup script in python but it doesn’t work.
I was wondering if I could, with an arduino, read a feed from the old version and write it in the new one but I don’t know how the timestamp request works …
There is someone who can give me some “simple” tips or advice since I’m not very grasped in programming but of good will :slight_smile:

thank you very much and have a nice day

Mauro

Can you share the commands you tried, or describe what you clicked, and how it doesn’t work, please? Absolutely no data? Mangled? Any error messages?

One for @TrystanLea I think.

Here I am…
I just launched the “data_downloader.py” file, entered the fields correctly (username, password and url of the site where emoncms 8.4.0 is) and everything I entered below comes out.

Except, for me, he can’t have downloaded everything in such a short time (we’re talking about 50 feeds weighing about 30/50 megabytes)

however he created a folder (see attached photo) and only inside “phptimeseries” there are some files, but too light to contain the data.

Am I doing something wrong?

thank you very much for the time you dedicate to me.
Have a good evening

Python 3.11.0 (main, Oct 24 2022, 18:26:48) [MSC v.1933 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license()" for more information.

= RESTART: C:\Users\Mauro\AppData\Local\Programs\Python\Python311\data_downloader.py
PHPTimeSeries id=37 downloaded: 5 bytes
PHPTimeSeries id=38 downloaded: 5 bytes
PHPTimeSeries id=39 downloaded: 5 bytes
PHPTimeSeries id=40 downloaded: 5 bytes
PHPTimeSeries id=41 downloaded: 5 bytes
PHPTimeSeries id=42 downloaded: 5 bytes
PHPTimeSeries id=43 downloaded: 5 bytes
PHPTimeSeries id=44 downloaded: 5 bytes
PHPTimeSeries id=45 downloaded: 5 bytes
PHPTimeSeries id=46 downloaded: 5 bytes
PHPTimeSeries id=47 downloaded: 5 bytes
PHPTimeSeries id=48 downloaded: 5 bytes
PHPTimeSeries id=50 downloaded: 5 bytes
PHPTimeSeries id=52 downloaded: 5 bytes
PHPTimeSeries id=54 downloaded: 5 bytes
PHPTimeSeries id=56 downloaded: 5 bytes
PHPTimeSeries id=58 downloaded: 5 bytes
PHPTimeSeries id=60 downloaded: 5 bytes
PHPTimeSeries id=62 downloaded: 5 bytes
PHPTimeSeries id=80 downloaded: 5 bytes
PHPTimeSeries id=64 downloaded: 5 bytes
PHPTimeSeries id=65 downloaded: 5 bytes
PHPTimeSeries id=66 downloaded: 5 bytes
PHPTimeSeries id=67 downloaded: 5 bytes
PHPTimeSeries id=68 downloaded: 5 bytes
PHPTimeSeries id=69 downloaded: 5 bytes
PHPTimeSeries id=70 downloaded: 5 bytes
PHPTimeSeries id=71 downloaded: 5 bytes
PHPTimeSeries id=72 downloaded: 5 bytes
PHPTimeSeries id=73 downloaded: 5 bytes
PHPTimeSeries id=79 downloaded: 5 bytes

Cattura1

Hi, is there anyone who has any ideas I can do to fix the backup problem?

have a good day
Mauro

Really needs @TrystanLea but he is quite busy ATM.

Hello @fipetto does the content of feeds.json look correct?
If you open one of those files with a text editor do you see any text?
Did you expect to download PHPFina feeds or PHPTimeSeries only?

1 Like

I think I’ve spotted the issue, v8.4 required the apikey used to export data to be the read & write apikey.
Can you try changing this line in the script from:

apikey = user_auth['apikey_read']

to

apikey = user_auth['apikey_write']

https://github.com/emoncms/usefulscripts/blob/master/backup_py/data_downloader.py#L80

1 Like

Hi, thanks for the reply.
I haven’t opened the file you talk about, I’ll try tonight and also modify the line of code you told me, then I’ll let you know.

But one question, how do i import data into emoncms 10?
I mean those files that I downloaded do I have to copy them inside a folder of emoncms 10? Will this allow me to see past years’ charts on emoncms 10?

sorry for all these questions but it is not clear to me if the backup is for this (which is what I would like …)

thanks again, I’ll let you know as soon as I try.

Hello everybody

Ah, no backup_py is only for creating a local copy of the data.

You can use the php version for direct import into the latest emoncms, see: usefulscripts/backup at master · emoncms/usefulscripts · GitHub

Unfortunately I also tried that path … I went crazy because I had to install php via xampp on my pc, I downloaded the backup.php file and its lib folder. I put the apkey write, set the site where emoncms 8.4.0 resides, created the various folders ph … as written in the instructions and launched the program, which started working the first few seconds, then gave me the message that see in the picture and from there, even if I run the program again it always gives me the same writing … help :slight_smile:

for that I wrote at the beginning of the request if it was possible to solve with an arduino … my idea was to request the feed to emoncms 8.4.0 with the time set (date, hour and minutes) and write to Emoncms 10 l 'information with the same information of the time … but … I don’t know how to make the request, I didn’t understand if what I mean is called “timestamp” and I don’t know if emoncm 8.4.0 supports it … …

are you running the new emoncms in xampp?

no, both emoncms are installed on altervista.org a site that provides a hosting service.
The backup.php file I did not know how to run it in my windows pc and so I installed php and ran it but after downloading a little bit it got stuck with that feed and I’m stopped.

Consider that despite having a lot of good will in trying to understand, unfortunately I lack the basics to find the right path …

Arduino was an idea since I already understand it a little more but I miss the idea of “get …” and it was a way to avoid bothering the boxes :wink:

ps. always sorry for my english but i am using a translator.

No worries. The PHP script is designed to be ran on the server hosting emoncms itself. Is it a Linux server? or shared hosting?

unfortunately I can’t run it from inside the site … even here I hit my head a lot before asking the altervista forum, and I was told that there is a block for the programs inside the host space to create files … . I have understood this at least … in fact by installing it inside, the backup.php file and the library does not find anything even if configured (I think) correctly …
but maybe I didn’t do something wrong …

If I make you a screenshot of where it is installed and then another photo of the parameters that I have changed of the backup file can you tell me if I did it right?

how do i know if it is a linux server or shared hosting? I only know that they make available (for free) a portion of space where you can make websites. Instead I used it to upload your emoncms and from there it all started and everything works I can’t complain …
I know you have the ability to change php versions, you have a mysql database, phpadmin … I don’t know if I’m explaining myself well …

Surely the safest way to backup files is to produce a list on plain paper of all the files and their absolute/relative paths (using words “all files” or wildcards if necessary)?

Then the user can create their own script, use Filezilla, CPanel, the Operating System or whatever to copy/download/upload as appropriate.

@Robert.Wall a good idea!

@fipetto it may be easiest in this case to copy the mysql database across directly using mysqldump (there are guides online on how to do this). Then you can copy the phptimeseries and phpfina directories over as well. You can try that, it may or may not work and we don’t have a guide on how to do it that way…

In general we recommend using a small VM linux server such as the DigitalOcean droplets and then using our EmonScripts install script to get the full emoncms stack, you can then run the backup script to copy the data across… see this guide here: EmonScripts/digital-ocean-install.md at stable · openenergymonitor/EmonScripts · GitHub

Hello everyone, you don’t know how much envy I’m feeling because you write in such a technical language (I’d like to express myself like this too but … study Fipetto !! :-)) …

I’m trying to figure out which is the easiest way …
If I backup the mysql server (I saw on altervista.org that there is a way to do it) then how can I transport it to the mysql of the other installation?
Then just copy the folders you wrote me?

As for the linux server, I looked at the Borpin guide but honestly didn’t understand anything (my fault)

The data is stored in mysql right?

I just looked and from what I see the “weight” of the data I have them inside the emoncms folder and not on the database.
I am attaching the 2 screenshots (in Italian).

It is written that the files weigh 2.6 Gbytes, while the database a few kiloBytes …