Data from remote host to local pi (sd card image)

[boelle (Bo Herrmannsen)]

@borpin

So copy (sync or one off?) data that is currently stored on the hosted emoncms in mysql to a local Pi install.

one off

@borpin

Is this a new SD Card?

no from late 2016 early 2017

[borpin (Brian Orpin)]

@boelle

no from late 2016 early 2017

Could you update the local Card? Is there a reason not to?

What version of EmonCMS on the local instance?

[boelle (Bo Herrmannsen)]

@borpin

What version of EmonCMS on the local instance?

Powered by OpenEnergyMonitor.org | low-write 9.9.9

[borpin (Brian Orpin)]

@borpin

Could you update the local Card? Is there a reason not to?

As you are doing some structural changes, it might be a good opportunity to do this migration. The Sync module is installed by default on the newer version.

I’m equally not sure if Sync will pull in mysql feeds but I suspect it will, though whether it will convert to a different storage engine is another unknown to me.

@TrystanLea, could you chime in here?

[TrystanLea]

The sync module is only compatible with phpfina and phptimeseries feeds. Either a mysqldump or a direct copy of the mysql folders (e.g if mysqldump is too large/slow) is likely the easiest way to copy the data across.

It’s worth noting thought that mysql feed data is not optimised for low write load and so will likely wear the SD card faster, if you are using an SD card?

There’s an old archived script for converting mysql feed data to phptimeseries here: usefulscripts/sql_to_phptimeseries_mv.php at master · emoncms/usefulscripts · GitHub but its not something I’ve touched for many years and is archived so may not work.

[boelle (Bo Herrmannsen)]

i did some manual work to port data and settings but i screwed up a bit

some of the phpfina tables got mixed up, are there also scripts that can convert mysql to phpfina?

[boelle (Bo Herrmannsen)]

sql_to_phptimeseries_mv.php

would get me half the way

then i would just need the reverse of

phpfina_to_phptimeseries.php

[borpin (Brian Orpin)]

@boelle

then i would just need the reverse of

phpfina_to_phptimeseries.php

I’ll predicate this by saying I’m no expert in the data storage @TrystanLea is your man for that.

The documentation is here https://learn.openenergymonitor.org/electricity-monitoring/timeseries/History

I think that you can’t as PHPFINA is a fixed time slot, no average (FI = Fixed, NA = No Average) storage and the SQL is a variable time storage thus can be converted to a timeseries store.

[Bill.Thomson]

@borpin

(FI = Fixed, NA = No Average)

If memory serves, that’s Fixed Interval No Average.

[borpin (Brian Orpin)]

Couldn’t work out what the I was

[Bill.Thomson]

Hey, nice to see you using the

[borpin (Brian Orpin)]