Community
OpenEnergyMonitor

Community

CSV Export Problem

@TrystanLea

I’m running ver 10.1.6 and trying to do a CSV Export …
Meter pulse data from Apr 2019 to Sep 2019 at 1 min intervals - so approx 216,000 data lines
I use phptimeseries with a 10 sec feed interval

But I get the following error message after approx 89,000 downloaded lines …

Fatal error: Maximum execution time of 30 seconds exceeded in /var/www/emoncms/Modules/feed/engine/PHPTimeSeries.php on line 588

I’ve found 3 instances of php.ini at …

/etc/php/7.0/cli/php.ini
/etc/php/7.0/cgi/php.ini
/etc/php/7.0/apache2/php.ini

… and in each changed maximum_execution_time to 240
But still the same 30 secs error report - both before and after a reboot.

If I use a 5 mins interval then no problem.

Any suggestions most welcome
Thx

Hello @johnbanks

not sure why changing the setting in the ini file is not working but I notice I have the equivalent setting in php in the mysqltimeseries engine here: https://github.com/emoncms/emoncms/blob/master/Modules/feed/engine/MysqlTimeSeries.php#L549

You could try placing that line at the start of the export function here:
https://github.com/emoncms/emoncms/blob/master/Modules/feed/engine/PHPTimeSeries.php#L334

if that works Id be happy to include in the code.

@TrystanLea
Thx - I tried your suggestion but it did not work unfortunately.
After a bit more googling and on a hunch …
I inserted set_time_limit(300); as the second line immediately after the <?php in /var/www/emoncms/Modules/feed/engine/PHPTimeSeries.php
Then to ‘stress test’ I tried exporting 10 sec data in Excel time format for a 5 months period …
I got the warning message about the file size - 34 MB
It took 5 mins before Excel opened the csv file - 597K lines - and a fatal error that the 300 sec execution time had been exceeded.

It seems that second line in the php script is the correct place for the set time limit.

Might this also be the case for the ~~~~MysqlTimeSeries.php script?

Incidently, my googling suggested that changing the php.ini file did not work because php.ini was not in the /home directory

Thx

@TrystanLea
Does the above demonstrate the case for changing the code?
Regards

Raise a PR in GitHub proposing the change?

@borpin
Would if I knew how
@TrystanLea
Is this one for you?

Everyday a learning day…

For simple changes, it is easy to do within GitHub itself.

Go to the file page and click on edit
image
Don’t worry about the message.

Make the change and then submit it with a comment as to why. It will ask you to create a ‘Pull Request’. Again it will ask for a comment as to why.

There is nothing as satisfying as seeing one of your suggestions (PRs) being merged i.e. included in the system.