Feedwriter crashes constantly after upgrading from 9.8.x to 9.9.x

Emoncms
1

Hi,

feedwriter.php did hang tonight and after some debugging i found the following:

Processing feed=183 len=3 :
 Reading block 50000
 Invoking post_bulk_save engine=2
Processing feed=184 len=3 :
 Reading block 50000
 Invoking post_bulk_save engine=2
Processing feed=185 len=3 :
 Reading block 50000
 Invoking post_bulk_save engine=2
Processing feed=214 len=54 :
 Reading block 50000
 Invoking post_bulk_save engine=2
Processing feed=216 len=682 :
 Reading block 50000

mmap() failed: [12] Cannot allocate memory

mmap() failed: [12] Cannot allocate memory
PHP Fatal error:  Out of memory (allocated 538968064) (tried to allocate 534773768 bytes) in /var/www/emoncms/Modules/feed/engine/PHPFina.php on line 711

Fatal error: Out of memory (allocated 538968064) (tried to allocate 534773768 bytes) in /var/www/emoncms/Modules/feed/engine/PHPFina.php on line 711

If i restart the whole system or only the service it will break again.

Any idea? Should i open a GitHub Issue?

2

Hello @Chrischi, do you have any feeds that have not been posted to for a long time? PHPFina is trying to fill in a long period of missing data with null values and its running out of memory. Its trying to allocate 510 mb of memory which suggests perhaps a feed interval of 1s or a start time perhaps of 1970…

3

yes! it crashed tonight and tried to fill 12h of data afterwards but after a reboot and loosing the remaining data, it continues crashing with fresh data.

4

Does it always crash at this point?:

How many feeds do you have total?

5

Good Catch! It was one of the few feeds with the PHPFINA Engine.

I deleted feed 216 and it seems to work again.

I have 158 Feeds. Is there a easy way to auto. count the feeds?