Random Hangs of emonPi/emonSD

Trystan did some tests ages ago: Not all USB power supplies are created the same - Blog | OpenEnergyMonitor

1 Like

Had something similar happen to me because the Pi was running out of disk space.
Something in the EmonCMS UI was triggering the web server to write a lot of errors to its log file which would then automatically get cleaned up destroying the evidence of filling up the disk with error logs.
Maybe check it with “df” and make sure you have plenty of space.
Otherwise might need to change some linux logging settings which is what I did to get it to stop logging errors to disk.

Wish I could remember what it was that was causing all those errors. Maybe some kind of graph object or dashboard object that was referencing a feed that I had deleted or renamed or something like that. I remember having a lot of errors about something like that a while ago.

Mine is just plugged into a USB Socket as part of a 13A wall socket :laughing: It’s rock solid.

Just your luck I think.

This has been a problem in the past, but larger cards mean it is rarely an issue. You can check the available space from the Admin page (which is just a df call IIRC).

On logs, again, a load of work was done to make sure these rotate correctly and saved from the RAMLOG to disk. However, that looks as if it might be broken now.

That is odd. @TrystanLea - this probably needs investigating as it should not be the case…

1 Like

It looks like there is plenty of diskspace. The emonpi has been up for 8 days since the last hang…I’m keeping an eye on it to see when it happens next

I fully agree. My Red Pitayas (which I run as software defined radios) have similar problems with flaky 5V PSUs. When the FPGAs are running full tilt, any weak 5v PSU fails pretty quickly (i.e. within a few minutes). To reduce the SMPS induced rfi for my radios, I am switching to Bel Power HB5-3/OVP-AG linear 5V/3A supplies for the Red Pitayas. That might be overkill for the emonpi though

I will secure another good quality RPi PSU in case it’s the PSU. The emonpi has been running for 8 days since the last crash…ad seems OK at the moment.

Anyway to log low voltage events? My little USB UPS reduces PSU stresses:

Have a look at this thread - it describes a similar situation.
I implemented the reset script and set it to trigger once a day from crontab. The problem seems to have gone.

I experience more or less the same freezes which causes wrong kwh/d processing.
I do use a Raspberry Pi Zero 2 W with emonscripts to install emonCMS.

Today my emonCMS was very slow/laggy but I managed to login and check the admin page
After 4 dats of uptime:

After a reboot:

Could it be that the full swap file is causing this? Or the almost maxed out 512MB RAM from the RPi Z2W?

Does it make sense to increase the swap file?

What would happen if you disable the swap file on a 512MB RAM device? I heard/read that the swap file could reduce the SD lifetime. I use a genuine Sandisk Extreme with A2 rating for app performance.

@TrystanLea Looks like there is a memory leak somewhere.

This is an old PiZ running 10.2.6

@fluppie007 That shouldn’t be happening.Have you installed anything in addition to emoncms on the Pi?

Could you post an htop output sorted on MEM%

No. I think the emonSD setup is such as to ensure the SWAP is only in RAM so as to preserve the SDCard.

htop

Interesting re full swap, not something I’ve seen before.

Thanks for the htop @fluppie007 looks like something is going wrong there with mysql , don’t think there should be so many processes, perhaps just processes piling up due to laggy disk access caused by the swap issue?

What does it look like after a reboot?

Do you have any other software running on the pi zero apart from that installed by emonscripts?

Nope it’s a dedicated emonCMS zero 2 w pi. 64GB Sandisk Extreme A2 micro SD.
16h uptime:

After reboot:

Thanks! I think Im only seeing a single mysql process on mine,
What do you see with:

ps aux | grep mysql

This is what I see (just a single entry):

mysql      665  0.1  6.5 726140 61760 ?        Ssl  Jul22  29:00 /usr/sbin/mysqld

After 22h uptime:

pi@emonrpiz2w:~ $ ps aux | grep mysql
mysql 599 0.1 13.9 725048 61052 ? Ssl Aug01 2:09 /usr/sbin/mysqld
pi 24745 0.0 0.1 7344 580 pts/0 S+ 12:56 0:00 grep --color=auto mysql
pi@emonrpiz2w:~ $

I do have three RPi4’s with HomeAssistant sending some energy information every 5 minutes to emonCMS with the Emoncms History - Home Assistant integration.

But the RPi Z2W does not have anything else than emonCMS running.


Swapfile starts to fill, I expect lock-up in 1-2 days, maybe 3 days.

Note in htop it is the same PID so ‘ps’ will only list one. I see multiple
htop Entries

[edit]
htop shows threads not processes (I think).

Think we are close to a freeze, system gets slower and swap file is filling up.