Web Interface Inaccessible

[maximundo (Max Frimond)]

I have an emonpi that has been working perfectly since I bought it a few months ago. All of a sudden the web interface is very slow or not accessible at all.

I can SSH into the unit and run top, the output is below, but when I do Ctrl + C the putty session hangs.

Any ideas?

top - 21:48:11 up 19 min,  1 user,  load average: 0.00, 0.08, 0.12
Tasks: 114 total,   1 running, 113 sleeping,   0 stopped,   0 zombie
%Cpu(s):  2.4 us,  0.8 sy,  0.0 ni, 96.7 id,  0.0 wa,  0.0 hi,  0.1 si,  0.0 st
MiB Mem :    975.6 total,    568.0 free,    162.4 used,    245.2 buff/cache
MiB Swap:    100.0 total,    100.0 free,      0.0 used.    712.2 avail Mem

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
  364 emonhub   20   0   54412  18744   7336 S   6.0   1.9   1:00.51 python3
  539 redis     20   0   45640   4192   2896 S   2.3   0.4   0:17.76 redis-server
  634 root      20   0   80500  22316  17304 S   2.0   2.2   0:11.33 php
  493 root      20   0   61248  17460   6664 S   1.0   1.7   0:13.86 python
  561 pi        20   0   17912  10696   6200 S   1.0   1.1   0:11.61 python
  983 pi        20   0   10328   2884   2448 R   1.0   0.3   0:00.07 top
  962 www-data  20   0  213844  16376  11248 S   0.7   1.6   0:00.16 apache2
   10 root      20   0       0      0      0 I   0.0   0.0   0:01.77 rcu_sched
   11 root      20   0       0      0      0 I   0.0   0.0   0:00.00 rcu_bh
   12 root      rt   0       0      0      0 S   0.0   0.0   0:00.00 migration/0
   13 root      20   0       0      0      0 S   0.0   0.0   0:00.00 cpuhp/0
   14 root      20   0       0      0      0 S   0.0   0.0   0:00.00 cpuhp/1
   15 root      rt   0       0      0      0 S   0.0   0.0   0:00.00 migration/1
   16 root      20   0       0      0      0 S   0.0   0.0   0:00.06 ksoftirqd/1
   19 root      20   0       0      0      0 S   0.0   0.0   0:00.00 cpuhp/2
   20 root      rt   0       0      0      0 S   0.0   0.0   0:00.00 migration/2
   21 root      20   0       0      0      0 S   0.0   0.0   0:00.07 ksoftirqd/2
   23 root       0 -20       0      0      0 I   0.0   0.0   0:00.12 kworker/2:0H-kblockd
   24 root      20   0       0      0      0 S   0.0   0.0   0:00.00 cpuhp/3
   25 root      rt   0       0      0      0 S   0.0   0.0   0:00.00 migration/3
   26 root      20   0       0      0      0 S   0.0   0.0   0:00.08 ksoftirqd/3
   29 root      20   0       0      0      0 S   0.0   0.0   0:00.00 kdevtmpfs
   30 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 netns
   31 root      20   0       0      0      0 I   0.0   0.0   0:00.81 kworker/0:1-events
   32 root      20   0       0      0      0 I   0.0   0.0   0:00.15 kworker/1:1-events
   33 root      20   0       0      0      0 I   0.0   0.0   0:00.12 kworker/2:1-memcg_kmem_cache
   34 root      20   0       0      0      0 S   0.0   0.0   0:00.00 khungtaskd
   35 root      20   0       0      0      0 S   0.0   0.0   0:00.00 oom_reaper
   36 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 writeback
   37 root      20   0       0      0      0 S   0.0   0.0   0:00.00 kcompactd0
   38 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 crypto
   39 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 kblockd
   40 root      rt   0       0      0      0 S   0.0   0.0   0:00.00 watchdogd
   41 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 rpciod
   42 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 kworker/u9:0
   43 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 xprtiod
   46 root      20   0       0      0      0 S   0.0   0.0   0:00.00 kswapd0
   47 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 nfsiod
   58 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 kthrotld
   60 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 iscsi_eh
   61 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 dwc_otg
   62 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 DWC Notificatio
   63 root       1 -19       0      0      0 S   0.0   0.0   0:00.00 vchiq-slot/0
   64 root       1 -19       0      0      0 S   0.0   0.0   0:00.00 vchiq-recy/0
   65 root       0 -20       0      0      0 S   0.0   0.0   0:00.00 vchiq-sync/0
   66 root      20   0       0      0      0 S   0.0   0.0   0:00.00 vchiq-keep/0
   67 root      20   0       0      0      0 I   0.0   0.0   0:00.05 kworker/3:2-events
   68 root     -51   0       0      0      0 S   0.0   0.0   0:00.82 irq/86-mmc1
   71 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 mmc_complete
   72 root       0 -20       0      0      0 I   0.0   0.0   0:00.10 kworker/3:1H-kblockd
   73 root       0 -20       0      0      0 I   0.0   0.0   0:00.24 kworker/0:1H-mmc_complete
   74 root       0 -20       0      0      0 I   0.0   0.0   0:00.20 kworker/1:1H-kblockd
   75 root      20   0       0      0      0 S   0.0   0.0   0:00.00 jbd2/mmcblk0p2-
   76 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 ext4-rsv-conver
   78 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 ipv6_addrconf

[borpin (Brian Orpin)]

Try

df -h

Search for ‘/var/log’ filling up here.

[maximundo (Max Frimond)]

Disk space was my first thought but it’s got plenty remaining.

It seems to be working this morning. I’ll keep an eye on it.

[maximundo (Max Frimond)]

It is still very slow however. Where should I start to diagnose this?

[borpin (Brian Orpin)]

@borpin

Try

df -h

Search for ‘/var/log’ filling up here.

Please paste the output.

[maximundo (Max Frimond)]

Here you go:

pi@emonpi:~ $ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/root       4.0G  2.1G  1.7G  56% /
devtmpfs        484M     0  484M   0% /dev
tmpfs           488M     0  488M   0% /dev/shm
tmpfs           488M  6.5M  482M   2% /run
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           488M     0  488M   0% /sys/fs/cgrouptmpfs           1.0M     0  1.0M   0% /var/tmp
tmpfs            30M     0   30M   0% /tmp
tmpfs           1.0M     0  1.0M   0% /var/lib/php/sessions
/dev/mmcblk0p3   10G   53M  9.5G   1% /var/opt/emoncms
/dev/mmcblk0p1  253M   52M  201M  21% /boot
log2ram          50M   38M   13M  75% /var/log
tmpfs            98M     0   98M   0% /run/user/1000

[borpin (Brian Orpin)]

@maximundo

log2ram 50M 38M 13M 75% /var/log

This looks rather full. follow this

System working fine for months, all of a sudden, emonhub won't start

The issue is /var/log/ is full. You need to remove the big files. However before you do, please run this command and post it so we can see what the issue was. du -a /var/log | sort -n -r | head -n 20 To make the files smaller (deleting them can be a bad idea) use truncate -s 0 /var/log/<log file>

[TrystanLea]

@maximundo

It is still very slow however. Where should I start to diagnose this?

Given that your load average looks plenty low enough and the SD card is not full. Another possibility is some kind of sd card corruption after a power cut affecting another part of the system. I appreciate that its a pain to do, but if nothing else comes to mind you could try the “Upgrade to a new SD card” section here Update & Upgrade — OpenEnergyMonitor 0.0.1 documentation

[maximundo (Max Frimond)]

Thanks @TrystanLea but we’ve not had a power cut. The unit is not that old.
I’ve tested the performance of the SD card and it’s very quick. It is a very strange issue.

[TrystanLea]

Thanks @maximundo Im not sure how to solve unfortunately, we’d be happy to send a replacement SD card for you to try, if you email us at [email protected] we can get that sorted for you.

[maximundo (Max Frimond)]

Thanks for the offer @TrystanLea but no need. I’ve found the problem to be the Wi-Fi connection. It must have been a coincidence that when I was checking the Wi-Fi it was useable but slow. This evening it didn’t work at all. It has been fine for weeks which I’ve changed nothing!

Resorted to using a HomePlug and ethernet cable. Its now quick as a flash!

Thanks

[TrystanLea]

Great to hear @maximundo