Community
OpenEnergyMonitor

OpenEnergyMonitor Community

Emoncms web interface now takes 10s to respond

Hi
I’ve been using emoncms for a few months now and had it connected to home assistant for a single dashboard of all the data.
Now, for some reason the web requests to emoncms are taking 10s (curl -I https://emoncms.mydomain.com - this is not the real one!) , so HA is complaining (it has a timeout of 5s for its emoncms calls).
The system is a Pi3+, ubuntu 20.04 with an external SSD, with HA & emoncms running in docker, using an external mysql database for emoncms. There’s an nginx reverse proxy in front of docker.
I’ve rebuilt the docker container to the stable branch, so that is showing 10.7.0 (it was using 10.5.8 until I noticed the problem).
Comparatively, HA flies along, so it’s defo something in the emoncms container, but I’m not sure where I should look.
It may also help to say the emoncms dashboards load really, really slowly - as in about 10s per graph (of 8 on one screen).
Can anyone help?
My other option is running the container on a real machine with more grunt, but then again it did work fine when I first installed it.

Thanks,
Sean

Hello @Gadg3ts are you using mysql for feed data? or are your feeds listed as using the PHPFina engine? Does this affect all pages or only pages that need to load feed data?

for mysql I don’t know. What’s the default?
and the feeds page shows them all as phpfina yes.
but also the admin page takes ages.

Phpfina is fine, great. It’s been a while since Ive done any docker stuff, can you check to see if there are any processes using a lot of memory or processor? Can’t think of anything specific that would have slowed things down and I haven’t noticed any slow down here on my emonSD system running on a Pi3.

I would recommend using our emonSD builds over docker as the emoncms docker isn’t really maintained unfortunately.

It’s a docker image built locally from from the ‘stable’ branch, because it’s not the only thing running on that Pi - the other being HA.

Server Information

Server Information

Services

Emoncms

Server

  • OS :- Linux 5.4.0-1038-raspi
  • Host :- fb3ab21d0e94 | fb3ab21d0e94 | (172.19.0.4)
  • Date :- 2021-07-22 01:07:14 BST
  • Uptime :- 00:07:14 up 14:05, 0 users, load average: 0.50, 0.53, 0.54

Memory

  • RAM :- Used: 69.84%
    • Total :- 852.87 MB
    • Used :- 595.61 MB
    • Free :- 257.26 MB
      Write Load Period

Disk

  • / :- Used: 19.25%
    • Total :- 108.13 GB
    • Used :- 20.81 GB
    • Free :- 87.3 GB
    • Write Load :- n/a
  • /dev/shm :- Used: 0.00%
    • Total :- 64 MB
    • Used :- 0 B
    • Free :- 64 MB
    • Write Load :- n/a
  • /etc/hosts :- Used: 19.25%
    • Total :- 108.13 GB
    • Used :- 20.81 GB
    • Free :- 87.3 GB
    • Write Load :- n/a

HTTP

  • Server :- Apache/2.4.38 (Debian) HTTP/1.1 CGI/1.1 80

MySQL

  • Version :- 5.7.30-0ubuntu0.18.04.1
  • Host :- mysql (10.0.0.32)
  • Date :- 2021-07-22 00:07:13 (UTC 00:00‌​)
  • Stats :- Uptime: 76773288 Threads: 5 Questions: 121149446 Slow queries: 0 Opens: 48314 Flush tables: 1 Open tables: 2000 Queries per second avg: 1.578

Redis

  • Version :-
    • Redis Server :- 5.0.5
    • PHP Redis :- 5.3.4
  • Host :- redis:6379
  • Size :- 43 keys (804.17K)
  • Uptime :- 0 days

MQTT Server

  • Version :- Mosquitto ?
  • Host :- house:1883 (10.0.0.121)

PHP

  • Version :- 7.3.9 (Zend Version 3.3.9)
  • Modules :- apache2handlerCore ctype curl date dom v20031129fileinfo filter ftp gettext hash iconv json v1.7.0libxml mbstring mosquitto v0.4.0mysqli mysqlnd vmysqlnd 5.0.12-dev - 20150407 - $Id: 7cc7cc96e675f6d72e5cf0f267f48e167c2abb23 $openssl pcre PDO pdo_sqlite Phar posix redis v5.3.4Reflection session SimpleXML sodium SPL sqlite3 standard tokenizer xml xmlreader xmlwriter zlib
Client Information

Client Information

HTTP

  • Browser :- Mozilla/5.0 (Windows NT 10.0; ) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4086.0 Safari/537.36
  • Language :- en-US,en;q=0.9,en-GB;q=0.8

Window

  • Size :- 2040 x 1224

Screen

  • Resolution :- 1920 x 1080

Thanks @Gadg3ts I cant see anything wrong with that. I’ve started up the emoncms-docker container here and it seems to be pretty responsive, through of course that’s without data and lots of things posting to it.

I notice that there was a fix merged for a mysql issue causing long wait times last August, Mysql stall fix by kronenpj · Pull Request #33 · emoncms/emoncms-docker · GitHub do you have the latest container?

I’m not sure that mysql fix is relevant - it looks like that’s for initialising the db? (which was already done as was running fine for several months).
The docker image build is based on a posting I found here (can’t find the original link), which basically takes the ‘official’ docker image as a base and then clones the current ‘stable’ branches into it.
Dockerfile.txt (1.1 KB)
docker-compose.yml.txt (1.0 KB)
The above are what I’m using right now.
So it’s the latest stable code…
I have a suspicion it’s not apache itself that’s being slow, but that I can debug. I was just hoping for an easy answer! :wink:

Might be worth trying the docker container here? GitHub - emoncms/emoncms-docker: Emoncms docker container(s) but looks similar to what you have there. Is there a chance the external mysql server is causing an issue, perhaps latency between the two?

Perhaps someone else here can suggest something else to try.

That link with the build script is actually where I started. :wink:
To get it usable again, I might just try building the container on a proper machine and see what that gives me. Then look at profiling the components to see where the bottleneck is.

I could be way off base here, but does your Admin / System Info indicate the RAM use is growing or any Swap memory is being used?

My emonPi system experienced interface slow-downs and they were related to growing RAM use that led to growing Swap use. For me, the problem was a buggered up emonhub.conf file in /etc/emonhub/. I found 3 sets of duplicate node definitions in the file. I deleted the duplicates and my system had been stable and responsive ever since.

To extend my answer, I rsync’ed the build folder (ok, the docker & compose files) to another machine (Intel cpu) and rebuilt the docker image, with the same result. As in it was really slow .Which is weird.
So for the moment I’ve got the EmonTX data working directly in home Assistant via MQTT sensors (which actually wasn’t that hard).