Can't update 10.1.6 - fleeting php errors

Emoncms
1

Hi,

I’ve tried updating my Linux server installs on EmonCMS from 10.1.6 to the newest version. Unfortunately I get a very fleeting (200ms or so) message, shown here:

Screenshot%20from%202019-10-28%2008-58-04
Screenshot%20from%202019-10-28%2008-58-041294×185 16.7 KB

It looks to be something in admin_controller.php:232.

Fatal error: Uncaught Error: Call to a member function rpush() on boolean in /var/www/html/emoncms/Modules/admin/admin_controller.php:232
Stack trace:
#0 /var/html/www/emoncms/core.php(60): admin_controller()
#1 /var/html/www/emoncms/index.php(215): controller('admin')
#2 {main}
   thrown in /var/www/html/emoncms/Modules/admin/admin_controller.php on line 232

Line 232 of admin_controller.php is:

$redis->rpush("service-runner","$update_script $type $firmware>$update_logfile");

If I comment it out the update does nothing.

This problem is affecting my own personal server (Ubuntu Server 16.04 LTS) and my shared hosting server. PHP versions are 7.0.33 (local) and 7.1.32 (hosted).

My EmonPI updated quite happily to 10.1.10, but I think it was on 10.1.6beta.

I have tried installing 10.1.10 from git and migrating data, but I am getting missing feed metadata errors, such as with graph. “Request error: Error reading meta data feedid=49”

I am also unable to run update on 10.1.10:

Screenshot%20from%202019-10-28%2010-36-26
Screenshot%20from%202019-10-28%2010-36-261265×354 31.3 KB

This time the line with the error in admin_controller.php is 240:

$redis->rpush("service-runner","$update_script $type $firmware>$update_logfile");

I see a pattern here! I don’t have redis running, as I got a core dump when following the pecl build instructions, and my server has a hard drive, so no write wear prolems.

Any suggestions welcome.

2

Read properly (that is to me BTW)!!! Update will not work without Redis from the admin page.

Are the services Redis & service-runner running? You can see this on the admin page under ‘Server’

Could you try and install Redis again? The system more and more relies on it.

[edit]
Redis on the remote host may be more problematic of course but should work on the local server.

Have a look at the new install script and pick out the commands you need to try again (local).

Other than that it is possible to update manually.

How did you install EmonCMS?

3

Hi Brian,

I originally installed EmonCMS on my local server and the shared host by unpacking the ZIP file. I used GIT for the 10.1.10 install. I’m not sure I will be able to install redis on the shared host, and I’ll have another crack at getting all the modules installed. redis-server is installed via apt, but obviously there is more required. If redis is now required then I guess I’m stuck on 10.1.6. I hope there are really good reasons for this and it isn’t a minor convenience.

Having another go … using steps at emoncms/LinuxInstall.md at master · emoncms/emoncms · GitHub

Got this far:

    Installing '/usr/lib/php/20151012/redis.so'
    install ok: channel://pecl.php.net/redis-5.0.2
    configuration option "php_ini" is not set to php.ini location
    You should add "extension=redis.so" to php.in

For some reason, even though pecl install redis 5.0.2 the version that is running is 3.0.6. My RPI is running 3.2.6, so hopefully 3.0.6 is good enough. I’ve tried a ‘apt remove redis*’ and it looks like it wants to remove some other software that I run on the computer (ntop). Dependency hell!

I’ve added the “extension=redis.so” to /etc/php/7.0/mods-available/redis.ini and restarted apache. EmonCMS at least runs now, and shows redis running on the Admin status page.

When I try and do an update all I get is a quick flash of:
service-runner trigger sent

Have now removed ntopng and redis from apt, and am installing redis from source with instructions from How To Install and Configure Redis on Ubuntu 16.04 | DigitalOcean

A check of systemctl shows

systemctl status service-runner.service
    ● service-runner.service
       Loaded: not-found (Reason: No such file or directory)
       Active: inactive (dead)

Oh dear. That is going to slow things down isn’t it?

Have followed the instructions at emoncms/install-service-runner-update.md at master · emoncms/emoncms · GitHub
and get

   XXX@YYY:/$ systemctl status service-runner.service
    ● service-runner.service - Emoncms service-runner Input Script
       Loaded: loaded (/var/www/html/emoncms/scripts/services/service-runner/service-runner.service; enabled; vendor preset: enabled)
      Drop-In: /etc/systemd/system/service-runner.service.d
               └─override.conf
       Active: activating (auto-restart) (Result: exit-code) since Mon 2019-10-28 19:56:40 AEST; 18s ago
         Docs: https://github.com/emoncms/emoncms/blob/master/scripts/services/install-service-runner-update.md
      Process: 27310 ExecStart=/usr/bin/python /var/www/emoncms/scripts/services/service-runner/service-runner.py (code=exited, status=2)
     Main PID: 27310 (code=exited, status=2)

Oct 28 19:56:40 YYY systemd[1]: service-runner.service: Unit entered failed state.
Oct 28 19:56:40 YYY systemd[1]: service-runner.service: Failed with result 'exit-code'.

journalctl -f -u service-runner yields:
Oct 28 19:58:41 YYY service-runner[27343]: /usr/bin/python: can't open file '/var/www/emoncms/scripts/services/service-runner/service-runner.py': [Errno 2] No such file or directory
No surprise, as my install is in /var/www/html/emoncms. The hardcoding of locations is breaking things.

A quick tweak to service-runner.service to give me:

[Service]
Type=idle
ExecStart=/usr/bin/python /var/www/html/emoncms/scripts/services/service-runner/service-runner.py
User=

And now I am getting a python error:
ImportError: No module named redis

‘pip install redis’ gives me:
Requirement already satisfied: redis in /usr/local/lib/python3.5/dist-packages (3.3.11)

I downloaded the redis python source installed in manually with sudo python setup.py install

Now the service-runner service is all good!

On the admin page of EmonCMS I get:
service-runner Active Running
redis-server Active Running

Now to try the update again …
And the error is
/bin/sh: 1: /opt/openenergymonitor/emonpi/service-runner-update.sh: not found
This seems to be an emonpi thing, and the contents at emonpi/service-runner-update.sh at master · openenergymonitor/emonpi · GitHub don’t make a lot of sense to me.

I know this is a very long winded post, but I thought it might help someone else that is trying to get the service-runner and redis running on a standalone server. I’m learning more about services, python modules etc than I planned on!

David.
[edited to put code blocks in]

4

If you are adding in code, please use 3 ‘backticks’ on a line before and 3 on a line after to format it.

It isn’t required, but a number of things don’t work without it like MQTT, backup and update.

Ah multiple problems - all of which have been fixed by the new install scripts - there are instructions for an Ubuntu install - the other manual install docs are being retired.

The problem is the service-runner install docs need updating - it needs more in the drop-in now (as the service has changed) as you have found.

There is a difference between sudo pip and pip; both do an install but put the package globally or locally.

Has the update ever worked or is this a recent install?

5

Hi, yes the updater has worked before. I think it was 10.1.1 or something similar when I started using it and then it updated to 10.1.6beta. The 14th of September is the earliest data I have on the server and on the Pi, so the first version would have been what was around in the week or so before that.

I did try ‘sudo pip’ as well as ‘pip’, which normally fixes things.

After a reboot my redis is not working in EmonCMS any more, but it does work from the command line.

I have looked into the backup module too, and followed the instructions, but no luck. I’ve pointed a new install of 10.1.10 to the /var/lib/phpfiwa etc dir and DB, and it can see the incoming data, but it has the metadata problem (e.g. Request error: Error reading meta data feedid=51)

If I backup my ‘/var/lib/phpfiwa/’ and ‘’/var/lib/phpfina/’ directories & do a DB dump is there a straightforward way of reinstating the data into a new install? Things have got so broken that I think it is time to reinstall the server, and move to Ubuntu 18.04 (which I see you mention in the scripts). Fortunately I have a spare HDD so I can experiment and alternate if I miss something.

David.

6

Have you the backup module installed? You should be able to just run the emoncms-export.sh script. You may need to tweak the config file paths.

You can then import from the UI (if redis is installed).

7

Yes, I’ve installed the module and made the symlinks.

I’ve done the install.sh and run the export with emoncms-export.sh , and get these errors:

=== Emoncms export start ===
Mon 28 Oct 22:35:45 AEST 2019
Backup module version:
    "version"      : "2.1.6"
EUID: 1000
Reading ./config.cfg....
Location of databases: /var/lib
Location of emonhub.conf: 
Location of Emoncms: /var/www/html/emoncms
Backup destination: /home/XXX/emon
emoncms backup module location /var/www/html/emoncms/Modules/backup
Failed to stop feedwriter.service: Unit feedwriter.service not loaded.
PHP Notice:  Undefined variable: settings in /home/XXX/emon/modules/backup/get_emoncms_mysql_auth.php on line 31
PHP Notice:  Undefined variable: settings in /home/XXX/emon/modules/backup/get_emoncms_mysql_auth.php on line 31
PHP Notice:  Undefined variable: settings in /home/XXX/emon/modules/backup/get_emoncms_mysql_auth.php on line 31
Error: Cannot read MYSQL authentication details from Emoncms settings.php
Failed to start feedwriter.service: Unit feedwriter.service not found.

Line 31 of the get_emoncms_mysql_auth.php is
echo $settings['sql']['username'].":".$settings['sql']['password'].":".$settings['sql']['database'];

I’ll have to carry on with this tomorrow, but I can see what emoncms-export.sh is trying to do, so I’ll see if I can replicate it manually and feed the data into a clean build of EmonCMS.

David.

8

Look here LCD Screen says "Updating - Do Not Unplug" (updating system from V8.5) - #8 by borpin

You need to roll back to a previous version of the backup module.

9

Thanks – that was spot on.
I’ll now do a clean install using the scripts you’ve mentioned on a shiny new 18.04 build and bring that backup in.

1 Like
10

SUCCESS!!! [yes, just a little excited]

The scripts worked, after I fixed up my mistakes and got rid of the RPi settings properly. A few services needed to be manually installed and the low-write mode disabled.

I didn’t realise that the EmonPi would keep track of the data that was uploaded and then back fill – VERY IMPRESSED, but it was too big a backlog with redisbuffering enabled.

Anyway, the system runs updates, new data is collected.

Big thanks to Brian for your perseverance. Got there in the end, and would have take a lot longer without the pointers on the fixes. Hopefully everything will tick along now that redis etc is up and running.

David.

1 Like
11

Ah, that shouldn’t happen. Can you share what and the config.ini file you used please.

I’ve run the scripts on LTS and it worked fine.

Could you offer some improvements to the docs to help with this? i.e. what did you miss?

There are some PRs in train that might have solved this.

12

Hi Brian,

Here is my config file, with extra comments added just now using ‘//’ markers (to make it clear these weren’t in the config.ini that I actually used).
config.zip (1.1 KB)

The services I had to install manually, as they were showing up as red dots in the Admin page were:

./scripts/services/install-service-feedwriter.md
./scripts/services/install-service-runner-update.md
./scripts/services/install-service-emoncmsmqtt.md

I had to manually install the backup and demandshaper modules, but I think that was intended.

Where my confusion came from is in the differences of config between SD and non-SD.

  • Extra services to be disabled
  • Relevance of user name and host name.

Github didn’t clone initially, so I ran the emoncms_core installer script.
Also, I customised my mysql password, and it didn’t propagate through to settings.ini, so that required a manual change.

As for suggestions to docs, I have these:

  • Separate install scripts for EmonSD and for non-Pi versions.
  • I couldn’t find how to disable low-write mode in the docs, but by grepping ‘low-write’ I saw where the version flag was looking and tweaked settings.ini accordingly.
  • I found a page last night (but can’t now) that detailed the Ubuntu changes that were needed (disabling things), but the changes were at the end of the page and not the beginning. This detail would be good to have in the main scripts MD file.

I can’t recall much more than that I’m afraid, and my bash history review has gotten me the info above.

David.

13

Can’t see these in this file.

Did you run this command?

sudo echo $USER' ALL=(ALL) NOPASSWD: ALL' | sudo tee /etc/sudoers.d/$USER && sudo chmod 0440 /etc/sudoers.d/$USER

Should not have needed to do that.

No again should be automatic.

Yes, I can see that is an issue. We should improve the comment or change the term I think.

So that is a local issue to you but will affect the rest of the scripts.

I think we will include sample config files.

I have a PR in place to install services based on modules installed.

Yes user name is important! Hostname less so.

There isn’t any advantage to doing so - I just leave it be.

That may have been an old post of mine that was part of the testing plus I did an updated post. I agree, this needs pulling into the main instructions.

I’ve suggest to @TrystanLea that we need to create an install log. It may have allowed us to identify what went wrong.

I have also realised that the stable branch is behind the master branch for the scripts.

1 Like
14

Hi Brian,

Yes, I set sudo to work without a password.

Git works on the server, and things worked fine when I run the emoncms_core script by hand. Perhaps it failed because the user name in the config was wrong.

There were only a couple of comments in the config. Just to show where I got confused.

Separate install Configs and a log would be a big help. The modular scripts were a good way of recovering.

The level of automation you’ve achieved is very good, and managing the dependencies makes the install much easier. Hard to deal with everything that dopey users do though :slight_smile:

David.

15

That will not have helped. It is likely the services failed for the same reason.

We have moved the user and hostname section to the top of the config file in the master branch, plus added some extra stuff around not installing Pi specific things.