Mariadb fails after upgrade

Yesterday I did an apt update/upgrade and rpi-update on my emonpi which has been running fine for a year or so. I frequently do this as well as using the update from the emoncms dashboard. No problems before. Everything was up to date from this about a week ago.
I’ve seen some discussion about this but don’t really understand the fixes mentioned. Can someone tell me what I need to do here to get back online?
Here is some error information:

mariadb.service - MariaDB 10.1.38 database server
   Loaded: loaded (/lib/systemd/system/mariadb.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Tue 2019-04-30 19:01:47 EDT; 3min 0s ago
     Docs: man:mysqld(8)
           https://mariadb.com/kb/en/library/systemd/
  Process: 1319 ExecStartPre=/usr/bin/install -m 755 -o mysql -g root -d /var/run/mysqld (code=exited,

Apr 30 19:01:47 emonpiB3 systemd[1]: Starting MariaDB 10.1.38 database server...
Apr 30 19:01:47 emonpiB3 systemd[1]: mariadb.service: Control process exited, code=exited status=1
Apr 30 19:01:47 emonpiB3 systemd[1]: Failed to start MariaDB 10.1.38 database server.
Apr 30 19:01:47 emonpiB3 systemd[1]: mariadb.service: Unit entered failed state.
Apr 30 19:01:47 emonpiB3 systemd[1]: mariadb.service: Failed with result 'exit-code'.

My attempted access to http server:

Can’t connect to database, please verify credentials/configuration in settings.php
Error message: Connection refused

My settings.php:

//1 #### Mysql database settings
    $server   = "127.0.0.1";
    $database = "emoncms";
    $username = "emoncms";
    $password = "emonpiemoncmsmysql2016";
    $port     = "3306";
    // Skip database setup test - set to false once database has been setup.
    $dbtest = false;

Thanks. I’ll be happy to send more information if needed.

Bob
Ashland, VA. USA

Hi Bob, how did you upgrade? Did you do a dist-upgrade?

In the /var/www/emoncms/ folder please do;

git status
git describe

There is another thread here Mariadb cannot connect after dist-upgrade does that help at all?

The solution was in answer 4 Mariadb cannot connect after dist-upgrade - #4 by borpin

ProtectHome=false

[edit]
The correct way to amend the service file is outlined here
https://github.com/emoncms/emoncms/issues/1202

Thanks for the quick reply, Brian. I did apt update/upgrade and not dist-upgrade.
I also changed the true to false.

root@emonpiB3:/var/www/emoncms# git status
On branch stable
Your branch is up-to-date with 'origin/stable'.
Untracked files:
  (use "git add <file>..." to include in what will be committed)

        scripts/logger/logrotate.conf.old
        scripts/logger/logrotate.old
        settings.php.bak

nothing added to commit but untracked files present (use "git add" to track)
root@emonpiB3:/var/www/emoncms# git describe
9.9.4-247-gd0db7a57
root@emonpiB3:/var/www/emoncms#

Bob

Did you restart the service?
[edit]
You might also need a sudo systemctl daemon-reload first.

Yes, did all that plus reboot but still get the error that MariaDB didn’t start.

Same error when systemctl status mariadb.service?

Yes,I did that.

root@emonpiB3:/lib/systemd/system# systemctl restart mariadb.service
Job for mariadb.service failed because the control process exited with error code.
See "systemctl status mariadb.service" and "journalctl -xe" for details.

root@emonpiB3:/lib/systemd/system# systemctl status mariadb.service
● mariadb.service - MariaDB 10.1.38 database server
   Loaded: loaded (/lib/systemd/system/mariadb.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Wed 2019-05-01 11:52:37 EDT; 5min ago
     Docs: man:mysqld(8)
           https://mariadb.com/kb/en/library/systemd/
  Process: 9092 ExecStartPre=/usr/bin/install -m 755 -o mysql -g root -d /var/run/mysqld (code=exited,

May 01 11:52:37 emonpiB3 systemd[1]: Starting MariaDB 10.1.38 database server...
May 01 11:52:37 emonpiB3 systemd[1]: mariadb.service: Control process exited, code=exited status=1
May 01 11:52:37 emonpiB3 systemd[1]: Failed to start MariaDB 10.1.38 database server.
May 01 11:52:37 emonpiB3 systemd[1]: mariadb.service: Unit entered failed state.
May 01 11:52:37 emonpiB3 systemd[1]: mariadb.service: Failed with result 'exit-code'.

Is this a standard emonpi?

Are you on the old SD card? Try output from

 ls -la /boot/em*

root@emonpiB3:/home/pi# ls -la /boot/em*
-rwxr-xr-x 1 root root 1143 Oct 21 2018 /boot/emonSD-30Oct18
root@emonpiB3:/home/pi#

I think the SD card is about a year or less old. What I have is:

House power monitoring system (USA) with the following:

RPI 3 loaded with flashed OEM software on 32 GB SD (latest updates)

emonTXV3 + AC/AC power, RFM69Pi_433 (purchased July 2017)
2 x SCT023R (200A:50MA) on 220 mains (North America 2 x 110)

emonTH V2 (purchased Oct 2017)


weewx weather station (VP2) on Node 0

Ok so the ‘Stretch’ version.

I found a suggestion to delete the file /var/lib/mysql/tc.log and reboot.

Brian,
I have no file by that name:

root@emonpiB3:/home/pi# ll /var/lib/mysql
total 0
-rw-r–r-- 1 root root 0 Apr 30 18:06 debian-10.1.flag
root@emonpiB3:/home/pi#

root@emonpiB3:/home/pi# find / -name tc.log
root@emonpiB3:/home/pi#

Apologies, I checked on the wrongs system.

It will be in /home/pi/data/mysql/

What is in the file /var/log/mysql/error.log

My problem really is that there is nothing in the journal to indicate what has caused it not to load.

Bob, in the status line above, the end of this line is cut off. Could you widen the SSH terminal, restart the service and do a status output again please

Brian,
/var/log/MySQL/error.log is empty

● mariadb.service - MariaDB 10.1.38 database server
   Loaded: loaded (/lib/systemd/system/mariadb.service; enabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/mariadb.service.d
           └─override.conf
   Active: failed (Result: exit-code) since Wed 2019-05-01 15:09:48 EDT; 11s ago
     Docs: man:mysqld(8)
           https://mariadb.com/kb/en/library/systemd/
  Process: 1717 ExecStartPre=/usr/bin/install -m 755 -o mysql -g root -d /var/run/mysqld (code=exited, status=1/FAILURE)

Ah progress.

Check permissions

 ls -la /var/run/my*

Mine is

pi@emonpi:/var/log/mysql $ ls -la /var/run/my*
total 4
drwxr-xr-x  2 mysql root   60 May  1 19:09 .
drwxr-xr-x 29 root  root  960 May  1 15:55 ..
-rw-rw----  1 mysql mysql   6 May  1 19:09 mysqld.pid

[edit]
I note you are running as root. That can upset permissions if you run scripts etc as root rather than as pi and sudo. I wonder if that is the issue. But what particular permissions are wrong will be tricky to discover.

I don’t seem to have a mysqld.pid. Does this mean it’s not running?

Maybe I should load a new image and move the data over? My last backup is a couple of weeks old so I have to lose all the data.