MySQL Error: The partition with /home/pi/data/mysql is too full! ... failed!

emonSD
1

Today I downloaded the emonSD-07Nov16 image and booted it up on my Pi. I ran emonSDexpand, since I’m using a 32 GB SD. I was not able to log into the emoncms local site, getting the error:
Can’t connect to database, please verify credentials/configuration in settings.php
Error message: Can’t connect to local MySQL server through socket ‘/var/run/mysqld/mysqld.sock’ (2)

pi@emonpi(ro):~$ sudo systemctl status mysql.service -l
● mysql.service - LSB: Start and stop the mysql database server daemon
   Loaded: loaded (/etc/init.d/mysql)
   Active: failed (Result: exit-code) since Mon 2016-11-07 13:17:07 UTC; 10min ago
  Process: 838 ExecStart=/etc/init.d/mysql start (code=exited, status=1/FAILURE)

Nov 07 13:17:05 emonpi mysql[838]: df: '/home/pi/data/mysql/.': No such file or directory
Nov 07 13:17:05 emonpi mysql[838]: /etc/init.d/mysql: ERROR: The partition with /home/pi/data/mysql is too full! ... failed!
Nov 07 13:17:07 emonpi systemd[1]: mysql.service: control process exited, code=exited status=1
Nov 07 13:17:07 emonpi systemd[1]: Failed to start LSB: Start and stop the mysql database server daemon.
Nov 07 13:17:07 emonpi systemd[1]: Unit mysql.service entered failed state.

Any thoughts on how to proceed? Seems like it should be easier to get going from the SD image.
Thanks!
Brandon

2

The last question in the FAQ list should have the answer you’re looking for.

3

Thanks. I did try that and it doesn’t change the errors I get. Here is what things look like now:

pi@emonpi(ro):~$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/root       3.4G  1.9G  1.4G  59% /
devtmpfs        483M     0  483M   0% /dev
tmpfs           487M     0  487M   0% /dev/shm
tmpfs           487M  6.6M  480M   2% /run
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           487M     0  487M   0% /sys/fs/cgroup
tmpfs           1.0M  4.0K 1020K   1% /var/lib/dhcpcd5
tmpfs            40M  6.6M   34M  17% /var/lib/openhab
/dev/mmcblk0p1   60M   21M   40M  35% /boot
tmpfs            50M  896K   50M   2% /var/log
tmpfs           1.0M     0  1.0M   0% /var/lib/dhcp
tmpfs            30M   36K   30M   1% /tmp

And:

Command (m for help): p
Disk /dev/mmcblk0: 29 GiB, 31104958464 bytes, 60751872 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x0003ea3e

Device         Boot   Start      End  Sectors  Size Id Type
/dev/mmcblk0p1 *       8192   131071   122880   60M  6 FAT16
/dev/mmcblk0p2       131072  7288831  7157760  3.4G 83 Linux
/dev/mmcblk0p3      7288832 60751871 53463040 25.5G 83 Linux

Any ideas what to do next?

4

It looks like the data partition isn’t mounted.

can you post the output of these commands

sudo blkid

sudo mount

you could also try mounting it manually with

sudo mount /dev/mmcblk0p3 /home/pi/data

and post any objections raise if it doesn’t work, but it should mount automatically at boot unless you have edited /etc/fstab, should look like this

pi@emonpi(ro):~$ sudo cat /etc/fstab
tmpfs           /tmp            tmpfs   nodev,nosuid,size=30M,mode=1777 0  0
tmpfs           /var/log        tmpfs   nodev,nosuid,size=50M,mode=1777 0  0
tmpfs           /var/lib/dhcp   tmpfs   nodev,nosuid,size=1M,mode=1777 0  0
tmpfs           /var/lib/dhcpcd5 tmpfs nodev,nosuid,size=1M,mode=1777 0  0
tmpfs           /var/lib/openhab tmpfs  nodev,nosuid,size=40M,mode=1777 0  0
proc            /proc           proc    defaults 0 0
/dev/mmcblk0p1  /boot           vfat    defaults,noatime,nodiratime 0 2
/dev/mmcblk0p2  /               ext4    defaults,ro,noatime,nodiratime,errors=remount-ro 0 1
/dev/mmcblk0p3  /home/pi/data   ext2    defaults,rw,noatime,nodiratime,errors=remount-ro 0 2

have you rebooted since expanding?

5

I ran the two commands. Here’s the output. It didn’t seem to help.

pi@emonpi(ro):~$ sudo blkid
/dev/mmcblk0: PTUUID="0003ea3e" PTTYPE="dos"
/dev/mmcblk0p1: SEC_TYPE="msdos" LABEL="boot" UUID="7771-B0BB" TYPE="vfat" PARTUUID="0003ea3e-01"
/dev/mmcblk0p2: UUID="c7f58a52-6b71-4cea-9338-65f3b8af27bf" TYPE="ext4" PARTUUID="0003ea3e-02"
/dev/mmcblk0p3: UUID="36d54a99-6668-4459-8f5d-21d79314002a" TYPE="ext2" PARTUUID="0003ea3e-03"
pi@emonpi(ro):~$ sudo mount
/dev/mmcblk0p2 on / type ext4 (ro,noatime,nodiratime,errors=remount-ro,data=ordered)
devtmpfs on /dev type devtmpfs (rw,relatime,size=493896k,nr_inodes=123474,mode=755)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/lib/systemd/systemd-cgroups-agent,name=systemd)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/net_cls type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=300,minproto=5,maxproto=5,direct)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
mqueue on /dev/mqueue type mqueue (rw,relatime)
configfs on /sys/kernel/config type configfs (rw,relatime)
tmpfs on /var/lib/dhcpcd5 type tmpfs (rw,nosuid,nodev,relatime,size=1024k)
tmpfs on /var/lib/openhab type tmpfs (rw,nosuid,nodev,relatime,size=40960k)
/dev/mmcblk0p1 on /boot type vfat (rw,noatime,nodiratime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,errors=remount-ro)
tmpfs on /var/log type tmpfs (rw,nosuid,nodev,relatime,size=51200k)
tmpfs on /var/lib/dhcp type tmpfs (rw,nosuid,nodev,relatime,size=1024k)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,relatime,size=30720k)

I also tried the command to manually mount it. No objections were raised. But it also didn’t seem to get things working.

pi@emonpi(ro):~$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/root       3.4G  1.9G  1.4G  59% /
devtmpfs        483M     0  483M   0% /dev
tmpfs           487M     0  487M   0% /dev/shm
tmpfs           487M   56M  432M  12% /run
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           487M     0  487M   0% /sys/fs/cgroup
tmpfs           1.0M  4.0K 1020K   1% /var/lib/dhcpcd5
tmpfs            40M  6.6M   34M  17% /var/lib/openhab
/dev/mmcblk0p1   60M   21M   40M  35% /boot
tmpfs            50M  8.5M   42M  17% /var/log
tmpfs           1.0M     0  1.0M   0% /var/lib/dhcp
tmpfs            30M   36K   30M   1% /tmp
/dev/mmcblk0p3   25G   39M   24G   1% /home/pi/data
6

The first commands were to get info rather than do something to fix it,

They show the /data partition mounted, which is confirmed in the 2nd df -h output as the last line wasn’t present in the 1st df -h (or was that a cut and paste error?). If not were these 2 commends run after you tried to manually mount /data?

As the command to mount the data partition should have responded like so

pi@emonpi(ro):~$ sudo mount /dev/mmcblk0p3 /home/pi/data
mount: /dev/mmcblk0p3 is already mounted or /home/pi/data busy
       /dev/mmcblk0p3 is already mounted on /home/pi/data

if it wasn’t previously mounted, if there was no output that would mean it was silently successful in mounting a previously unmounted partition. If that is the case I wonder why it wasn’t mounted, can you confirm your fstab is the same as I posted (or post yours) and if the Pi has been rebooted since you expanded the SDcard?

Manually mounting the /data partition will not get things working by itself as the services that require access to the /data partition will have already tried and failed. It was a test to see if it was possible to mount the partition or if something was objecting. we would need to work out why the partition isn’t mounted at boot to fix it.

7

It wasn’t a cut and paste error the /dev/mmcblk0p3 partition is there after running the command to manually mount it, and not before.

I got no output, so it seems that it did silently successfully mount it.

[quote=“pb66, post:6, topic:4123”]
If that is the case I wonder why it wasn’t mounted, can you confirm your fstab is the same as I posted (or post yours) and if the Pi has been rebooted since you expanded the SDcard?
[/quote]

I have definitely rebooted since expanding. It is one of the steps listed in the FAQ that Bill pointed me to. I’ve rebooted several times, as I’ve tried various things.

My fstab file looks like this:

sudo cat /etc/fstab
tmpfs           /tmp            tmpfs   nodev,nosuid,size=30M,mode=1777 0  0
tmpfs           /var/log        tmpfs   nodev,nosuid,size=50M,mode=1777 0  0
tmpfs           /var/lib/dhcp   tmpfs   nodev,nosuid,size=1M,mode=1777 0  0
tmpfs           /var/lib/dhcpcd5 tmpfs nodev,nosuid,size=1M,mode=1777 0  0
tmpfs           /var/lib/openhab tmpfs  nodev,nosuid,size=40M,mode=1777 0  0
proc            /proc           proc    defaults 0 0
/dev/mmcblk0p1  /boot           vfat    defaults,noatime,nodiratime 0 2
/dev/mmcblk0p2  /               ext4    defaults,ro,noatime,nodiratime,errors=remount-ro 0 1
#/dev/mmcblk0p3  /home/pi/data   ext2    defaults,rw,noatime,nodiratime,errors=remount-ro 0 2

Note the last line is commented out. Not sure why. I’m going to uncomment it, reboot and see what happens.

8

I think that will resolve the issue. Any idea how it became commented out?

9

Uncommenting #/dev/mmcblk0p3 in /etc/fstab worked. When I rebooted all the services started up as expected.

I have no idea how that line would have become commented out. I’m curious why others haven’t experienced this, because I really just took a clean copy of the emonSD from November, booted it up, and ran emonSDexpand, just like in the instructions. Then ran the further instructions to manually expand the partition that Bill pointed out in the FAQ.

Is it worth doing it again to see if I can re-create the problem?

10

After running the expand, maybe the sdpart_imagefile script did not finish and remove the #. If I read the script correctly, the # gets added in one line and removed in another line.

11

If I’m reading the script correctly, and I may well not be, one of the last things that happen before rebooting, is the line gets commented out.

107 # Make the resize2fs_once script executable
108 chmod +x /home/pi/resize2fs_once.sh
109 # Ensure that the /data partition is unmounted after a reboot
110 sed -i 's/\/dev\/mmcblk0p3/#\/dev\/mmcblk0p3/' /etc/fstab
111 echo
112 echo
113 echo ======================================================================
114 echo
115 echo So far, so good... in 20s your system will reboot 
116 echo and resize the /home/pi/data filesystem to fill the resized partition.
117 echo **THIS OPERATION WILL TAKE UP TO 20 MINUTES**
118 echo When completed the Pi will poweroff and close down.
119 echo PLEASE DO NOT UNPLUG THE POWER FOR AT LEAST 30min
120 echo because it will likely result in an unusable filesystem.
121 echo
122 echo ======================================================================
123 sleep 20
124 /bin/sync
125 /sbin/reboot
126 fi
127 ###################################################################
128 # END OF SCRIPT
129 ###################################################################
12

Thank you, everyone. This is almost definitely a case of user error. As I look at that script, I am pretty sure that when the script finished, and gave me back a prompt, that I typed the command to restart the Pi. My assumption was that it would give me some kind of countdown if it was still in a mode where it was going to restart itself. So I guessed it had finished in much less time than the estimated 20 minutes, but had failed to restart itself and needed me to do that.

So, I thank everyone for the help and apologize for the trouble.

13

Brandon - When the script finishes (after 20+ minutes), it does a shutdown of the Raspberry Pi / emonPi. So if you got a prompt, then something did not finish correctly.

 

EDIT: from the df -h above it looks like the resize happened OK. If you type in cat /home/pi/resize2fs_once.sh you should see an error:

pi@emonpi(ro):~$ cat /home/pi/resize2fs_once.sh
cat: /home/pi/resize2fs_once.sh: No such file or directory