emonPi update oddness: no update

Hello! I just tried to run an emonPi update and things didn’t work as hoped. After click the emonPi Update (on Setup > Administration webpage) the log from 4 days ago went flying by in the Update log window. After waiting a few minutes I did a reboot.

The emonpi update seems to have happen before the emonPi reboot. And since there are LOTS of errors in the emonpiupdate.log (enclosed below) I tried to do another emonpi update. But the same issue (the log from an hour ago went flying by in the Update log window).

I’m hoping if I can get the emonPi update to run then the emonpiupdate.log errors might disappear…

48%20PM%20copy%202

2018-06-18 at 16.14.42 - emonpiupdate.log.txt (45.3 KB)

EDIT: added server info

Server Information
Emoncms Version low-write 9.8.30 : 2018.05.08
Modules Administration : App v1.1.0 : Backup v1.1.2 : EmonHub Config v1.0.0 : Dashboard v1.3.0 : EventProcesses : Feed : Graph v1.2.0 : Input : postprocess : CoreProcess : Schedule : setup : sync : Time : User : Visualisation : WiFi v1.1.1
Buffer loading…
Writer Daemon is running with sleep 60s
Server OS Linux 4.9.35-v7+
Host emonpi emonpi (127.0.1.1)
Date 2018-06-18 17:12:47 CDT
Uptime 17:12:47 up 59 min, 0 users, load average: 0.20, 0.30, 0.43
HTTP Server Apache/2.4.10 (Raspbian) HTTP/1.1 CGI/1.1 80
MySQL Version 5.5.60-0+deb8u1
Host localhost (127.0.0.1)
Date 2018-06-18 17:12:47 (UTC -05:00‌​)
Stats Uptime: 3535 Threads: 3 Questions: 10306 Slow queries: 0 Opens: 59 Flush tables: 1 Open tables: 52 Queries per second avg: 2.915
Redis Version 2.8.17
Host localhost:6379 (127.0.0.1)
Size 635 keys (618.53K)
Uptime 0 days
MQTT Version
Host localhost:1883 (127.0.0.1)
Pi CPU Temp 36.86°C
Release emonSD-07Nov16
File-system Set root file-system temporarily to read-write, (default read-only)
Memory RAM Used: 30.38% Total: 970.93 MB Used: 294.93 MB Free: 676.01 MB
Disk Mount Stats
/ Used: 77.08% Total: 3.33 GB Used: 2.57 GB Free: 616.38 MB
/boot Used: 36.35% Total: 59.95 MB Used: 21.79 MB Free: 38.16 MB
/home/pi/data Used: 34.23% Total: 3.75 GB Used: 1.29 GB Free: 2.28 GB
/mnt/usb Used: 29.20% Total: 57.81 GB Used: 16.88 GB Free: 37.99 GB
PHP Version 5.6.33-0+deb8u1 (Zend Version 2.6.0)
Modules apache2handler : bcmath : bz2 : calendar : Core v5.6.33-0+deb8u1 : ctype : curl : date v5.6.33-0+deb8u1 : dba : dio v0.0.4RC4 : dom v20031129 : ereg : exif v1.4 : fileinfo v1.0.5 : filter v0.11.0 : ftp : gettext : hash v1.0 : iconv : json v1.3.6 : libxml : mbstring : mcrypt : mhash : mosquitto v0.3.0 : mysql v1.0 : mysqli v0.1 : openssl : pcre : PDO v1.0.4dev : pdo_mysql v1.0.2 : Phar v2.0.2 : posix : readline v5.6.33-0+deb8u1 : redis v2.2.7 : Reflection : session : shmop : SimpleXML v0.1 : soap : sockets : SPL v0.2 : standard v5.6.33-0+deb8u1 : sysvmsg : sysvsem : sysvshm : tokenizer v0.1 : wddx : xml : xmlreader v0.1 : xmlwriter v0.1 : Zend OPcache v7.0.6-devFE : zip v1.12.5 : zlib v2.0 :

Client Information
HTTP Browser Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/11.1.1 Safari/605.1.15
Screen Resolution 2560 x 1440
Window Size 1296 x 1240

EDIT 2: I ran this to force an update.

I still have lots of errors related to emonpi2c

rm: cannot remove ‘hardware/emonpi/emonpi2c/emonpi2c_V1.2/emonPi2C_V1.2_BOM.ods’: Permission
denied

And I cannot get the emopi Update to run via the webpage. ugh!

to correct these errors:

rm: cannot remove ‘hardware/emonpi/emonpi2c/emonpi2c_V1.2/emonPi2C_V1.2_BOM.ods’: Permission denied
rm: cannot remove ‘hardware/emonpi/emonpi2c/emonpi2c_V1.2/emonPi2C_V1.2_GERBERS/emonPi2C_V1.2.GTO’: Permission denied
rm: cannot remove ‘hardware/emonpi/emonpi2c/emonpi2c_V1.2/emonPi2C_V1.2_GERBERS/emonPi2C_V1.2.TXT’: Permission denied
rm: cannot remove ‘hardware/emonpi/emonpi2c/emonpi2c_V1.2/emonPi2C_V1.2_GERBERS/emonPi2C_V1.2.GBS’: Permission denied
rm: cannot remove ‘hardware/emonpi/emonpi2c/emonpi2c_V1.2/emonPi2C_V1.2_GERBERS/emonPi2C_V1.2.GTL’: Permission denied
rm: cannot remove ‘hardware/emonpi/emonpi2c/emonpi2c_V1.2/emonPi2C_V1.2_GERBERS/emonPi2C_V1.2.gpi’: Permission denied
rm: cannot remove ‘hardware/emonpi/emonpi2c/emonpi2c_V1.2/emonPi2C_V1.2_GERBERS/emonPi2C_V1.2.GBO’: Permission denied
rm: cannot remove ‘hardware/emonpi/emonpi2c/emonpi2c_V1.2/emonPi2C_V1.2_GERBERS/emonPi2C_V1.2.dri’: Permission denied
rm: cannot remove ‘hardware/emonpi/emonpi2c/emonpi2c_V1.2/emonPi2C_V1.2_GERBERS/emonPi2C_V1.2.GTS’: Permission denied
rm: cannot remove ‘hardware/emonpi/emonpi2c/emonpi2c_V1.2/emonPi2C_V1.2_GERBERS/emonPi2C_V1.2.GBL’: Permission denied
rm: cannot remove ‘hardware/emonpi/emonpi2c/emonpi2c_V1.2/emonPi2C_V1.2_GERBERS/emonPi2C_V1.2.GTP’: Permission denied
rm: cannot remove ‘hardware/emonpi/emonpi2c/emonpi2c_V1.2/emonPi2C_V1.2.brd’: Permission denied
rm: cannot remove ‘hardware/emonpi/emonpi2c/emonpi2c_V1.2/emonPi2C_V1.2.sch’: Permission denied
rm: cannot remove ‘hardware/emonpi/emonpi2c/emonpi2c_V1.1/emonPi2C_V1.1_Gerbers.zip’: Permission denied
rm: cannot remove ‘hardware/emonpi/emonpi2c/emonpi2c_V1.1/emonPi2C_V1.1.sch’: Permission denied
rm: cannot remove ‘hardware/emonpi/emonpi2c/emonpi2c_V1.1/Eagle_7.3_gerber_export.cam.txt’: Permission denied
rm: cannot remove ‘hardware/emonpi/emonpi2c/emonpi2c_V1.1/emonPi2C_V1.1.brd’: Permission denied
rm: cannot remove ‘hardware/emonpi/emonpi2c/emonpi2c_dev/emonpi2c.sch’: Permission denied
rm: cannot remove ‘hardware/emonpi/emonpi2c/emonpi2c_dev/emonpi2c.brd’: Permission denied

 

this line needs to have root permissions:

Hi @Jon, looking at your log (among other things) it looks like your file system was in read-only mode for the larger latter portion and therefore many commands failed and those that didn’t will not have stuck.

The update script doesn’t call rpi-rw very often so once it’s gone RO it’s unlikely to recover, and even if it did, then there could be missing bit go un-noticed or partial updates that cause issues. When I wrote an install/update script fora RO emonSD image just before the launch of the emonPi, I wrapped each command in a function that checked the FS status and called rpi-rw (if needed) at every step but that approach is not used in the emonSD so this sort of thing can happen at anytime.

The FS can be mounted RO at anytime due to a user command (rpi-ro) in another console/window or indeed another script of some description running in the background that has the same ability as the update script to lock the FS when it’s done. The way I did it, the status of the FS immediately before the wrapped command was called determined if the FS got locked or not on completion, so there was no chance of one process/script locking down the FS while another was still in progress.

There is also the “auto-lock” function that runs hourly and mounts the FS read-only, if you are really unlucky you could call the update seconds before that cron task runs and locks the FS again before the update completes, in this instance though I think the node-red update was just so long that it eventually timed out, that has been reported before here on the forum.

One way to “dodge” the node-red update (for test or to split the updates into smaller chunks) might be to use “update emonbase” rather than “update emonpi” as I previously noticed only emonpi users get the node-red update, it’s not part of the emonBase update routine. So you could run the update twice (once emonBase and then emonPi) to get the full update.

Or you could look at the /etc/crontab file and determine when your cron-hourly is run and work around that knowing the FS will re lock each hour at that time (eg 17mins past the hour in the following example) or you could temporarily disable the cron job.

# /etc/crontab: system-wide crontab
# Unlike any other crontab you don't have to run the `crontab'
# command to install the new version when you edit this file
# and files in /etc/cron.d. These files also have username fields,
# that none of the other crontabs do.

SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

# m h dom mon dow user  command
17 *    * * *   root    cd / && run-parts --report /etc/cron.hourly
25 6    * * *   root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
47 6    * * 7   root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly )
52 6    1 * *   root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly )
#

You could also keep an eye on the /var/log/service-runner.log file (or something close to that name) for errors as not all the update commands output goes to the update log.

[edit]

…and if you have not successfully updated since 7th June then there is the added possibility that some but not all of the changes made to switch the service-runner to using a redis queue may have stuck but not others if it bombed out part way though, potentially leaving you with a broken service-runner/updater. I only say this based on your above comment, I have not looked closely at how it’s done to know if it’s definitely the case.

sorry for the late reply! my bad! I got the first set of errors fixed with the sudo command.

The second set of errors was as you said. my emonPi went into the read-only mode part way thru the emonPi update. I got two of those types of errors in a row (odd coincidence?).

I looked at the service-runner and all looked like a correct copy of the service runner on GitHub. But I do have a recent problem with redis.

I still haven’t figured out why I cannot get the emopi Update to run via the webpage. Too many odd issues! No worries, I’ll figure it out someday!