Permanent "Raspberry Pi Booting..." after failed emonPi Update

Emoncms
1

Hi There,
I tried to update my EmonPi as usual by EmonPiUpdate button on emocms aAdmin Menu. Update didn’t get through as my image is probaly too old:

Starting update via service-runner-update.sh (v2.0) >
Filesystem is unlocked - Write access
type ’ rpi-ro ’ to lock

…since then, the display on my EmonPI is permanently lit with “Raspberry Pi Booting…” message. Everthing else works fine, the display is just stuck there.

Shall I reflash my SD with a fresh image?

Thanks

Cris

image
image1296×504 27.3 KB

2

TBH Cris, backing up your data and flashing a new image would be the best route although this error should not happen. @glyn.hudson @TrystanLea any suggestions?

I’d suggest using a new SD card so you at least have this one to go back to. Be patient on the first boot.

3

For some reason the “sate-list” of updatable images changed recently from
emonSD-07Nov16
emonSD-03May16
emonSD-26Oct17
emonSD-13Jun18
emonSD-30Oct18

to just

emonSD-30Oct18
emonSD-26Oct17

so I expect you to be just the first of many unhappy campers.

There has been no announcement to this effect and I can see no real reason for this except maybe only those 2 images have been tested with the new update routine and the list is currently incomplete. Some time ago Glyn made a commitment to always support updating the current images moving forward so it would be quite peculiar to now only support 2 images, if you are going to drop support, you may as well start with a fresh image and update procedure rather than struggling to support just 2 previous images, that were previously similar in update requirements to those now dropped.

If you really want to update, i would suggest backing up your data and changing your emonSD version file (by editing to a later version to fool the check) temporarily to try the updater, if it doesn’t work, then you can try a new image.

1 Like
4

At sometime you need to drop older images, but I agree there should be an announcement, the backup facility should be installed anyway (in some form - an easy to find script in the home folder perhaps) and even then (without an update), it should not get stuck like this.

1 Like
5

I don’t disagree, but support should be planned and announced and more importantly “make sense”.

Again I don’t disagree, the problem is that at the time the update button was pressed it (obviously) wasn’t running the latest software. The update script that was initiated was most likely

The current image would pass the then valid list of supported images and stopped the emonpilcd service whilst it pulled in all the new repo’s, the first of which is the emonpi repo.

Then much later in that script there are commands such as

/home/pi/emonpi/rfm69piupdate.sh
/home/pi/emonpi/emonpiupdate
/home/pi/emonpi/emonhubupdate
/home/pi/emonpi/emoncmsupdate

but none of these paths are valid anymore due to the reshuffle, so the script will fail before it gets to the point where the emonpilcd is restarted again.

The image is now in no mans land since it has pulled in the latest repos but it has not completed any of the sub-scripts to implement required changes. But subsequent presses of the update button will not allow an update to start because the previously executed script has now been replaced with one that no longer supports the older images. A reboot might even be catastrophic, but even if the system does come back on line it will not be properly updated, nor will it be as it was before, it will be orphaned and in an unknown incomplete state.

This is what used to happen to the 2015 images and it’s exactly the position these checks are intended to prevent, but in this instance it has caused the problem as it has allowed the update to be initiated but will not allow it to complete purely due to the change in “safe-list” if the safe list mirrored the old safe-list it would not of happened. Editing the image version to fool the updater is probably the safest way out, aside from abandoning the image.

If we are going to drop older images, and roll out massive changes to the way images are built and updated. It would be better to start afresh, not support just one old RO Jessie image and one old (incomplete) RW Stretch image, that just seems random and will cause difficulties for the updater scripts moving forward, it would be better to avoid including backward compatibility that might hinder progress, for just 2 images that will rapidly be out of date with the new image builder, even the “image version checks” will be out of date once an image can be built on the fly.

1 Like
6

Paul,
I tried to fool the check as you sugested by TTY this:
sudo mv /boot/emonSD-07Nov16 /boot/emonSD-26Oct17
After that the update went through successfully!

In your opinion, should I now revert the to the original emonSD version by
sudo mv /boot/emonSD-26Oct17 /boot/emonSD-07Nov16
?

Thanks a bunch!

1 Like
7

I’d load a fresh image on a new card having backed up the data.

Is the backup module installed? If not you can clone it from here GitHub - emoncms/backup: emoncms backup module. No need to fully set it up, just copy modify the config file to suit, then run the export script from the command line.

You can then simply import the data to the new image.

Best done this way so if anything goes wrong, you can try again from the original setup.

8

Will do once I’ll get a fresh 8Gb SD card.
As far as backup is concerned, I’d use the built in function in the web interface

image

Is that what you mean by backup module?

9

Yes. Just use that.

10

Good question! I don’t think it matters much right now, perhaps change it back just in case you forget it’s been done? But that means future updates will not work as your image is “unsupported” again, but it’s easy enough to deliberately change it if you need to update.

It will be interesting to hear what @glyn.hudson and @TrystanLea say on the matter. If it is just an incomplete “tested with” list, then perhaps the fact you have effectively tested the new update on the emonSD-07Nov16 image might mean they can add it to the repo version and next time you try an update, it will qualify even without fudging the version.

11

My EmonPI is on emonSD-07Nov16, and I am not sure what I should do now. Leave it as is, or update it in some specific sequence? As a layman, I am a bit lost with this. Not even sure where the SD card is. Any tips are much appreciated.

12

Hi Juerg, do you have SSH access to the emonpi? and are you ok with running a couple of commands from the command line? If so I can walk you through doing a update as Cristiano as done last week. the alternative is to wait and see if/when this gets addressed, I would of expected that to have happen already this past week or at least commented on by now, so I can’t say when that might be.

Since the 07Nov2016 image is not that different to the 26Oct17 image (ref image changelog) I’m unsure why it’s not included in the new safe list, but Cristiano seems to have had no problem (once the incomplete update was resolved again) so until I’m told otherwise I believe it to be safe.

First we have to change the version file so the updater thinks this is a later image, this can be done via the commandline without physically accessing the sdcard using this command

sudo mv /boot/emonSD-07Nov16 /boot/emonSD-26Oct17

then to avoid the issues Cristiano had due to the old updater script already having been triggered before all the file paths change, we do a manual update to just that repo so it should update ok first time.

git -C /home/pi/emonpi pull
[ $? -eq 0 ] && echo "EMONPI REPO UPDATED OK!" || echo "EMONPI REPO NOT UPDATED!"

The last line of the output should spell out if the repo was updated ok in capitals (to make it stand out).

(If for any reason this didn’t succeed then post the whole output here for us to check)

If it was “OK!” then run the “update all” from within emoncms and give it plenty of time to complete, you can watch the progress in the updater log window, when it is complete, post a copy of the updateemonpi.log here and I will take a look to see if anything looks iffy.

once it has been successfully updated we should probably put the image version back to what it was just to be safe

sudo mv /boot/emonSD-26Oct17 /boot/emonSD-07Nov16

but don’t do this last step until we know it’s ok just incase we need to run the updater again.

13

I’ve opened a couple of issue trackers on this to hopefully catch someone’s attention soon.

14

@CrisPastore are you happy that the update completed ok on the emonSD-07Nov16 image? It sounds like we should have added this image to the safe list and that I missed it by only testing emonSD-26Oct17. Im happy to add it in, I think @pb66 is right that the changes are minimal and should not affect a successful update.

I noticed in the discussion above a reference to broken paths and just wanted to clarify how the new update process works.

emonpi/emoncmsupdate is retained as a redirect so that the older images can access the newer update process, see: emonpi/emoncmsupdate at master · openenergymonitor/emonpi · GitHub

As part of the work on the new update process I also fixed the safe check, which was not previously working, it now makes a curl request to check the list of safe images: emonpi/service-runner-update.sh at master · openenergymonitor/emonpi · GitHub before running git pull on the emonpi repository here: emonpi/service-runner-update.sh at master · openenergymonitor/emonpi · GitHub

Unfortunately the first time the update process is ran on older images this check gets incorrectly passed as it is still the old implementation of the check, the emonpi repository is then updated. The rest of the update however does not continue, as the now redirected emoncmsupdate performs the safe check correctly. This avoids issues with a full emonpi update on an old image that is not safe to update but does mean the emonpi repo is updated which includes the emonPiLCD scripts - which is probably the reason your LCD showed booting.

Perhaps if someone else gets to the same point again it would be interesting to know if restarting the EmonPi gets rid of the Booting message and whether the rest of the system is otherwise not updated. I will try a test of this in the office next week.

15

@TrystanLea yes, so far I’m totally happy with my update on the emonSD-07Nov16 image.
My doubt now is whether should I take the time to flash a newer image or be worried that something similar will happen again on my next update.

Should you reccomend to switch to a newer image, there’s a way to get this done without disassembling my unit (by TTY perhaps)?

16

Good to hear that your happy that its working. I would hold off switching to the latest image for now. We are working on a completely new image based on an automated installer script, I think it would be better to switch to that once it is finished than go to the effort to switch twice, assuming your happy with your emonSD-07Nov16 updated system in the meantime.

17

Hi Paul,

Many thanks for the instructions. I should be able to apply that using my SSH. I hope to find some time tomorrow for this.

First though, it appears I will have to do these steps every time in the future before attempting to run ‘update all’ from within emoncms, correct?

If correct, it makes we wonder whether I perhaps should consider not updating anymore at all, and leave everything forever at the current state? After all, it does what it needs to, and I don’t seem to need new features or bug fixes. What are the drawbacks of not updating anymore?

Many thanks for sharing your insights.

18

Hello @Juerg1 we are looking into adding emonSD-07Nov16 to the safe to update list, so you should be able to update without the SSH changes. Most updates are feature or bug fix updates but there have also been security updates over the last couple of years that would be worth updating to keep your system secure (this is the main one I can think of: Important Emoncms Security Update from Nov 17).

1 Like
19

Thank you @TrystanLea, great news about planned updates to safe-list. I will happily wait till then.

Good to know about the security updates too. Of course, they are important, and a good reason to stay up to date.

For now, I will monitor progress on this post, and update my EmonPi when updates are again easy to do.

Many thanks to @Paul for the workaround too!

1 Like
20

Hello @Juerg1 Nov16 has now been added to the safe list, full details here:
https://community.openenergymonitor.org/t/upgrading-emonsd-07nov16-to-latest-via-emonpi-emonbase-update/11051

1 Like