Upgrading from low-write V9.8.30 to latest - anything to watch out for?

Tags: #<Tag:0x00007f87b3f66610>

I’m running EMONCMS V9.8.30 on 64Bit Debian and using MQTT exclusively for inputs. Its been pretty reliable but I think I should bring it up to date.
Are there any breaking changes or general bits of linux mischief I need to look out for? Should I update to an interim version along the way or just move to the latest? Do I simply need to change to the EMONCMS folder and issue a GiT Pull or is there more to do? My setup is based on the Pi low write version with the feeds stored on a separate /home/pi/data partition

What is the base Debian version? Jessie can cause some issues.

What feed engines are you using? IIRC I needed to convert my data to a different engine at some point in time but I’m not sure what version that was.

There have been quite a lot of changes since then. Do you have much in the way of dashboards? Whilst the update script tries to accommodate the changes, I’d be inclined to try for a fresh install.

I’m guessing this is not RPi based. I’m wondering, if you have a spare RPi, you could use it as a stepping stone; set it up, copy the data across and see if it will run. As you are using MQTT, running 2 along side each other should not be an issue. Once that is running, you can blat the original and do a fresh install there and then simply use the export/import functionality to move the data back.

Make sure you have a good backup!!!

No there is quite a bit more involved than that. Additional modules need updating as does the way the services work & get started.

I’m running Debian GNU/Linux 9 (stretch) which will get updated as part of this on a Mini PC. I have one Dashboard plus the MySolar App.
I’m using phpfina, phptimeseries and sql for my ram feeds plus MQTT Publish to feed some things to OpenHAB. The admin page says I have the following modules

Administration | app | Backup v1.1.2 | Dashboard v1.1.1 | EventProcesses | Feed | Input | CoreProcess | Schedule | Time | User | Visualisation

As I have a very extensive OpenHAB system on the same box doing a simple format and reload is likely to be very time consuming although I will image the hard disk first in case it screws up. I do have a spare Pi2 that I could play with if needs be, although I’d need to go through my inputs to get rid of those that publish to MQTT before it would run in parallel

Is there any documentation I can look at before I start? I slipped on quite a few banana skins before I got a stable workable system and I am keen not to do it again.

That should be OK

No real issues there.



My immediate thoughts for potential banana skins are;

  • A simple git pull will not work.
  • You will need the Device, config, and Graph modules installed (not part of the core).
  • The way the services are run has changed from init.d to systemd
  • There have been issues reported on the Mosquitto version. In the past Emoncms used a beta version, the ‘newer’ beta version might be a problem. Depending on how you set this up, your installation may still be inclined to pull that one down.
  • The MQTT input and publish mechanism has changed.

Other thoughts on how you are setup -

  • Do you use REDIS & Feedwriter (used to be referred to as ‘low-write’)?
  • Is service-runner service running?
  • Are the logs on tempfs?

@pb66 and @TrystanLea - do you have any further thoughts?

Have a look at the update scripts that are normally used plus the newer install scripts There is also an sql backup script here.

TBH there are lots of potential pitfalls but as long as the data is backedup and you can live with a bit of a gap in your data, you should be fine.

I did something similar a number of years back, let myself get behind the releases, and it is a bit of a PITA to then update.

I do use REDIS & Feedwriter, service-runner is running and the logs are on tempfs.
Do you know if the backup scripts will remember everything including all the input setup details? It seems like I’d be better off deleting it all and starting again after changing the box over to buster. I had to do a lot of messing with SQL before it’d work last time around when I built it on stretch, and as I’m running from a SSD I’d like to keep the low write stuff.
So far my SSD seems to be wearing out at around 1% per year which is good considering what I have running on it

OK, we’ve been looking at using a different approach to managing the logs on tempfs as there have been a few issues.

It should do :grin: I have moved data a number of times on my Pi-Hole but that was largely between the same version number. Coming from your version might cause some issues.

I do think that starting from scratch may well be easier in the long run. I would though, be inclined to use the backup and try and import it onto a new SD card on a separate system to prove the concept (of the export/import) and to have a running system while you rebuild the main system.

I’ve moved away from running everything on one machine for just this sort of reason. Now, most of my stuff (Emoncms, Node-red, MQTT, Pi-Hole, and HomeAssistant) all run on separate VMs under EXSI on an HP Microserver. It means I can snapshot and rapidly restore any of them and the only interruption is any data coming from them. Has made my life so much simpler than worrying about interdependencies on a single machine.

Thanks for the info, I’ll take a full image backup when I have a spare few hours and start experimenting. Experience tells me to not go near this when I’m short of time!
One final question, will my current version likely survive an OS update to Buster?

Short answer - probably, but that is the best I can do. Take the backup first!

Thanks very much!
I’ll do this in stages when I have spare time and update this thread so it helps anyone else following on

1 Like