New emonSD dropping read-only root filesystem requirement

glyn.hudson · 2018-08-20T10:24:19.442Z

After testing and monitoring, we’ve decided to remove the limitation of the root file-system being kept in read-only mode.

Logfiles will still be in ram (/var/log mounted tmpfs) and Emoncms data will still be stored to a special ext2 partition with small block size (mounted in ~/data) however, the root file system will by default be in read-write mode. This helps solve a lot of compatibility issues and allows us to keep the image closer to Raspbian stock which will help with future updates.

We have been monitoring the filesystem to ensure no processing and unnecessarily writing to the SD card, we don’t expect any detrimental SD card wear as a result of this change.

Paul · 2018-08-20T22:45:42.973Z

Great news!
hurray

Paul · 2018-08-20T23:17:39.932Z

glyn.hudson:

…and allows us to keep the image closer to Raspbian stock

Which Raspbian stock image are you intending to use, the full Raspbian Stretch, or (HOPEFULLY NOT) Raspbian Stretch Lite.
Emoncms is not so demanding to need the ‘Lite’ OS, and in the interest of full IoT integration the full Raspbian OS would bring longer term benefits.

glyn.hudson · 2018-08-21T14:13:06.881Z

Paul:

Which Raspbian stock image are you intending to use, the full Raspbian Stretch, or (HOPEFULLY NOT) Raspbian Stretch Lite.

Sorry, we’re using the lite version again. I don’t see the reason to include a full desktop UI on a unit which will be running in headless mode. If required the desktop UI can be installed with:

sudo apt install raspberrypi-ui-mods

Paul · 2018-08-21T14:35:02.626Z

This would be good time to look at Restructure data and settings paths · Issue #795 · emoncms/emoncms · GitHub and try to draw some commonality with self-installs.

TrystanLea · 2018-08-21T15:29:16.832Z

@Paul trying to find the original forum thread on the topic of #795, can you remind us?

Paul · 2018-08-21T15:32:34.065Z

Was it Move data & settings file to a common location?

danbates · 2018-08-21T18:10:02.255Z

Will this fix the trait of the file system returning to read only after a set time?

Paul · 2018-08-21T18:25:02.805Z

danbates:

Will this fix the trait of the file system returning to read only after a set time?

Yes, because the code required to put the file system in RO will be removed.
It will avoid a lot of problems that are being reported in the forum, so a good move.

danbates · 2018-08-21T18:35:13.000Z

Weheeeeey! Yes!

Thank god.

In the short time I’ve been going in deep, I can definitely see this a positive step forwards.

sudo nano … !!!

borpin · 2018-08-22T09:50:05.628Z

glyn.hudson:

Paul:

Which Raspbian stock image are you intending to use, the full Raspbian Stretch, or (HOPEFULLY NOT) Raspbian Stretch Lite.

Sorry, we’re using the lite version again

Any thoughts on using DietPi as the base image? With a good set of install instructions it would then be available for a far wider set of boards including VMs. The automated install of the associated systems (such as node-red) is really good.

The install it currently has is actually the emonhub bit rather than a full emoncms install.

I’m also using lighttpd rather than Apache with absolutely no issues.

Putting both together there is a significantly reduced processor load.

Paul · 2018-08-22T10:36:48.728Z

borpin:

Any thoughts on using DietPi as the base image?

That would move the emonpi even further away from self builds, instead of pulling them closer together, and also increase support demands as it’s not a mainstream OS.

danbates · 2018-08-22T11:01:28.000Z

Regarding file structure. I’m in favour of making something standard along Linux lines as has been mentioned… I felt the desire for this as I was getting to grips with it.

On the emonSD download page… A weblink to a page showing the emoncms, emonHub, settings, and logs folder structure would be useful for intrepid users.

A minor point… You know the folder ‘usefulscripts’? Aren’t the scripts contained therein useful for something by definition?

borpin · 2018-08-22T12:25:16.253Z

Paul:

borpin:

Any thoughts on using DietPi as the base image?

That would move the emonpi even further away from self builds, instead of pulling them closer together, and also increase support demands as it’s not a mainstream OS.

Not really. It is a Debian based OS package. You still need to generate a base system with a webserver stack be it with Apache or Lighttpd; mariadb or sqlite, REDIS, MQTT, node-red etc. The installation of these via DietPi is remarkably simple and repeatable for many different base systems and boards (I think they install mqtt and redis via PECL for instance). Trying to maintain instructions for different (and disparate) base systems is difficult - providing a base system that can be used wider that the Pi systems is probably easier .

Once you have a base system, it is then a case of overlaying emoncms onto that.

For the specific use case of a RaspberryPi (or compatible - I use an Orange Pi Zero) with an RFM module, you need the emonhub installation - this actually can be independent of emoncms install.

Finally you get to the emonpi which is again a specific use case.

The advantage of using DietPi as the base is that you effectively outsource, underlying update to systems such as node-red, PHP etc which currently lie fallow once a base image has been done.

There is also a builtin backup system for dietpi plus the ability to move the data partition off to a HD plus a whole host of other utilities.

You should still be able to provide a prebuilt card, but then supporting a wider base may well be easier.