emonSD next steps

Discussing with @glyn.hudson here it looks like the /usr directory may not be the best location for the installation.


/usr is the second major section of the filesystem. /usr is shareable, read-only data

Large software packages must not use a direct subdirectory under the /usr hierarchy.

It looks like /usr/local is best suited for applications that fit the /usr/local/bin, /usr/local/src /usr/local/etc hierarchy.

/opt/ looks like a better install location /opt/emon/


/opt is reserved for the installation of add-on application software packages.

A package to be installed in /opt must locate its static files in a separate /opt/<package> or /opt/<provider> directory tree, where <package> is a name that describes the software package and <provider> is the provider’s LANANA registered name.

The structure of the directories below /opt/<provider> is left up to the packager of the software, though it is recommended that packages are installed in /opt/<provider>/<package> and follow a similar structure to the guidelines for /opt/package.

any views?

we should be able to make this flexible, set in the installation configuration file, so its not an urgent question.

Sort of, but it still ends up with a hard coded path.

The install/update script I have seen (and cannot remember what now), simply wrote the required text to a new file (having checked and deleted any existing file) direct to the required systemd folder (/lib/systemd/system) - no symlink required. Any paths can then be generated in the script before writing the text.

No views.

If you remember the Pauls (@pb66 @Paul) had a discussion suggesting a data structure that accounted for multiple installations. This seemed to only cover the data though not the other bits.

1 Like

Thanks @borpin I like the ‘paths can be generated in the script’ approach. I have this now working for both the emonhub and demandshaper services, e.g: https://github.com/emoncms/demandshaper/blob/master/install.sh#L20 - the install.sh script in the demandshaper module is called from emoncms_modules.sh

Thanks for the link, that discussion did cross my mind when thinking about the /var/lib/phpfina etc locations. I will have a good read through again.

That works. What I had seen was actually redirecting an echo output as the required file.

I would add a comment in though :smile:

1 Like

Comment added :slight_smile:

I’ve moved much of the emonhub installation process to the emonhub repo itself, replacing the existing install scripts that where emonpi specific. emonhub/install.sh at emon-pi · openenergymonitor/emonhub · GitHub
This script is then called from a much shorter script in the original installer directory.

Getting there.

1 Like

Great. I raised a specific question there.

At one point there were different emonhub versions for the emonpi and other setups. Has this all been merged into one now?

[edit] Found the question I asked a while ago

1 Like

FWIW, that’s an obsolete version of the spec - fifteen years old. The current version is Filesystem Hierarchy Standard

/opt/package-name for code and move data directories to /var/opt/package-name for data after installation are reasonable places for external products to be placed. If a distro packages a product and releases it, then it will typically put it under /usr with data under /var/lib
So the package should make sure it uses relative, and configurable paths to enable such changes. Ditto for its dependencies on other packages, which may live in different places on different systems. That also allows for multiple versions if the version number is in the path somewhere.


Thanks @djh that is useful to know, looks like we are on the right track then moving to /opt. I will also move the data directories to /var/opt/emon/phpfina etc (configurable of course) thanks for the tip on that. Any objections?

I still need to look the idea of supporting multiple emoncms installations on one machine as suggested by @pb66 as part of this.

1 Like

I suspect that will simply be a need to configure the initial path layout so it is extensible.

Perhaps just offer a commented out alternative structure for a multi install environment as it is an edge case.

I think that may well be the case, should be straightforward.

1 Like

A little more progress on this.

  • There is now a wifiAP install script, with the access point tested and working.
  • Default paths changed and configurable to: /opt/emon and emoncms data: /var/opt/emon/phpfina…
  • changes to emonPiLCD script that make it path flexible merged into master

I’ve run the build script again from the start on a fresh raspbian strech and it works well (a final sudo reboot is required at the end).

Im going to review the remaining steps in post 92 above next, but will need to come back to it after attending to a number of other things first.


Noting your comment here @borpin for inclusion Unexpected output from git describe - #11 by borpin

1 Like

Further progress on the install script:

I have made a start on documenting the install scripts here:

Noted at the top are the items left to complete, which I have separated into releases. I hope to complete the first release this week.

Todo 1st release

Todo 2nd release

  • fix flexible emoncms_core install location (currently /var/www/emoncms symlinked to /var/www/html)
  • review emoncms logfile location
  • review /var/www/html/emoncms symlink

6 posts were split to a new topic: emonSD next steps: filesystem

:anguished: It’s annoying that after you split the topic, I no longer get notification emails to tell me there’s been any activity. Short of changing discourse to also split the notifications, can I suggest that as a courtesy whenever anybody splits a topic, they post a message to the original topic stating that that is what they have done? :grin:

I’m not sure if you are using the latest Raspbian Lite image, but the readme for expanding the file system does not seem to match this image after first boot. I suspect the file system has already been expanded.

Disk /dev/mmcblk0: 14.5 GiB, 15552479232 bytes, 30375936 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x8b1df36f

Device         Boot Start      End  Sectors  Size Id Type
/dev/mmcblk0p1       8192    96042    87851 42.9M  c W95 FAT32 (LBA)
/dev/mmcblk0p2      98304 30375935 30277632 14.4G 83 Linux

Trying to run the install and I fell at the first hurdle.

[edit] decided to run it anyway. Ran fine until here where it wanted some input.

rm: remove write-protected regular file 'log2ram/.git/objects/pack/pack-e60fc815bb6beda9136261e26e2b90e3e9bc867e.pack'?

Other than that seems to work :smile:. Did an export/import - all OK.

1 Like

@TrystanLea, one other point as part of the install, I think this needs a 'default.config.ini` file that can be copied and changed for use but ignored by git.

At the very least a warning to the user that it is using the default config ‘do you want to continue’ type question.

Raspbian images now resize automatically, If you look at a virgin cmdline.txt there is a entry like init=/some/path/init_resize.sh, if you delete that before putting into a Pi, it will not resize. I think it completes the full resize on the second reboot, but if you add a 3rd partition before that it might interfere with that process. I would need to look closer at it to be sure, I’m working from memory here. You won’t find the init= in cmdline.txt after it has resized though, it gets removed by raspi-config.

1 Like

For the install script I think we should work on the basis it has been resized so need to shrink first. Goes back to where we started - do we actually need to do so?

I understand what you are saying and don’t disagree entirely. But you cannot make any assumptions and start moving partitions around unless you know the resize has run it course or you take control and block, complete or abort that resize.

I don’t know what error Trystan actually saw when he wrote the steps to create 3rd partition

sudo fdisk -l
Note end of last partition (5785599 on standard sd card)
sudo fdisk /dev/mmcblk0
enter: n->p->3
enter: 5785600
enter: default or 7626751
enter: w (write partition to disk)
fails with error, will write at reboot
sudo reboot

but it’s possible that error might have been caused by unsaved changes to the partition table or saved changes not applied by partprobe (or reboot).

It could be a game of chance or bit of wrestle between the install script and the default init_resize.sh script to dictate the partitions unless the 2 are choreographed to play nicely together.