emonSD issues created using new scripts + emonhub environment file

Does it need to be strictly correct? The goal here is to make things more consistent and easier to find and use, not to meet some regulation at any cost. For a user that has some linux exposure, a good guess at where the emonhub.conf might be is /etc likewise with the data files, mysql, mariadb and redis all have thier data files in /var/lib and that is traditionally where the php??? folders have resided to (until they started getting accessed directly on the data partition of the emonSD images without symlinking).

I see no advantage to adding /opt to the various paths just to conform strictly to FHS, when I (started using and) suggested /opt to install the repo’s to, it was to locate the cloned repo’s to a single point somewhere out of the mainstream FS haunts as just a place to clone and manage the repo’s rather than having them scattered across the FS or all lumped into /home/pi. The concept of symlinking the various bits into their final positions is so that it operates and appears as if installed “correctly”, by “correctly” in this context I mean as per other softwares, ie the conf is in /etc, the log is in /var/log, the data is in /var/lib and the executable is in /usr/local/bin/ (or similar!) this could of course have been done whilst the repo’s were in /home/pi too, but that location wasn’t right either, hence the move to /opt, but if the move to /opt means all the paths need to include /opt to be strictly correct then perhaps /opt is the wrong place!

Going way back to v8 emoncms and when it was being packaged for Debian, a very good case was put forward around why we should not be operating software directly from cloned repo’s. To be strictly correct we should be downloading the source (usually via a zip, but cloning can work too) and moving, coping installing things to their final locations, disconnecting the software from the source repo’s. Git is a version control system not a package manager. Whilst I understand why git shouldn’t strictly be used for updating and upgrading deployed software, I also recognise that OEM is dependent on doing so and that to change that would be a whole new level of headache, plus since the OEM project is pretty much in constant a permanent state of development, it’s not entirely “wrong” as such, we just need to accept that the way OEM is installed, including the emonSD, is “structured for development” (even if SSH is discouraged and released “stable” branches are in play). SO, in my mind, when we are cloning all the repo’s to /opt we are not installing them! The “installation” should/could consist of moving all the required stuff into place (which would then render the /opt location irrelevant) BUT since we use git as an update/upgrade tool, instead we symlink the various bits to “where they should be”, which in my mind is not relative to the fact we are (arguably) incorrectly using the cloned repo’s directly rather than moving stuff out of where ever the cloned repo might be placed (just temporarily during a conventional install) ie /opt in this case.

Long story short, I just don’t see that typing a more obscure path for every component to include /opt to highlight the source is in /opt brings anything but confusion (and extra keyboard wear) to the party when the concept of symlinking the various bits is to essentially to emulate NOT being installed in /opt.

Just to be clear here, I’m not advocating we change the way OEM installs and updates to not use cloned repo’s (I think it’s essential we use cloned repo’s to rapidly deploy fixes as the releases are rarely fully tested and bug free). But the fact we DO use cloned repo’s could be done in such a way that it isn’t set in concrete. If/when the OEM stuff reaches maturity and ends up formally packaged (for example) emonhub.conf will still be at /etc/emonhub regardless of whether it is Debian packaged, cloned and symlinked or downloaded and installed in a conventional manner. All the paths needn’t have /opt in them and if they do not contain /opt it facilitates a move to a package/conventional installer by just replacing the symlinking with move/copy/install actions and the existing paths will remain valid, no further upheaval/reeducation of new paths then without /opt is required.

In operation /opt path would only be used “behind the scenes” for cloning&symlinking during install or upgrades etc as it is a convenient place to group the cloned repo’s. All standard operation and path’s in configuration files etc etc should use “non-opt” paths, there is no reason for a regular user to even be aware of /opt as everything will be accessible via an alternative “non-opt” path, either due to symlinks to the source or because user data and config files are not located in /opt at all. /opt/emon should IMO be just clean cloned repo’s, no edits or user files at all!!

Also, I think this discussion should be part of the emonSD next steps: filesystem & logrotate discussion rather than here (or at least linked?), these are development considerations rather than “issues created by new scripts” (strictly speaking).

As examples of what I say above, look at log2ram/install.sh and dev-emonhub/install, not because I think either is perfect, purely because they are examples I know demonstrate the 2 approaches. Log2ram attempts to install conventionally by downloading the zip from github (via curl) and then installing each component to it’s final location. You could delete the L2R repo after it’s installed with no problems arising. In contrast to that, the dev-emonhub install script tries to emulate a packaged install by using symlinks to the cloned repo so that quick and easy changes can be made by pushing and pulling the repo to github, for the purpose of development.

I think L2R is possibly “more correct” than the emonhub installer but the way the emonhub installer works is more aligned with the rest of the OEM installs and update routines etc. I will (when I get a moment) be changing my fork of L2R to use symlinks rather than install (even if only temporarily) as I think that will assist developing L2R further, eg when I changed the rsync command recently, that could easily be pulled in via git if “cloned&symlinked”, but as it stands, L2R needs to be uninstalled and reinstalled to get that minor edit. I might however do it in such a way that passing a --dev switch to the installer results in a symlinked clone and the original install method is also retained. This “switch” clearly demonstrates the difference between a “conventional” and a “development” install and from that we can see that all OEM stuff is generally installed using a “development” method. The /opt folder is nothing more than a convenient place to clone the repo’s (IMO!).