It seems to me that this is a separate but extremely important issue. Surely it would make more sense to have the classical three repos: development, testing and production?
So all this stuff would be going on in the development repo, which people who wish could use on test installation, but nobody would use for their actual energy monitoring. Then when there was an apparently stable release candidate, it could be cloned to testing and more people would use it, including me if we resolve this paths issue, and a few brave souls might actually use it for live monitoring. Finally, when its known to work, it’s cloned to a production repo and downloaded by the great unwashed and used for producing factory product etc.
I part company with you here. /opt is a sensible place for binaries and /etc/opt is a sensible place for their config files etc. So there’s no need to symlink them to other places.
But again, the first step is an environment file. Then the actual paths are moot. I can install it on my openSUSE machine. I could submit an OBS ( https://build.opensuse.org/ ) job to build packaged versions for openSUSE and Debian etc.
Note that /var/www is not where FHS says to put web data - that’s /srv/www - and that’s where openSUSE puts it.
There is a bit of history in play and how it has ‘developed’ over time. For some of the repos, there is a ‘stable’ branch which is effectively a release branch. However, some repos do not have a ‘stable’ branch.
I have some heavily customised wiki software on my PC that I use for my diary and making notes etc. It’s installed in /opt and runs happily from there. It isn’t symlinked below /srv/www; I just tell apache where to find it with an apache config file - I need that anyway because there are specials like rewrite rules that the wiki software needs.
Because this isn’t just about the path to the binary, it’s also about the apache path. It’s essential for ALL paths to be in the Environment File (and that is a systemd concept just as much as a sysV one). There are very few runtime environments that can’t read environment variables, and that’s for a very good reason.
As I pointed out elsewhere, Debian and Raspbian don’t follow FHS, so you’re arguing against yourself. One reason there aren’t more installs on other distros is precisely because paths are hardcoded to Debian and other distros use other paths.
IMHO, one Environment File is a whole lot simpler and cleaner than a bunch of symlinks.
My immediate reaction to that was to file a bug for it. But then I thought that you know a whole lot more than me about git and the best ways to use it, so perhaps you’d like to write a bug that proposes a sensible setup? Pretty please
I definitely would not go that far! It isn’t that simple. The releases for the main repo now use tags so that is an improvement but for now we need to live with how it is and just recognize the limitations.
Given that in this case the distro is not packaging the ‘product’ is it not therefore correct to use /opt directly?
The precise location aside, one of the goals of this installation script is to provide flexible installation paths rather than nailing everything down. I know we are still a number of steps away from that (e.g /var/www discussion), but perhaps using the environment file to allow emonhub to either be installed by our installer or perhaps if someone wants to do so be packaged independently and installed by the distro into /usr/local or other relevant location?
I haven’t tried using an environment file with systemd yet so I’m assuming this is possible and not bad practice
Note that the environment file should include the /var/www/emoncms path and all other paths. You can read the file into the environment in a systemd unit with the EnvironmentFile directive. It looks like the best way to read them into a PHP environment is with the dotenv package as described at How to Use Environment Variables ― Scotch.io . There’s also a python-dotenv package that does something similar.
I think apache requires special handling, since it only accepts environment variables from the process that starts it, and the startup mechanism for apache varies between distros and is changing over time. It may be simplest to make the apache vhost conf file use Define statements within it for the paths. Then in order to install it in a non-default place it will be necessary to edit that file as well as the environment file before copying/moving it to wherever it is needed on that system. A bit more hassle but I don’t see a simpler way and it shouldn’t be a problem as long as it is documented. If anybody can think of a better way, I’m all ears!
I suggest using an EMON_ prefix for all the variables.
With a view to emonhub being able to be installed seperatly, ideally the environment file would be specific to emonhub in this case rather than one file for all aspects of the emonSD installation. Can you see any issue with this?
Not off the top of my head. It obviously complicates things with multiple files then necessary for the complete system. Equally, having extra environment variables defined in the file wouldn’t have a negative effect on anything if they weren’t used in an emonhub-only installation, so it might be easier to keep everything in one file.
[edit]: perhaps the best idea depends on the directory structure. I think I would have one .env file in each top-level directory. So if the plan is to have emonhub as a package in /opt/emonhub and emoncms (or whatever) in /opt/emoncms then I would have an environment file in each. Say emonhub.env and emoncms.env
Are you going to write the Environment file on the fly rather than have one in the git repo(s)? How will you handle updates (where a user may have edited it)?
This takes us back to the hierarchical settings issue IMHO. You could employ the general principle of drop-ins - effectively the same as my hierarchical settings proposal from way back.
The standard way to handle such issues is to have a file named emonhub-defaults.env or similar in the repo, so that is what gets overwritten in an update. But the running system looks for emonhub.env to get its paths so it can stop with an error message if there is no such file, or perhaps automatically copy the defaults file to the operational file location if it doesn’t exist. That way any customisation is never overwritten accidently.
Not a standard that is currently used here - which is my point.
Oh and the problem with that is, that, unless you load both (unclear if you meant that), if a new variable is added, the file in use does not have that variable. The emoncms settings file is an example of the issues.
I don’t think that’s a serious difficulty. Firstly, I think it will be fairly rare for a new path to be added to the environment; most changes are within existing trees. Secondly, the update can check whether the file in use was modified and if not use the new default. Thirdly, if it was modified then the user clearly understands enough to do that, and likely expects future maintenance and a clear and comprehensive prompt about what needs doing should be enough to get them going.
(I know the first point about emonhub.py path is still under discussion, but) In connection with your recently released fix (Fix for full /var/log issue, causing inputs to stop updating is now available) and the comments in the above mentioned issue, the forth point about the emonhub service creating it’s own log folder/file would, if adopted for the emoncms services too, negate any need for entries in the rc.local for those services, and since feedwriter, emoncms_mqtt and service-runner would all check and create /var/log/emon if required, the folder and emoncms.log would already exist before any webpage is served so /var/log/emon would not need to be created in rc.local in the interim. (as described in /var/log still filling up - #27 by pb66)
Both the emonhub.conf and emonhub.log paths may well be configurable, but the ability to manage it’s own log folder/file is a good addition regardless, it’s more robust, especially when there’s a possibility of it being used with any form of RAM based log partition. The pending decision on paths, should not prevent the unit’s from creating their own log folder/files
With regards to the postprocess,backup,sync module logs and the update logs, these can only work with service-runner active so there is no chance of them being triggered without a log in place. It’s only if emoncms is installed with no services at all that a provision to create the log folder (and file) is needed, that’s not the position on a emonSD so emoncms.log would’nt need creating in rc.local at all.
