The bit that may well cause an issue is the use of Log2Ram. I would disable that in the install scripts and, if on an SDCard, use DietPi’s own RAM logging system.
However, it looks like you are on a VM (HDD?) so there is absolutely no reason to use a tmpfs location for the log folder.
You can change what is installed in the config.ini.
Any particular reason to do this?
Testing here on jessie based emonSD-07Nov16, it works fine.
yes I think use a default that is copied to /etc/emonhub/emonhub.env
Thanks @alexandrecuer, to start with I only wish to use the .env file for this minor task of setting the emonhub paths. I think your suggestion could fit in the hierarchical settings discussion but I would like to come back to that one after reaching the first release of the image build scripts.
Thanks, emonhub should be failing at the moment as it needs these changes for the paths to work but its getting there!
Thanks @borpin
Sure I did not install log2ram…I put the emonSD var to 0 in the config.ini
and I did not install emonhub as @TrystanLea is modifying settings and proceeding to experiments…
emonSD_pi_env=0
install_emonhub=false
As far as redis is concerned, I have no rational explanation why it did not want to start but it was a log problem…there was no redis folder in /var/log
now there is one and it belongs to the user redis / group redis
@djh I was thinking about that for the modules…once emoncms core and its admin interface are running, it would be very interesting to have control over the module configuration from the admin interface…my subject was just this one…
I don’t know how many folks here are aware of this, so I thought I’d mention it…
LTS support to be provided until June 6, 2020
After June 17, 2018, the Debian Long Term Support (LTS) project will continue to provide support for various essential packages and architectures for two more years, until June 6, 2020, when the Debian GNU/Linux 8 “Jessie” operating system series will reach end of life and it won’t be available to download anymore.*
Among the supported hardware architectures, we can mention 32-bit (i386), 64-bit (amd64), Armel, and ARMhf. However, those who still want to use Debian GNU/Linux 8 “Jessie” on their computers should keep in mind that Debian LTS support is provided by a group of volunteers, not by the Debian Security team.
So it looks like we’ve got about one more year until it’s totally unsupported / gone.
Thanks @Bill.Thomson this is a good point and its a good reason to do a full image update (with an export/import of data) rather than use the emonpi updater for updates that are more than 1-2 years.
Following from emonSD next steps: filesystem & logrotate - #192 by TrystanLea
The merge of the emonhub.env version of emonhub is ready to go, the update procedure to install the emonhub.env file is now live in the latest emonpi repository: add emonhub.env settings file and install as part of update · openenergymonitor/emonpi@693ebd7 · GitHub
The final step is to merge the following emonhub pull request Use emonhub.env to set emonhub path names by TrystanLea · Pull Request #82 · openenergymonitor/emonhub · GitHub. I will merge this tomorrow unless there are any final objections
I have to say that I’m still not sold this yet. I want to be on the same page but I must be missing something as I can see no great benefit, in fact I see it as restricting the flexibility and choice of paths.
There are 3 paths defined in the env file supposedly to make them configurable.
EMONHUB_SRC=/opt/emon/emonhub/src
This path to the executable is unlikely to change because the source is currently expected to always be installed to /opt/emon/emonhub.
EMONHUB_CONF=/etc/emonhub
Since the location of the env file itself is hardcoded in the service unit to this same path, it seems unlikely that any user is likely to install emonhub.conf anywhere else but here.
EMONHUB_LOG=/var/log/emonhub
Of the 3 this, I thought was the least likely to be moved. If the user was to consider another log location, I would assume that location might well be syslog, alas changing this path will not facilitate logging to syslog. For that to happen the --logfile arg needs to be removed entirely.
So whilst i see having an env file does breakout 3 paths to be user configurable, it doesn’t really add much flexibility in practice. What it does add is an extra configuration file and the associated installer lines
This additional config file just adds complexity for all users to provide a narrow avenue of flexibility to a few but not all users. This option came about due to not agreeing about where the executable should be installed/symlinked, but you have opted for using the cloned source path directly, that decision alone removes the larger share of any potential ambiguity over paths.
Since these same 3 paths will be used in a majority of cases, there seems no point having the env file there for all users, especially sat in the same folder as the emonhub.conf we encourage users to edit, that’s just too tempting for some tinkerers.
If the service unit was hardcoded with the same paths, it would be cleaner and simpler for the majority of users. When an alternative configuration is needed a simple drop-in can be added either via systemctl edit or by adding the file directly.
That drop-in has the ability to edit any of the same variables as the env file approach, but it also has the ability to revert to syslog, change the executable or python version and doesn’t demand another hardcoded path to the env file. Yes the drop-in file path is fixed, but we are already tied to the systemd paths anyway.
With the word “final” being used in the last paragraph not once but twice, I thought I ought to go on record with my opinion rather than waiting until we are evaluating and testing. I get the impression that the move from here to delivered is likely to be quite swift.
I think I am probably in agreement here. Remind me, what problem was the ENV file trying to solve? IIRC it was around using the home folder as the install location. If this is no longer being used, what advantage does it give?
The key problem that this .env file solves is the issue of the emonhub src path. Yes your right the log file and config location are not strictly needed and are unlikely to be changed, but the configurability of the emonhub src path will be used straight away, it solves the problem of supporting emonhub on the existing emonSD images where the src path is /usr/share/emonhub and the new build script where the src path is /opt/emon/emonhub.
@djh made a clear point about the unsuitability of /usr/share and /usr/local/bin for emonhub installation or symlink above. The .env file provides flexibility.
An alternative could be to fix on /opt/emon/emonhub as the install path and move emonhub to this location on older images as part of the update process. Personally I like the .env approach, it works.
Ah yes.
personally, I find the approach very attractive for code maintenance…it will throw the config out of the code which is clever…
If I understand correctly, the log files would all be in
/var/log/<module_name> and the conf files in /etc/<module_name>
so the PHP controllers should be able to access environment variables in a fairly simple way with a file_get_contents for example…
@TrystanLea: would you be interested in a proposal like this for the config module ?
if ($env = file_get_contents("/etc/emonhub/emonhub.env")){
preg_match_all("/([\w_]+)=([\/\w]+)\n/",$env,$vars);
$ehenv=array_combine($vars[1],$vars[2]);
$emonhub_config_file=$ehenv["EMONHUB_CONF"];
$emonhub_logfile=$ehenv["EMONHUB_LOG"];
}
probably the best would be to implement this in settings.php and to use global vars in modules controllers…
Along time ago in a galaxy far, far away I mentioned:
Thanks @alexandrecuer probably the best place for this would be in process_settings.php with the .env path settable in settings.php . I think it would also be good to come back to this as part of the hierarchical settings discussion.
Use The Force, Luke, er Dave. 
Sorry. Couldn’t resist. 
As does adding a drop-in with an adjusted ExecStart line like so
sudo mkdir -p /etc/systemd/system/emonhub.service.d/
printf "[Service]\nExecStart=\nExecStart=/usr/share/emonhub/emonhub.py --config-file=/etc/emonhub/emonhub.conf --logfile=/var/log/emonhub/emonhub.log" | sudo tee /etc/systemd/system/emonhub.service.d/override.conf > /dev/null
I do not see this as any harder than editing the default env file for use on the existing emonSD’s.
Adding an env file is not necessary just to accommodate the old path on existing emonSD’s, however if you were adding an env file, the correct location would be /etc/default/emonhub IMO, there is already a file there that contains the default paths for the previous init.d script.
I disagree with that choice too, but will come back to that
He did indeed offer some wise words. Firstly /usr/share
This path has always been wrong for the emonSD, it was originally used in my separate “for development” install script which purposely mimicked a Debian package as that’s what we were developing for at that time. It was never intended and should never have been included in the emonpi emonhub repo as a mainstream installer. I have already pointed out more than once (even in this same thread) this path is not right.
Secondly . . .
and provides further explanation of why both the exectable and conf paths should include /opt as per the FHS if the repo is installed to /opt.
Whilst I do not disagree with the exact interpretation of the FHS, I reiterate that my suggestion to use /opt was purely for a convenient place to clone the repo’s, so that we could install via symlinks to the various distro specific locations.
So if understand correctly the 2 different strategies seem to be
clone the repo’s to some arbitrary location and then (on Debian/Raspbian) use more familiar paths such as /etc/emonhub and /usr/local that are also configurable to the various distro’s, using symlinks to the cloned source where appropriate, much like to the service units and various other files are already. For the record, this doesn’t need to be cloned to /opt specifically it can be anywhere, and the exact /usr/local path can be tweeked if there is a better (non-opt) suggestion.
install the repo’s to /opt and adhere to the FHS by installing the conf files to an opt path too. I presume embracing this strategy would also involve serving the emoncms www files from and opt path too? This will obviously not avoid the use of symlinks for many things such as service units and conf drop-ins and it presents a new fixed set of (opt) paths rather than a universal staging poimnt from which various distro’s can be accommodated.
Your current implementation doesn’t follow either of these, instead you have chosen to use the /opt executable path in the service units whilst keeping the conf files in the general FS. This (IMO) seriously reduces flexibility, confuses strategies and offers very little consistency. Using (for example) a symlink at /usr/local is in keeping with using /etc/emonhub, consistent with symlinking the service unit etc and detaches the front line paths from the cloned repo paths, offering a layer of flexibility.
Imagine for a moment the emonSD image wasn’t already using the wrong path, say it had always used or perhaps gets repaired in an update using something like
if [ -h /usr/share/emonhub ]; then sudo ln -s /home/pi/emonhub/src /usr/local/bin/emonhub; sudo rm /usr/share/emonhub; fi
(rolled out the same time as the new service unit!)
then the same path would be portable across images regardless of whether the source is cloned to /opt/emon, /home/pi or /on/the/moon and we wouldn’t need to be having a env file just to accommodate a previous error that can be resolved in so many other ways that would just disappear with the old images as they disappear rather than being stuck with an env file for eternity for the wrong reasons.
Just to be clear here, I’m not entirely against using an env file(s) for the wider project, that’s why I was biting my tongue earlier and hoping for the use of the env file to evolve into something else, to justify it’s existence. IMO there is a good reason to have a system wide env file (or directory of drop-in files) of paths for use at the install/update stages, but they would not be used in normal running.
For example, if the installer had no hardcoded paths at all and used a env file to set the systemd paths, the cloned repo paths, the log paths, the config paths etc etc etc, then at install time, regardless of where the repos were cloned, it would create the symlinks as prescribed by the “debian” env file, or alternatively the “opensuse” env file (for example). These env files would only be used during “install” and “update” and would promote an easy route to installing to any (systemd) distro as users/devs can create their own env files (and submit them!).
In short, there are 2 methods provided above to accommodate the old images and I can think of several more, none of which have a lasting effect beyond the existence of the old emonSD images, this is not a good reason for the env file, there are however, other good reasons for a env file, but not in it’s current form. And we really should try to remove ALL hardcoded paths from the installer.
If the use of /opt is muddying the waters, lets use something else eg /repos or /emon in the root of the fs?
PS as a side note, when I suggested using ‘/opt’ I actually envisaged /opt/emoncms/<repo> and /opt/openenergymonitor/<repo> (not /opt/emon) being used so that it was clear where the source was to be found on github and how the emoncms modules were in separate repo’s and the various FW repo’s etc. Since it’s arbitrary where the repo’s are installed as long as they are all together and fairly separate from the main FS, putting them in <git-organistation> subfolders provides additional info to the user regards navigating the repos, both locally and at github.
The point of .env files is that they can be read and used by all programs written in whatever language. So there can/should be no hard-coded paths anywhere in the system. The only hard-coded paths should be the env files and paths within them. Then there’s some chance of portability.
Although there’s still the question of service names for e.g. the redis service. Those can go in the env files or separate conf files, but should not be hard-coded throughout the system.
I’d like to try and help bring this to a conclusion - I do not have a strong opinion. I am going to try and take it bit by bit.
Firstly
I really like this idea for the install path - logical and easily maintained. I propose we adopt this for installing the repos.
Is there anything else to consider? Default permissions?
I have implemented the /opt/emoncms & /opt/openenergymonitor installation directories in a EmonScripts development branch here: Implementation of /opt/emoncms & /opt/openenergymonitor installation directories by TrystanLea · Pull Request #3 · openenergymonitor/EmonScripts · GitHub
I’m also working through the support for this in emoncms.
I think moving away from the $usrdir and $homedir naming is a good idea, Im not 100% on the separation of the openenergymonitor directory and the emoncms directory but I cant also see a strong reason against it, so have decided to take your direction and make the change.
Update: Testing a full image build with the oem_emoncms_dirs development branch works great.