Any volunteer?
It would be great if the script also prompted the user for a ‘user’ password, which would then go on to setting up the emoncms db user too.
This would then simplify the ‘Setup the Mariadb server (MYSQL)’ section to just one sentence.
Thanks for adding that, I assumed it would need tinkering with as I just quickly drew this together from several sources that all instructed the same common tasks (albeit all with slightly different commands structures).
No idea! But I agree, if the unix recommended secure installation script does it, we should try to follow that until we know differently (eg the root password), bear in mind all this is all new to this install procedure, the secure install script was not used in the previous guides and the steps were not manually done either. So some testing and tinkering may be required.
I have been making notes/changes as I work through the guide in it’s current form. My intention was to make a script, predominantly for my own use but I’m happy to share it or contribute to a team effort.
There are 1 or 2 thing’s I do differently that may not be agreeable to some. For example I do not edit /etc/apache2/apache2.conf, I create a emoncms.conf virtualhost file in /etc/apache2/sites-available and enable the site using the correct a2ensite command and disabling the 000-default.conf with a2dissite
I also put the feed folders in a “emoncms” sub-folder rather than directly into /var/lib.
Regarding the emoncms password, I prefer a non-interactive installer and in the past I have just had the script generate a strong random password which is used to set up the emoncms user and then written into the settings.php.
For my purpose the install script should have one optional command-line argument, name, if left blank then the instance will be called “emoncms” and if provided eg “emoncms1” the emoncms table, user, virtual hast, feeds folder etc would all be created using “emoncms1” rather than the default “emoncms”.
I am also considering a matching uninstaller so test setups can be removed as easily as they are added although this will be interactive for a confirmation before removal. but I’m guessing there will be less interest in that.
That would be possible, although I would recommend doing it in a couple of steps (or more) so that it can be explained, sorting the security, adding a db and setting up a user in one chunk of code with no comments or documentation isn’t a good idea for future dev or debugging.
Thanks for the comments @pb66 & @Greebo, I’ve now updated the build guide in my repo if you could take a look please.
I’ve used it to build a system this evening, which worked fine.
Paul - Very much agree about keeping the emoncms feed folders together in a sub-folder of their own as it compartmentalizes the feeds, makes backups easier, and could also be created automatically by adding the (almost) empty folders to the emoncms repo, to cut down the installation process further.
It would be good to discuss with G&T making this change global, but not sure how backwards-compatibility could be managed.
As for managing apache via a virtualhost file, yes, gets my vote.
Many aspects of the emoncms build guide have been copied & pasted forward over the years, and maybe changes such as these need to be integrated as part of continual development.
// Engines working folder. Default is /var/lib/phpfiwa,phpfina,phptimeseries
the feeds are then defined as
in the default.emonpi.settings.php so some consistency is indeed desirable.
At least if emonpi installs are pointing directly at the /home/pi/data folder we should be able to easily add a /var/lib/emoncms folder with 3 symlinks in it to gain some consistency so that all feed folders can be found in /var/lib/emoncms on all (unix) installs without changing any thing structural in emonpi, as long as the emoncms folder and symlinks get added at the same time as the settings.php gets updated all should be fine.
Yeah, I agree. My notes (thus far) leave very little unchanged, which is why I have reservations about how that might be received, so I will just share what I’ve done and let you and the others pick and choose what you might or might not want to use.
I haven’t made much progress on this (some! but not a lot), however I am in the process of setting up a new emoncms server and I am going to try Debian9 Stretch, although it’s not a Pi the process should be pretty much the same with only a couple of minor differences.
I have started a thread on migrating an existing mysql db into a freshly installed image that has mariadb installed rather than mysql, (Restoring mySQL database to mariadb) I currently believe it should just drop in as long as “mysql_upgrade” is used afterwards, I will update there with any findings on that, however…
Whilst reading up on this I found there seems to be a Debian mariadb-client package and we appear to be still using the mysql-client in this “Raspian Stretch” emoncms install guide and thought I would just make a note here for consideration.
However. The Debian recommended route to getting the right mysql database is to install via the default-mysql-server and default-mysql-client package names and for Stretch it would default to installing mariadb-server and mariadb-client (see Chapter 2. What's new in Debian 11)
So should we be using the default-mysql-server and default-mysql-client packages and let the installer do it’s thing OR should we dictate using the mariadb-client with the mariadb-server for a fully mariadb install OR are we deliberately hanging on to the mysql-client for a good reason?
When I read this statement it suggests (but doesn’t definitively say) when a system gets upgraded to Stretch it would replace mysql-client with mariadb-client, so could we end up with some users using mysql-client and others using mariadb-client depending on whether they fresh installed or upgraded? and/or would that matter?
They all seem to play happily with each other, but it would probably make more sense to use the default-mysql-server and default-mysql-client packages.
I can see that they are both available in the Stretch sources list, but if you can confirm successful/trouble-free installation with them Paul, then sure, let’s update the build guide.
pb66@test2:~$ sudo dpkg -l | grep mysql
ii dbconfig-mysql 2.0.8 all dbconfig-common MySQL/MariaDB support
ii default-mysql-client 1.0.2 all MySQL database client binaries (metapackage)
ii default-mysql-server 1.0.2 all MySQL database server binaries and system database setup (metapackage)
ii mysql-common 5.8+1.0.2 all MySQL database common files, e.g. /etc/mysql/my.cnf
ii php-mysql 1:7.0+49 all MySQL module for PHP [default]
ii php7.0-mysql 7.0.19-1 amd64 MySQL module for PHP
pb66@test2:~$ sudo dpkg -l | grep mariadb
ii mariadb-client-10.1 10.1.26-0+deb9u1 amd64 MariaDB database client binaries
ii mariadb-client-core-10.1 10.1.26-0+deb9u1 amd64 MariaDB database core client binaries
ii mariadb-common 10.1.26-0+deb9u1 all MariaDB common metapackage
ii mariadb-server-10.1 10.1.26-0+deb9u1 amd64 MariaDB database server binaries
ii mariadb-server-core-10.1 10.1.26-0+deb9u1 amd64 MariaDB database core server files
and yes so far it been a totally successful/trouble-free installation but the server isn’t fully operational yet.
I’m basically experimenting and writing myself a revised build guide with a couple of automated install scripts, once I have perfected the scripts and practiced the backup/restore routine I will wipe the VM and fully re-install from fresh again.
That was my thinking originally, but actually as I see little real benefit to using that method over defining mariadb-server and maridb-client specifically as we still need to maintain separate package lists for Jessie and Stretch because there is no universal way of selecting php5 or php7 packages. I’m now thinking we should just define mariadb with php7 in Stretch and mysql with php5 in Jessie, but I’m fine with either way for now.
Note, I have added make to the package list, IMO it should actually be a dependency of php-pear deb package because I couldn’t install phpredis without installing make on vanilla Debian9 (Stretch), although it isn’t Raspbian the install is pretty much the same so I would recommend adding it, it won’t do any harm if it’s already in Raspian Stretch, much like the git-core package. I’ve also removed ntp from this package list as it is not needed for emoncms, it should be in the read-only OS section.
I got practically the same! (I installed mine a few weeks ago now & my version was armhf)
pi@raspberrypi:~ $ sudo dpkg -l | grep mysql
ii dbconfig-mysql 2.0.8 all dbconfig-common MySQL/MariaDB support
ii libdbd-mysql-perl 4.041-2 armhf Perl5 database interface to the MariaDB/MySQL database
ii mysql-common 5.8+1.0.2 all MySQL database common files, e.g. /etc/mysql/my.cnf
ii php-mysql 1:7.0+49 all MySQL module for PHP [default]
ii php7.0-mysql 7.0.19-1 armhf MySQL module for PHP
pi@raspberrypi:~ $ sudo dpkg -l | grep mariadb
ii libmariadbclient18:armhf 10.1.23-9+deb9u1 armhf MariaDB database client library
ii mariadb-client-10.1 10.1.23-9+deb9u1 armhf MariaDB database client binaries
ii mariadb-client-core-10.1 10.1.23-9+deb9u1 armhf MariaDB database core client binaries
ii mariadb-common 10.1.23-9+deb9u1 all MariaDB common metapackage
ii mariadb-server 10.1.23-9+deb9u1 all MariaDB database server (metapackage depending on the latest version)
ii mariadb-server-10.1 10.1.23-9+deb9u1 armhf MariaDB database server binaries
ii mariadb-server-core-10.1 10.1.23-9+deb9u1 armhf MariaDB database core server files
I did create a script that I have used succesfully, however it is really quite specialized for my own use at the moment (and very untidy).
The vhosts presents a bit of an obstacle for making this fit for public release, plus there are significant differences to the established norm which I need to document or at least have time to explain/defend which I currently do not. I will have to make further decisions on how to make it more “universal”.
I am also (when I can) working on an auto-update script which will be able to be used standalone or triggered from within emoncms like the current emonpi updater. The end goal is that if any part of emoncms is not up to date the new versions will be listed on the admin page and an “update now” button revealed.
The updater and installer are linked only in that I’m considering auto installing modules and their location/symlinking is important to get right to work for both cases.