Raspbian Stretch

Any volunteer?
It would be great if the script also prompted the user for a ‘user’ password, which would then go on to setting up the emoncms db user too.
This would then simplify the ‘Setup the Mariadb server (MYSQL)’ section to just one sentence.

Paul

Thanks for adding that, I assumed it would need tinkering with as I just quickly drew this together from several sources that all instructed the same common tasks (albeit all with slightly different commands structures).

No idea! But I agree, if the unix recommended secure installation script does it, we should try to follow that until we know differently (eg the root password), bear in mind all this is all new to this install procedure, the secure install script was not used in the previous guides and the steps were not manually done either. So some testing and tinkering may be required.

I have been making notes/changes as I work through the guide in it’s current form. My intention was to make a script, predominantly for my own use but I’m happy to share it or contribute to a team effort.

There are 1 or 2 thing’s I do differently that may not be agreeable to some. For example I do not edit /etc/apache2/apache2.conf, I create a emoncms.conf virtualhost file in /etc/apache2/sites-available and enable the site using the correct a2ensite command and disabling the 000-default.conf with a2dissite

I also put the feed folders in a “emoncms” sub-folder rather than directly into /var/lib.

Regarding the emoncms password, I prefer a non-interactive installer and in the past I have just had the script generate a strong random password which is used to set up the emoncms user and then written into the settings.php.

For my purpose the install script should have one optional command-line argument, name, if left blank then the instance will be called “emoncms” and if provided eg “emoncms1” the emoncms table, user, virtual hast, feeds folder etc would all be created using “emoncms1” rather than the default “emoncms”.

I am also considering a matching uninstaller so test setups can be removed as easily as they are added although this will be interactive for a confirmation before removal. but I’m guessing there will be less interest in that.

That would be possible, although I would recommend doing it in a couple of steps (or more) so that it can be explained, sorting the security, adding a db and setting up a user in one chunk of code with no comments or documentation isn’t a good idea for future dev or debugging.

1 Like

Thanks for the comments @pb66 & @Greebo, I’ve now updated the build guide in my repo if you could take a look please.
I’ve used it to build a system this evening, which worked fine.

Paul - Very much agree about keeping the emoncms feed folders together in a sub-folder of their own as it compartmentalizes the feeds, makes backups easier, and could also be created automatically by adding the (almost) empty folders to the emoncms repo, to cut down the installation process further.
It would be good to discuss with G&T making this change global, but not sure how backwards-compatibility could be managed.

As for managing apache via a virtualhost file, yes, gets my vote.

Many aspects of the emoncms build guide have been copied & pasted forward over the years, and maybe changes such as these need to be integrated as part of continual development.

Paul

Well! Funny you should say that…

Over on the “Local emoncms processes and feeds lost after change to a process - Help please” I just discovered that the feeds folders on an emonPi are no longer symlinked and despite the default.emonpi.settings.php file stating

// Engines working folder. Default is /var/lib/phpfiwa,phpfina,phptimeseries

the feeds are then defined as

in the default.emonpi.settings.php so some consistency is indeed desirable.

At least if emonpi installs are pointing directly at the /home/pi/data folder we should be able to easily add a /var/lib/emoncms folder with 3 symlinks in it to gain some consistency so that all feed folders can be found in /var/lib/emoncms on all (unix) installs without changing any thing structural in emonpi, as long as the emoncms folder and symlinks get added at the same time as the settings.php gets updated all should be fine.

Yeah, I agree. My notes (thus far) leave very little unchanged, which is why I have reservations about how that might be received, so I will just share what I’ve done and let you and the others pick and choose what you might or might not want to use.

1 Like

I haven’t made much progress on this (some! but not a lot), however I am in the process of setting up a new emoncms server and I am going to try Debian9 Stretch, although it’s not a Pi the process should be pretty much the same with only a couple of minor differences.

I have started a thread on migrating an existing mysql db into a freshly installed image that has mariadb installed rather than mysql, (Restoring mySQL database to mariadb) I currently believe it should just drop in as long as “mysql_upgrade” is used afterwards, I will update there with any findings on that, however…

Whilst reading up on this I found there seems to be a Debian mariadb-client package and we appear to be still using the mysql-client in this “Raspian Stretch” emoncms install guide and thought I would just make a note here for consideration.

I do not think this is a problem par se as apparently “The mysql-client package also works with MariaDB server.” plus you guys have it working and haven’t suggested there are any issues.

However. The Debian recommended route to getting the right mysql database is to install via the default-mysql-server and default-mysql-client package names and for Stretch it would default to installing mariadb-server and mariadb-client (see Chapter 2. What's new in Debian 11)

So should we be using the default-mysql-server and default-mysql-client packages and let the installer do it’s thing OR should we dictate using the mariadb-client with the mariadb-server for a fully mariadb install OR are we deliberately hanging on to the mysql-client for a good reason?

When I read this statement it suggests (but doesn’t definitively say) when a system gets upgraded to Stretch it would replace mysql-client with mariadb-client, so could we end up with some users using mysql-client and others using mariadb-client depending on whether they fresh installed or upgraded? and/or would that matter?

They all seem to play happily with each other, but it would probably make more sense to use the default-mysql-server and default-mysql-client packages.
I can see that they are both available in the Stretch sources list, but if you can confirm successful/trouble-free installation with them Paul, then sure, let’s update the build guide.

Paul

Using this package list

sudo apt-get install -y apache2 default-mysql-server default-mysql-client php7.0 libapache2-mod-php7.0 php7.0-mysql php7.0-gd php7.0-opcache php7.0-curl php7.0-dev php7.0-mcrypt php7.0-common php-pear redis-server git-core make

I ended up with the mysql packages

pb66@test2:~$ sudo dpkg -l | grep mysql
ii  dbconfig-mysql                2.0.8                          all          dbconfig-common MySQL/MariaDB support
ii  default-mysql-client          1.0.2                          all          MySQL database client binaries (metapackage)
ii  default-mysql-server          1.0.2                          all          MySQL database server binaries and system database setup (metapackage)
ii  mysql-common                  5.8+1.0.2                      all          MySQL database common files, e.g. /etc/mysql/my.cnf
ii  php-mysql                     1:7.0+49                       all          MySQL module for PHP [default]
ii  php7.0-mysql                  7.0.19-1                       amd64        MySQL module for PHP
pb66@test2:~$ sudo dpkg -l | grep mariadb
ii  mariadb-client-10.1           10.1.26-0+deb9u1               amd64        MariaDB database client binaries
ii  mariadb-client-core-10.1      10.1.26-0+deb9u1               amd64        MariaDB database core client binaries
ii  mariadb-common                10.1.26-0+deb9u1               all          MariaDB common metapackage
ii  mariadb-server-10.1           10.1.26-0+deb9u1               amd64        MariaDB database server binaries
ii  mariadb-server-core-10.1      10.1.26-0+deb9u1               amd64        MariaDB database core server files

and yes so far it been a totally successful/trouble-free installation but the server isn’t fully operational yet.

I’m basically experimenting and writing myself a revised build guide with a couple of automated install scripts, once I have perfected the scripts and practiced the backup/restore routine I will wipe the VM and fully re-install from fresh again.

That was my thinking originally, but actually as I see little real benefit to using that method over defining mariadb-server and maridb-client specifically as we still need to maintain separate package lists for Jessie and Stretch because there is no universal way of selecting php5 or php7 packages. I’m now thinking we should just define mariadb with php7 in Stretch and mysql with php5 in Jessie, but I’m fine with either way for now.

Note, I have added make to the package list, IMO it should actually be a dependency of php-pear deb package because I couldn’t install phpredis without installing make on vanilla Debian9 (Stretch), although it isn’t Raspbian the install is pretty much the same so I would recommend adding it, it won’t do any harm if it’s already in Raspian Stretch, much like the git-core package. I’ve also removed ntp from this package list as it is not needed for emoncms, it should be in the read-only OS section.

Out of interest, using the existing build guide;

sudo apt-get install -y apache2 mariadb-server mysql-client php7.0 libapache2-mod-php7.0 php7.0-mysql php7.0-gd php7.0-opcache php7.0-curl php-pear php7.0-dev php7.0-mcrypt php7.0-common redis-server php-redis git-core build-essential ufw ntp

I got practically the same! (I installed mine a few weeks ago now & my version was armhf)

pi@raspberrypi:~ $ sudo dpkg -l | grep mysql
ii  dbconfig-mysql                        2.0.8                                all          dbconfig-common MySQL/MariaDB support
ii  libdbd-mysql-perl                     4.041-2                              armhf        Perl5 database interface to the MariaDB/MySQL database
ii  mysql-common                          5.8+1.0.2                            all          MySQL database common files, e.g. /etc/mysql/my.cnf
ii  php-mysql                             1:7.0+49                             all          MySQL module for PHP [default]
ii  php7.0-mysql                          7.0.19-1                             armhf        MySQL module for PHP
pi@raspberrypi:~ $ sudo dpkg -l | grep mariadb
ii  libmariadbclient18:armhf              10.1.23-9+deb9u1                     armhf        MariaDB database client library
ii  mariadb-client-10.1                   10.1.23-9+deb9u1                     armhf        MariaDB database client binaries
ii  mariadb-client-core-10.1              10.1.23-9+deb9u1                     armhf        MariaDB database core client binaries
ii  mariadb-common                        10.1.23-9+deb9u1                     all          MariaDB common metapackage
ii  mariadb-server                        10.1.23-9+deb9u1                     all          MariaDB database server (metapackage depending on the latest version)
ii  mariadb-server-10.1                   10.1.23-9+deb9u1                     armhf        MariaDB database server binaries
ii  mariadb-server-core-10.1              10.1.23-9+deb9u1                     armhf        MariaDB database core server files

So long story short, it doesn’t much matter what you ask for you get what your given!

I used default-mysql-client, you used mysql-client and one would hope when mariadb-client is used, all result in mariadb-client-10.1 being installed.

Odd how I didn’t get mariadb-server however you did specifically ask for it.

I assume libmariadbclient18:armhf is an additional package required for armhf, likewise for libdbd-mysql-perl unless you installed it seperately.

Ha ha! appears so.

No, Just like the above, they wanted me to have them!!

Hi Paul, followed the link and got a 404. Is the script around somewhere?

Hi Brian - The link was probably lost when the “stretch” page became the default “readme”

OK thanks. Did you create a script to install Emoncms?

Hi Brian, yes that fork was merged into emoncms, and then deleted. I’ll remove the link.

Paul

Are you referring to a script to install emoncms?

I was replying to the question which you asked me;

It’s Paul - @pb66 who was looking at a script.

Paul

Ah yes, sorry, wrong Paul :slight_smile:

I did create a script that I have used succesfully, however it is really quite specialized for my own use at the moment (and very untidy).

The vhosts presents a bit of an obstacle for making this fit for public release, plus there are significant differences to the established norm which I need to document or at least have time to explain/defend which I currently do not. I will have to make further decisions on how to make it more “universal”.

I am also (when I can) working on an auto-update script which will be able to be used standalone or triggered from within emoncms like the current emonpi updater. The end goal is that if any part of emoncms is not up to date the new versions will be listed on the admin page and an “update now” button revealed.

The updater and installer are linked only in that I’m considering auto installing modules and their location/symlinking is important to get right to work for both cases.

1 Like

12 posts were split to a new topic: Updating system libraries used by emoncms