Install dependencies

Just going through an install of Emoncms on DietPi so I can document it. The key thing is installing as many of the dependencies as possible via the DietPi software management system.

Checking what is left, I find I need php7.0-dev and build-essentials.

Can anyone confirm they are still required and not something left over from the development phase. Just seems that they may be historical.

This gives you the phpenmod and phpdismod commands eg when enabling phpredis sudo phpenmod redis the alternative is to manually create (or delete) 2 separate symlinks to enable (disable) each php module.

Is used by PECL to build compile the redis and mqtt php modules.

1 Like

Back onto this tonight - looking at the dependencies from this install instruction;

git-core - I think this should now be simply git
ufw is listed then never configured or enabled. Strictly not a requirement for emoncms.

Hi Paul, I do not see this command in the installation instructions.

Good point, I wasnā€™t aware that git-core was just a ā€œtransitional packageā€ so I think we should indeed move to using the git package. (See Debian -- Error for details)

Yes I see your points, both that it is/was incomplete/inconsistent and that ā€œstrictlyā€ speaking itā€™s not a dependency, but IMO a firewall is an essential part of installing a LAMP and exposing a device to the outside world so I would prefer it was fully documented rather than removed. If we donā€™t recommend installing a firewall here, there isnā€™t a more suitable place for. Personally I always install and configure UFW on any device that has emoncms installed to it and I have included it in my emoncms installer scripts too.

This is something that comes up regularly, Iā€™ve been banging on about installing the php modules the ā€œright wayā€ for as long as Iā€™ve been banging on about not installing the dio module that we stopped using 4-5 years ago.

Recent discussionā€™s about redoing the docs look promising and Glyn did actually edit the Linux guide to use the correct commands (see https://github.com/emoncms/emoncms/blob/master/docs/LinuxInstall.md#install-php-pecl-dependencies).

However, the Stretch documentation was revised to use apt-get to install php-redis in place of using PECL. Itā€™s been a while since I have installed via apt-get so I cannot recall if the symlinks are done for you or not. But, regardless, the version installed by apt-get is 3.1.1 which we know has issues with rounding as floats are 32bits rather than 64bits when versions prior to 3.1.3 are used (see https://github.com/emoncms/emoncms/issues/660) this was fixed in 3.1.3 so we should be using PECL not apt-get as per the Stretch guide (see the apt-get install list for "php-redis https://github.com/emoncms/emoncms/blob/master/docs/RaspberryPi/readme.md#installation).

Prior to recent months the norm has been to write the 2 ā€œenablingā€ files for each module manually rather than adding a ā€œavailableā€ file and using the correct command to auto generate 2 symlinks to that single file to enable it. (see https://github.com/emoncms/emoncms/blob/master/docs/RaspberryPi/jessie.md#preparation) where

sudo sh -c 'echo "extension=dio.so" > /etc/php5/apache2/conf.d/20-dio.ini'
sudo sh -c 'echo "extension=dio.so" > /etc/php5/cli/conf.d/20-dio.ini'
sudo sh -c 'echo "extension=redis.so" > /etc/php5/apache2/conf.d/20-redis.ini'
sudo sh -c 'echo "extension=redis.so" > /etc/php5/cli/conf.d/20-redis.ini'

should be reduced to

printf "extension=redis.so" | sudo tee /etc/php5/mods-available/redis.ini 1>&2
sudo php5enmod redis

or for php7.0 installs

printf "extension=redis.so" | sudo tee /etc/php/7.0/mods-available/redis.ini 1>&2
sudo phpenmod redis

this is the resulting file and symlinks created by the above commands.

pi@raspberrypi:~ $ sudo ls -la /etc/php/7.0/{*/conf.d,*/}/*redis*.*
lrwxrwxrwx 1 root root 37 Feb 12 12:12 /etc/php/7.0/apache2/conf.d/20-redis.ini -> /etc/php/7.0/mods-available/redis.ini
lrwxrwxrwx 1 root root 37 Feb 12 12:12 /etc/php/7.0/cli/conf.d/20-redis.ini -> /etc/php/7.0/mods-available/redis.ini
-rw-r--r-- 1 root root 18 Feb 12 12:12 /etc/php/7.0/mods-available/redis.ini

We have been discussing the documentation and the overall structure in the Move data & settings file to a common location? - #3 by borpin thread (as you know) and also in the Updating system libraries used by emoncms - #12 by glyn.hudson thread and also back in the Raspbian Stretch thread too.

Iā€™m hoping we can arrive at a uniform ā€œLinuxā€ installation method that can then be documented and maintained in one place for all users. This was difficult in the past as the emonPi had special requirements, but as Trystan has shown interest in revising the emoncms structure we may be able to get there now.

Over the weekend I wrote (or rather revised again) an install script that I have tested on Raspbian Stretch, Debian 9 and Ubuntu 16.04 so far. I might release a pre-view version shortly, but I would ideally like to make it ā€œmulti-pass safeā€ so that it can also be re-run to set up secondary emoncms instances and/or to update or ā€œfixā€ existing installs eg correcting permissions and replacing accidentally deleted symlinks etc.

Which I have just edited it to install ā€œgitā€ not ā€œgit-coreā€ thanks :slight_smile:

Iā€™ll update, thanks Brian.

Itā€™s added automatically, so thereā€™s no need to add it manually.
Itā€™s installed at /etc/php/7.0/mods-available/redis.ini

I do agree with @pb66 that a firewall is essential, and UFW should be utilized. If anyone wants to contribute a PR to write up about setting this up, I would suggest that itā€™s included in general.md and can be linked in the ā€˜System Optionsā€™ section of the build guides.
Iā€™ll remove it as a requirement temporarily.
PS For Cloudflare users, Iā€™ve a script to utilise UFW in my repo.

Yes. At the time I wrote the guide, the dependencies were not available in PECL for Stretch.
Are you using PECL for Stretch installs currently? If so, please feel free to update the docs (I donā€™t have a spare Ras Pi to test at the moment).

Paul

@Paul can you run

sudo ls -la /etc/php/7.0/{*/conf.d,*/}/*redis*.*

to see what output you get (assuming youā€™ve used apt-get install php-redis) if itā€™s the same as I have here

pi@raspberrypi:~ $ sudo ls -la /etc/php/7.0/{*/conf.d,*/}/*redis*.*
lrwxrwxrwx 1 root root 37 Feb 12 12:12 /etc/php/7.0/apache2/conf.d/20-redis.ini -> /etc/php/7.0/mods-available/redis.ini
lrwxrwxrwx 1 root root 37 Feb 12 12:12 /etc/php/7.0/cli/conf.d/20-redis.ini -> /etc/php/7.0/mods-available/redis.ini
-rw-r--r-- 1 root root 18 Feb 12 12:12 /etc/php/7.0/mods-available/redis.ini

I strongly suspect the phpenmod redis and phpdismod redis commands would still work, itā€™s a requirement of Debian to fully activate the installed module where as PECL just installs hence the need to add the file and run the enable command.

iā€™m not sure what you mean there, PECL is installed as part of the php-pear package that is already in the apt-get list for Stretch. All thatā€™s needed is to drop ā€œphp-redisā€ from that same list and add

# Install php-redis extension via PECL
pecl install redis
printf "extension=redis.so" | sudo tee /etc/php/7.0/mods-available/redis.ini 1>&2
sudo phpenmod redis

I wouldnā€™t bother removing it just to add it back in soon, at least with it there users may be inclined to use it or enquire about it. Activating it for ssh and http is as easy as

sudo ufw allow SSH
sudo ufw allow WWW
sufo ufw --force enable

The ā€œā€“forceā€ can be omitted in the docs I guess, I use that arg for unattended installation scripts.

Thatā€™s a shame I was going to suggest you try my script as it now conforms to what we were discussing about the emoncms file structure. If we adopt a common install procedure, several of the current install guides can be replaced with just one ā€œLinux install guideā€

pi@raspberrypi:/etc/php5/mods-available $ sudo ls -la /etc/php/7.0/{*/conf.d,*/}/*redis*.*
lrwxrwxrwx 1 root root 37 Sep 12 19:23 /etc/php/7.0/apache2/conf.d/20-redis.ini -> /etc/php/7.0/mods-available/redis.ini
lrwxrwxrwx 1 root root 37 Sep 12 19:23 /etc/php/7.0/cli/conf.d/20-redis.ini -> /etc/php/7.0/mods-available/redis.ini
-rw-r--r-- 1 root root 19 Feb  1  2017 /etc/php/7.0/mods-available/redis.ini

Iā€™ll change it tomorrow -bed-time!!! Thanks.

More Piā€™s on order! should arrive soonā€¦

Paul

Agreed, Just wanted to point out the disconnect.

Might I suggest the apt-get install -y ufw be a separate section with basic setup instructions and remove it from the bulk dependencies instruction?

Yes I thought this might be related as Iā€™d seen the other discussions. Iā€™m just working through this DietPi install instruction where I want to use the DietPi software install mechanism as far as possible.

Iā€™ll need to go back and do the REDIS bit again :frowning:

How can I test to see if REDIS has installed correctly and is working generally and for Emoncms?

Have a look at the emoncms Administration page, you can see what is cached pending a write.

Yes, thatā€™s basically what I suggested above.
UFW was removed from the main dependencies in the git repo last night.
Paul

Sorry had not seen the subsequent posts - page had not updated :frowning:

In which case perhaps the guide should be split in 2 so that the basic LAMP including UFW is separate to the setting up of emoncms, eg when setting up emoncms on an existing server.

I also think redis should be removed from the main apt-get install as users may not be installing redis eg shared hosting. so there should be a Redis sub-section that goes something like

# If using redis (recommended)
    sudo apt-get install redis-server php-pear make php7.0-dev

    sudo PECL  install redis
    printf "extension=redis.so" | sudo tee /etc/php/7.0/mods-available/redis.ini 1>&2
    sudo phpenmod redis

# now go to settings.php and set redis_enabled = "true"

Much like the mqtt is a separate section

1 Like

Actually looking closer at the the list of packages, they all relate to the LAMP or Redis or php extensions. The only additional dependency for emoncms on a fully operational LAMP is git to clone the repoā€™s.

Yes please. Iā€™m fiddling with a LLAP setup (which works) as Iā€™ve read it is significantly less resource intensive.

Do you need redis-server as well at this point? Which should be installed first; Redis or Redis-server?

Iā€™m getting a ā€™ Unable to locate package build-essentialsā€™. Should it be build-essential (which actually gets installed as part of php7.0-dev)?

My bad, that should be redis-server not redis. I didnā€™t expect that ā€œthat goes something likeā€ psudo guide to be taken literally so I was less careful.

Yes, I actually cut and pasted that from your opening post as I knew those packages were needed for compiling the redis extension from PECL, but now I checked, you are right it should be just php7.0-dev.

redis-server first would be the natural choice incase the extension tests for it, you can have redis-server without the php-redis extension but there would be no use having the extension without the server. Plus you need php-pear and the other bits to install and complile the extension.

Can I join the party? :smile:
On my DietPi LLAP setup I have

root@DietPi-Emon:/var/www# ls -la /etc/php/7.0/{*/conf.d,*/}/*redis*.*
lrwxrwxrwx 1 root root 37 Feb 14 11:58 /etc/php/7.0/cgi/conf.d/20-redis.ini -> /etc/php/7.0/mods-available/redis.ini
lrwxrwxrwx 1 root root 37 Feb 14 11:58 /etc/php/7.0/cli/conf.d/20-redis.ini -> /etc/php/7.0/mods-available/redis.ini
lrwxrwxrwx 1 root root 37 Feb 14 11:58 /etc/php/7.0/fpm/conf.d/20-redis.ini -> /etc/php/7.0/mods-available/redis.ini
-rw-r--r-- 1 root root 18 Feb 14 11:57 /etc/php/7.0/mods-available/redis.ini

I did run ā€˜phpenmod redisā€™ after the pecl inslall of redis.

Well although I donā€™t know exactly what you should expect, it looks good to me as it appears to have worked and you have different end points, Iā€™m pretty damn sure you didnā€™t want any ā€œapache2ā€ links in there so this is another very good example of why you should use the right tool for the job (post now bookmarked) and not just write your own files in.

{EDIT}

Iā€™m glad the command worked for you, I wrote that knowing where I expected the apache2 files to reside, It wasnā€™t intended to be generic :slight_smile:

Would you therefore suggest installing by aptā€¦

php7.0-dev
php-pear
redis-server
php-redis

And then Redis by pecl?

Iā€™m slightly confused as there seems to be no specific installation of php-redis listed but I am sure that is just a lack of knowledge/understanding on my part.

Iā€™m just working through it trying to make it repeatable. So far I have not done it the same way twice :smile:.

I went back and corrected my post above, I currently have

sudo apt-get install redis-server php-pear make php7.0-dev

I think make is also required to compile the extension (although it is included in some distā€™s like git) and php-redis is definitely not required from apt-get as that is what you are installing with PECL.

No, itā€™s not you, itā€™s just really confusing naming

redis-server is the actual Redis database package in Debian

phpredis is the redis extension for php, it allows easy use of Redis from within php code. Itā€™s github repo is ā€œphpredisā€, when packaged for debian itā€™s called ā€œphp-redisā€ (currently at version 3.1.1) whilst on PEAR/PECL package manager it is called just ā€œredisā€ as it is a php extension.

[edit]
Similarly the ā€œmosquittoā€ mqtt extension for php is called ā€œMosquitto-PHPā€ but when viewed on PECL or once installed it is just the ā€œmosquittoā€ extension for php. Very different to the ā€œmosquittoā€ Debian package which is THE mosquitto server package, And just to add further confusion (just in case you werenā€™t confused enough) the ā€œmosquittoā€ php extension must be called ā€œmosquitto-alphaā€ or ā€œmosquitto-betaā€ to install it as it hasnā€™t reached a production version yet.

1 Like

In testing this I am finding that installing php7.0-dev installs make and php-pear as well.

When installing redis from pecl, If I then run the command;

printf "extension=redis.so" | tee /etc/php/7.0/mods-available/redis.ini 1>&2

ā€¦ then looking for the links with ls -la /etc/php/7.0/{*/conf.d,*/}/*redis*.* I get

ls: cannot access '/etc/php/7.0/*/conf.d/*redis*.*': No such file or directory
-rw-r--r-- 1 root root 18 Feb 14 14:12 /etc/php/7.0/mods-available/redis.ini

If I then run phpenmod redis and check again, I get what I expectā€¦

lrwxrwxrwx 1 root root 37 Feb 14 14:12 /etc/php/7.0/cgi/conf.d/20-redis.ini -> /etc/php/7.0/mods-available/redis.ini
lrwxrwxrwx 1 root root 37 Feb 14 14:12 /etc/php/7.0/cli/conf.d/20-redis.ini -> /etc/php/7.0/mods-available/redis.ini
lrwxrwxrwx 1 root root 37 Feb 14 14:12 /etc/php/7.0/fpm/conf.d/20-redis.ini -> /etc/php/7.0/mods-available/redis.ini
-rw-r--r-- 1 root root 18 Feb 14 14:12 /etc/php/7.0/mods-available/redis.ini

Seems like this method may be generic to the stack used.

However when I now try and run this brand new emoncms install, I get an error with Redis enabled

[edit] I rebooted the VM and it all now works :smile::smile: