Install dependencies

Just going through an install of Emoncms on DietPi so I can document it. The key thing is installing as many of the dependencies as possible via the DietPi software management system.

Checking what is left, I find I need php7.0-dev and build-essentials.

Can anyone confirm they are still required and not something left over from the development phase. Just seems that they may be historical.

This gives you the phpenmod and phpdismod commands eg when enabling phpredis sudo phpenmod redis the alternative is to manually create (or delete) 2 separate symlinks to enable (disable) each php module.

Is used by PECL to build compile the redis and mqtt php modules.

1 Like

Back onto this tonight - looking at the dependencies from this install instruction;

git-core - I think this should now be simply git
ufw is listed then never configured or enabled. Strictly not a requirement for emoncms.

Hi Paul, I do not see this command in the installation instructions.

Good point, I wasn’t aware that git-core was just a “transitional package” so I think we should indeed move to using the git package. (See Debian -- Details of package git-core in stretch for details)

Yes I see your points, both that it is/was incomplete/inconsistent and that “strictly” speaking it’s not a dependency, but IMO a firewall is an essential part of installing a LAMP and exposing a device to the outside world so I would prefer it was fully documented rather than removed. If we don’t recommend installing a firewall here, there isn’t a more suitable place for. Personally I always install and configure UFW on any device that has emoncms installed to it and I have included it in my emoncms installer scripts too.

This is something that comes up regularly, I’ve been banging on about installing the php modules the “right way” for as long as I’ve been banging on about not installing the dio module that we stopped using 4-5 years ago.

Recent discussion’s about redoing the docs look promising and Glyn did actually edit the Linux guide to use the correct commands (see emoncms/LinuxInstall.md at master · emoncms/emoncms · GitHub).

However, the Stretch documentation was revised to use apt-get to install php-redis in place of using PECL. It’s been a while since I have installed via apt-get so I cannot recall if the symlinks are done for you or not. But, regardless, the version installed by apt-get is 3.1.1 which we know has issues with rounding as floats are 32bits rather than 64bits when versions prior to 3.1.3 are used (see Emoncms calculation errors using redis 3.1.0 and 3.1.2 · Issue #660 · emoncms/emoncms · GitHub) this was fixed in 3.1.3 so we should be using PECL not apt-get as per the Stretch guide (see the apt-get install list for "php-redis emoncms/readme.md at master · emoncms/emoncms · GitHub).

Prior to recent months the norm has been to write the 2 “enabling” files for each module manually rather than adding a “available” file and using the correct command to auto generate 2 symlinks to that single file to enable it. (see emoncms/jessie.md at master · emoncms/emoncms · GitHub) where

sudo sh -c 'echo "extension=dio.so" > /etc/php5/apache2/conf.d/20-dio.ini'
sudo sh -c 'echo "extension=dio.so" > /etc/php5/cli/conf.d/20-dio.ini'
sudo sh -c 'echo "extension=redis.so" > /etc/php5/apache2/conf.d/20-redis.ini'
sudo sh -c 'echo "extension=redis.so" > /etc/php5/cli/conf.d/20-redis.ini'

should be reduced to

printf "extension=redis.so" | sudo tee /etc/php5/mods-available/redis.ini 1>&2
sudo php5enmod redis

or for php7.0 installs

printf "extension=redis.so" | sudo tee /etc/php/7.0/mods-available/redis.ini 1>&2
sudo phpenmod redis

this is the resulting file and symlinks created by the above commands.

[email protected]:~ $ sudo ls -la /etc/php/7.0/{*/conf.d,*/}/*redis*.*
lrwxrwxrwx 1 root root 37 Feb 12 12:12 /etc/php/7.0/apache2/conf.d/20-redis.ini -> /etc/php/7.0/mods-available/redis.ini
lrwxrwxrwx 1 root root 37 Feb 12 12:12 /etc/php/7.0/cli/conf.d/20-redis.ini -> /etc/php/7.0/mods-available/redis.ini
-rw-r--r-- 1 root root 18 Feb 12 12:12 /etc/php/7.0/mods-available/redis.ini

We have been discussing the documentation and the overall structure in the Move data & settings file to a common location? - #3 by borpin thread (as you know) and also in the Updating system libraries used by emoncms - #12 by glyn.hudson thread and also back in the Raspbian Stretch thread too.

I’m hoping we can arrive at a uniform “Linux” installation method that can then be documented and maintained in one place for all users. This was difficult in the past as the emonPi had special requirements, but as Trystan has shown interest in revising the emoncms structure we may be able to get there now.

Over the weekend I wrote (or rather revised again) an install script that I have tested on Raspbian Stretch, Debian 9 and Ubuntu 16.04 so far. I might release a pre-view version shortly, but I would ideally like to make it “multi-pass safe” so that it can also be re-run to set up secondary emoncms instances and/or to update or “fix” existing installs eg correcting permissions and replacing accidentally deleted symlinks etc.

Which I have just edited it to install “git” not “git-core” thanks :slight_smile:

I’ll update, thanks Brian.

It’s added automatically, so there’s no need to add it manually.
It’s installed at /etc/php/7.0/mods-available/redis.ini

I do agree with @pb66 that a firewall is essential, and UFW should be utilized. If anyone wants to contribute a PR to write up about setting this up, I would suggest that it’s included in general.md and can be linked in the ‘System Options’ section of the build guides.
I’ll remove it as a requirement temporarily.
PS For Cloudflare users, I’ve a script to utilise UFW in my repo.

Yes. At the time I wrote the guide, the dependencies were not available in PECL for Stretch.
Are you using PECL for Stretch installs currently? If so, please feel free to update the docs (I don’t have a spare Ras Pi to test at the moment).

Paul

@Paul can you run

sudo ls -la /etc/php/7.0/{*/conf.d,*/}/*redis*.*

to see what output you get (assuming you’ve used apt-get install php-redis) if it’s the same as I have here

[email protected]:~ $ sudo ls -la /etc/php/7.0/{*/conf.d,*/}/*redis*.*
lrwxrwxrwx 1 root root 37 Feb 12 12:12 /etc/php/7.0/apache2/conf.d/20-redis.ini -> /etc/php/7.0/mods-available/redis.ini
lrwxrwxrwx 1 root root 37 Feb 12 12:12 /etc/php/7.0/cli/conf.d/20-redis.ini -> /etc/php/7.0/mods-available/redis.ini
-rw-r--r-- 1 root root 18 Feb 12 12:12 /etc/php/7.0/mods-available/redis.ini

I strongly suspect the phpenmod redis and phpdismod redis commands would still work, it’s a requirement of Debian to fully activate the installed module where as PECL just installs hence the need to add the file and run the enable command.

i’m not sure what you mean there, PECL is installed as part of the php-pear package that is already in the apt-get list for Stretch. All that’s needed is to drop “php-redis” from that same list and add

# Install php-redis extension via PECL
pecl install redis
printf "extension=redis.so" | sudo tee /etc/php/7.0/mods-available/redis.ini 1>&2
sudo phpenmod redis

I wouldn’t bother removing it just to add it back in soon, at least with it there users may be inclined to use it or enquire about it. Activating it for ssh and http is as easy as

sudo ufw allow SSH
sudo ufw allow WWW
sufo ufw --force enable

The “–force” can be omitted in the docs I guess, I use that arg for unattended installation scripts.

That’s a shame I was going to suggest you try my script as it now conforms to what we were discussing about the emoncms file structure. If we adopt a common install procedure, several of the current install guides can be replaced with just one “Linux install guide”

[email protected]:/etc/php5/mods-available $ sudo ls -la /etc/php/7.0/{*/conf.d,*/}/*redis*.*
lrwxrwxrwx 1 root root 37 Sep 12 19:23 /etc/php/7.0/apache2/conf.d/20-redis.ini -> /etc/php/7.0/mods-available/redis.ini
lrwxrwxrwx 1 root root 37 Sep 12 19:23 /etc/php/7.0/cli/conf.d/20-redis.ini -> /etc/php/7.0/mods-available/redis.ini
-rw-r--r-- 1 root root 19 Feb  1  2017 /etc/php/7.0/mods-available/redis.ini

I’ll change it tomorrow -bed-time!!! Thanks.

More Pi’s on order! should arrive soon…

Paul

Agreed, Just wanted to point out the disconnect.

Might I suggest the apt-get install -y ufw be a separate section with basic setup instructions and remove it from the bulk dependencies instruction?

Yes I thought this might be related as I’d seen the other discussions. I’m just working through this DietPi install instruction where I want to use the DietPi software install mechanism as far as possible.

I’ll need to go back and do the REDIS bit again :frowning:

How can I test to see if REDIS has installed correctly and is working generally and for Emoncms?

Have a look at the emoncms Administration page, you can see what is cached pending a write.

Yes, that’s basically what I suggested above.
UFW was removed from the main dependencies in the git repo last night.
Paul

Sorry had not seen the subsequent posts - page had not updated :frowning:

In which case perhaps the guide should be split in 2 so that the basic LAMP including UFW is separate to the setting up of emoncms, eg when setting up emoncms on an existing server.

I also think redis should be removed from the main apt-get install as users may not be installing redis eg shared hosting. so there should be a Redis sub-section that goes something like

# If using redis (recommended)
    sudo apt-get install redis-server php-pear make php7.0-dev

    sudo PECL  install redis
    printf "extension=redis.so" | sudo tee /etc/php/7.0/mods-available/redis.ini 1>&2
    sudo phpenmod redis

# now go to settings.php and set redis_enabled = "true"

Much like the mqtt is a separate section

1 Like

Actually looking closer at the the list of packages, they all relate to the LAMP or Redis or php extensions. The only additional dependency for emoncms on a fully operational LAMP is git to clone the repo’s.

Yes please. I’m fiddling with a LLAP setup (which works) as I’ve read it is significantly less resource intensive.

Do you need redis-server as well at this point? Which should be installed first; Redis or Redis-server?

I’m getting a ’ Unable to locate package build-essentials’. Should it be build-essential (which actually gets installed as part of php7.0-dev)?

My bad, that should be redis-server not redis. I didn’t expect that “that goes something like” psudo guide to be taken literally so I was less careful.

Yes, I actually cut and pasted that from your opening post as I knew those packages were needed for compiling the redis extension from PECL, but now I checked, you are right it should be just php7.0-dev.

redis-server first would be the natural choice incase the extension tests for it, you can have redis-server without the php-redis extension but there would be no use having the extension without the server. Plus you need php-pear and the other bits to install and complile the extension.

Can I join the party? :smile:
On my DietPi LLAP setup I have

[email protected]:/var/www# ls -la /etc/php/7.0/{*/conf.d,*/}/*redis*.*
lrwxrwxrwx 1 root root 37 Feb 14 11:58 /etc/php/7.0/cgi/conf.d/20-redis.ini -> /etc/php/7.0/mods-available/redis.ini
lrwxrwxrwx 1 root root 37 Feb 14 11:58 /etc/php/7.0/cli/conf.d/20-redis.ini -> /etc/php/7.0/mods-available/redis.ini
lrwxrwxrwx 1 root root 37 Feb 14 11:58 /etc/php/7.0/fpm/conf.d/20-redis.ini -> /etc/php/7.0/mods-available/redis.ini
-rw-r--r-- 1 root root 18 Feb 14 11:57 /etc/php/7.0/mods-available/redis.ini

I did run ‘phpenmod redis’ after the pecl inslall of redis.

Well although I don’t know exactly what you should expect, it looks good to me as it appears to have worked and you have different end points, I’m pretty damn sure you didn’t want any “apache2” links in there so this is another very good example of why you should use the right tool for the job (post now bookmarked) and not just write your own files in.

{EDIT}

I’m glad the command worked for you, I wrote that knowing where I expected the apache2 files to reside, It wasn’t intended to be generic :slight_smile:

Would you therefore suggest installing by apt…

php7.0-dev
php-pear
redis-server
php-redis

And then Redis by pecl?

I’m slightly confused as there seems to be no specific installation of php-redis listed but I am sure that is just a lack of knowledge/understanding on my part.

I’m just working through it trying to make it repeatable. So far I have not done it the same way twice :smile:.

I went back and corrected my post above, I currently have

sudo apt-get install redis-server php-pear make php7.0-dev

I think make is also required to compile the extension (although it is included in some dist’s like git) and php-redis is definitely not required from apt-get as that is what you are installing with PECL.

No, it’s not you, it’s just really confusing naming

redis-server is the actual Redis database package in Debian

phpredis is the redis extension for php, it allows easy use of Redis from within php code. It’s github repo is “phpredis”, when packaged for debian it’s called “php-redis” (currently at version 3.1.1) whilst on PEAR/PECL package manager it is called just “redis” as it is a php extension.

[edit]
Similarly the “mosquitto” mqtt extension for php is called “Mosquitto-PHP” but when viewed on PECL or once installed it is just the “mosquitto” extension for php. Very different to the “mosquitto” Debian package which is THE mosquitto server package, And just to add further confusion (just in case you weren’t confused enough) the “mosquitto” php extension must be called “mosquitto-alpha” or “mosquitto-beta” to install it as it hasn’t reached a production version yet.

1 Like

In testing this I am finding that installing php7.0-dev installs make and php-pear as well.

When installing redis from pecl, If I then run the command;

printf "extension=redis.so" | tee /etc/php/7.0/mods-available/redis.ini 1>&2

… then looking for the links with ls -la /etc/php/7.0/{*/conf.d,*/}/*redis*.* I get

ls: cannot access '/etc/php/7.0/*/conf.d/*redis*.*': No such file or directory
-rw-r--r-- 1 root root 18 Feb 14 14:12 /etc/php/7.0/mods-available/redis.ini

If I then run phpenmod redis and check again, I get what I expect…

lrwxrwxrwx 1 root root 37 Feb 14 14:12 /etc/php/7.0/cgi/conf.d/20-redis.ini -> /etc/php/7.0/mods-available/redis.ini
lrwxrwxrwx 1 root root 37 Feb 14 14:12 /etc/php/7.0/cli/conf.d/20-redis.ini -> /etc/php/7.0/mods-available/redis.ini
lrwxrwxrwx 1 root root 37 Feb 14 14:12 /etc/php/7.0/fpm/conf.d/20-redis.ini -> /etc/php/7.0/mods-available/redis.ini
-rw-r--r-- 1 root root 18 Feb 14 14:12 /etc/php/7.0/mods-available/redis.ini

Seems like this method may be generic to the stack used.

However when I now try and run this brand new emoncms install, I get an error with Redis enabled

[edit] I rebooted the VM and it all now works :smile::smile: