Local Ubuntu Emoncms and Redis

You’ve lost me a bit there, the guide in the emoncms repo is complete, we do not edit php.ini as a rule. The correct way is to use phpenmod top create symlinks in the necessary places to allow php to dynamically create it’s settings.

Aside from installing the Debian packages for php-pear and php-dev etc included in the main package list, installing the redis extension consists of the following 4 steps

  1. Updating the PECL index channel with sudo pecl channel-update pecl.php.net
  2. Install the redis package with sudo pecl install redis (or maybe a specific version)
  3. Make that extension “available” using printf "extension=redis.so" | sudo tee /etc/php/7.0/mods-available/redis.ini 1>&2 to add a file in “mods-available”
  4. Enable the extension with sudo phpenmod redis

at that point you should be done!

If there are any further steps required due to unusual paths etc because of the environment you are installing to I’m afraid I do not know about those.

running pecl info redis to tell you what version phpredis is installed ie available to enable.

running php -m will tell you what modules are enabled, “redis” should be hopefully be listed.

from the local machine you should also be able to navigate to http://myserver/emoncms/php-info.php to see a fair bit of info and at the end there should be a “redis” section that tells you about the phpredis extension.

And also if you look at the last section on the admin page there should be a “redis” entry with version number in there too if it’s successfully enabled, installed but not enabled modules are not listed there.

This is not to be confused with the Redis server section on the admin page which is the database application itself, not the php extension, that is what I believe you are looking at when you say “Emoncms reports Redis 3.06” as you have already said you’ve installed v4.0.2 and mentioned “rolling back to 3.1.6” which you cannot do if you are on 3.0.6.

Here is a “System information” dump that I have just pinched from a recent post by @Paul

Server Information
Emoncms Version 9.8.29 : 2018.05.01
Modules Administration : App v1.1.0 : Dashboard v1.1.1 : EventProcesses : Feed : Input : CoreProcess : Schedule : Time : User : Visualisation
Server OS Linux 4.9.59-v7+
Host raspberrypi raspberrypi (127.0.1.1)
Date 2018-05-03 15:43:33 BST
Uptime 15:43:33 up 6 min, 3 users, load average: 0.45, 0.98, 0.62
HTTP Server Apache/2.4.25 (Raspbian) HTTP/1.1 CGI/1.1 443
MySQL Version 5.5.5-10.1.23-MariaDB-9+deb9u1
Host localhost (127.0.0.1)
Date 2018-05-03 15:43:33 (UTC 01:00‌​)
Stats Uptime: 394 Threads: 3 Questions: 21 Slow queries: 0 Opens: 19 Flush tables: 1 Open tables: 13 Queries per second avg: 0.053
Redis Version 3.2.6
Host localhost:6379 (127.0.0.1)
Size 130 keys (701.08K)
Uptime 0 days
MQTT Version 1.4.10
Host localhost:1883 (127.0.0.1)
Pi CPU Temp 54.23°C
Memory RAM Used: 39.82% Total: 875.78 MB Used: 348.72 MB Free: 527.06 MB
Swap Used: 0.00% Total: 100 MB Used: 0 B Free: 100 MB
Disk Mount Stats
/ Used: 67.00% Total: 6.86 GB Used: 4.6 GB Free: 1.93 GB
/boot Used: 50.62% Total: 41.15 MB Used: 20.83 MB Free: 20.32 MB
/mnt/data Used: 17.82% Total: 458.49 GB Used: 81.71 GB Free: 376.78 GB
PHP Version 7.0.27-0+deb9u1 (Zend Version 3.0.0)
Modules apache2handler : bz2 v7.0.27-0+deb9u1 : calendar v7.0.27-0+deb9u1 : Core v7.0.27-0+deb9u1 : ctype v7.0.27-0+deb9u1 : curl v7.0.27-0+deb9u1 : date v7.0.27-0+deb9u1 : dio v0.1.0 : dom v20031129 : exif v7.0.27-0+deb9u1 : fileinfo v1.0.5 : filter v7.0.27-0+deb9u1 : ftp v7.0.27-0+deb9u1 : gd v7.0.27-0+deb9u1 : gettext v7.0.27-0+deb9u1 : hash v1.0 : iconv v7.0.27-0+deb9u1 : json v1.4.0 : libxml v7.0.27-0+deb9u1 : mbstring v7.0.27-0+deb9u1 : mcrypt v7.0.27-0+deb9u1 : mosquitto v0.4.0 : mysqli v7.0.27-0+deb9u1 : mysqlnd : openssl v7.0.27-0+deb9u1 : pcre v7.0.27-0+deb9u1 : PDO v7.0.27-0+deb9u1 : pdo_mysql v7.0.27-0+deb9u1 : Phar v2.0.2 : posix v7.0.27-0+deb9u1 : readline v7.0.27-0+deb9u1 : redis v3.1.6 : Reflection v7.0.27-0+deb9u1 : session v7.0.27-0+deb9u1 : shmop v7.0.27-0+deb9u1 : SimpleXML v7.0.27-0+deb9u1 : sockets v7.0.27-0+deb9u1 : SPL v7.0.27-0+deb9u1 : standard v7.0.27-0+deb9u1 : sysvmsg v7.0.27-0+deb9u1 : sysvsem v7.0.27-0+deb9u1 : sysvshm v7.0.27-0+deb9u1 : tokenizer v7.0.27-0+deb9u1 : wddx v7.0.27-0+deb9u1 : xml v7.0.27-0+deb9u1 : xmlreader v7.0.27-0+deb9u1 : xmlwriter v7.0.27-0+deb9u1 : xsl v7.0.27-0+deb9u1 : Zend OPcache v7.0.27-0+deb9u1 : zip v1.13.5 : zlib v7.0.27-0+deb9u1 :

In there you will see the Redis server is in it’s own section and at version 3.2.6 and in the last “PHP - Modules” section, the Redis extension for PHP is listed as “redis v3.1.6” (listed means it’s enabled).

As for the Sync module, I think we should confirm redis is ok before attempting the sync module, there are many other bits to that process that could stand between a button press and a succesful download, not just redis. It needs to create a temp flag file, there needs to be a cronjob running to pick up on that flag and run a script which has to reach the server and be able to write it locally so there are many permissions and paths that may need checking if this is not a straightforward emonPi/emonSD.

So are you able to confirm redis is both installed and enabled using one or more of the above methods?

Since this is a fresh install and on a more exotic OS/machine and you are trying to use some very new software that hasn’t been widely tested, perhaps you should roll-back to 3.1.6, there are too many unknowns, if the module is enabled and the sync module still doesn’t work we don’t know if it’s due to a version 4.0.2 issue or not, at least with 3.1.6 we know that should work ok and reduces the unknowns to just an uncommon environment and an untested emoncms module.

Didn’t know the emoncms site had its own phpinfo page. I wrote my own!

Anyway, I have everything as per Paul’s checklist. So I’ll assume my redis is set up.

Just for info, on my Adminstration page Redis (server) is at 3.0.6 and PHP redis is at 4.0.1

Everything appears to work fine. But I haven’t used sync.

So your redis server is behind our normal version and your redis extension is ahead of our norm (ie server 3.2.6 and extension 3.1.6) I have no idea how or if that combo will work so please let us know how you get on in due course.

Thanks you for the comprehensive answer!

Emoncms and PECL reports redis 4.02 in the PHP section. PHP confirm redis. I’ll try the roll back

3.1.6 install OK. What do you make of this?:

Build process completed successfully
Installing ‘/usr/lib/php/20151012/redis.so’
install ok: channel://pecl.php.net/redis-4.0.2
configuration option “php_ini” is not set to php.ini location
You should add “extension=redis.so” to php.ini

I noticed that too. But I haven’t specified a version. This is what apt and/or pecl have installed.
I just ran an apt update and there no new redis-server for Ubuntu 16.04 on the repositories.

Yes - created a crontab in my user home directory.

Now I have installed redis 3.1.6 All Pauls check reports redis 3.1.6 is up and running and on top I did a reboot of the VM.
Unfortunately no progress on the sync tool:

I pushed the two missing download buttoms, the feeds appear in the feed list, but no data

Was that before or after the rollback? It still shows redis-4.0.2.

If you uninstalled and reinstalled it might have removed the file and enabled symlinks (steps 3 and 4 from my post above) and needs them re doing perhaps?

Perhaps the php.ini file needs to be edited for your particular environment? I cannot say as I do not use synology or VMs.

It’s not just the OS but a combination of the OS and the Architecture eg there might be a different version for a ARM device running Stretch to a amd64 device running Stretch etc

Sorry i have little or no experience with the sync module, I will see what @Trystan comes up with before I wade into that unknown :slight_smile:

@Lykke Could you try setting $homedir in your emoncms settings.php file to reflect your system and try again? If that does not work could you try running sync/sync_run.php directly

sudo php sync_run.php

It might be a path issue.

No luck with activating $homedir in settings.php

Running sync_run.php produces this output, guess you are onto something:

lykke@lykke-Standard-PC-i440FX-PIIX-1996:~/sync$ php sync_run.php
SYNC: Starting
PHP Warning: chdir(): No such file or directory (errno 2) in /home/lykke/sync/sync_run.php on line 8
PHP Warning: require(process_settings.php): failed to open stream: No such file or directory in /home/lykke/sync/sync_run.php on line 9
PHP Fatal error: require(): Failed opening required ‘process_settings.php’ (include_path=‘.:/usr/share/php’) in /home/lykke/sync/sync_run.php on line 9

I’ve changed /var/www/emoncms to /var/www/html/emoncms and set the $homedir to my home folder. That helped, but:

lykke@lykke-Standard-PC-i440FX-PIIX-1996:~/sync$ php sync_run.php
SYNC: Starting
SYNC: Home dir: /home/lykke
SYNC: Connected to Redis
{“action”:“download”,“local_id”:4,“remote_server”:“https://emoncms.org”,“remote_id”:xxxxx,“remote_apikey”:“yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy”}
Cannot open local metadata file
{“action”:“download”,“local_id”:3,“remote_server”:“https://emoncms.org”,“remote_id”:xxxxxx,“remote_apikey”:“zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz”}
Cannot open local metadata file

Looks like your almost there :slight_smile: In your last step can you try: sudo php sync_run.php

Fault 40 :slight_smile: Cleared that problem, but feeds are still not downloading…

lykke@lykke-Standard-PC-i440FX-PIIX-1996:~/sync$ sudo php sync_run.php
[sudo] password for lykke:
SYNC: Starting
SYNC: Home dir: /home/lykke
SYNC: Connected to Redis
lykke@lykke-Standard-PC-i440FX-PIIX-1996:~/sync$

Edit: Tried it again. If I push the “download” button in emoncms and then manually start the sync script, data is downloaded.

Great to hear that you have data downloading working now.

It would be great if you can get service-runner to work so that you dont have to run sync_run.php manually. Do you see anything in:

tail -f /var/log/service-runner.log

lykke@lykke-Standard-PC-i440FX-PIIX-1996:~/emonpi$ tail -f /var/log/service-runner.log
STARTING: /home/lykke/sync/emoncms-sync.sh > /home/lykke/data/emoncms-sync.log
sudo: no tty present and no askpass program specified
/home/lykke/emonpi/service-runner: line 25: /home/lykke/data/emoncms-sync.log: No such file or directory
touch: cannot touch ‘/home/lykke/data/emoncms-sync.log’: No such file or directory
COMPLETE: /home/lykke/sync/emoncms-sync.sh > /home/lykke/data/emoncms-sync.log

I created /home/lykke/data/ so it had a place to put the log file, assume it should be in /var/log/, but I’m not sure how to do that. This error persists: " no tty present and no askpass program specified"

Managed to get sync up and running without errors - problem seemed to be that the sync program wasn’t able to supply the required password. Probably a bit of a hack, but I ended up doing this:

Placing a file with mode 0440 in /etc/sudoers.d/myuser with following content:

myuser ALL=(ALL) NOPASSWD: ALL
Should fix the issue.

Do not forget to:

chmod 0440 /etc/sudoers.d/myuser

Great to hear, does that solve the service runner issue? Does the download work now without manually running sync_run?

As far as I can tell, everything now works as intended. Push the “Download” button and it will fetch the data, no need for manually running sync_run.

1 Like

I realise the sudoers modification is already in place on the pi of course :slight_smile:

sudo nano /etc/sudoers

Line is just below line #includedir /etc/sudoers.d

pi ALL=(ALL) NOPASSWD: ALL

I’ve added a note to the installation guide: sync/readme.md at master · emoncms/sync · GitHub