Admin display currently installed Emoncms modules

[pb66 (Paul)]

Can I tag a feature request to this thread?

I thought it would be good to show the additional emoncms-modules installed, We have the PHP modules shown but not emoncms.

It would be easy to simply list the content of the emoncms/Modules folder, but it would be nicer to either show only the additionally installed modules or better still to highlight the additionally installed non-core modules so it can be seen which are core and which have been added…

I use a simple bash script for updating emoncms that tests for the existence of a .git sub-folder in each folder in emoncms/Modules.

#!/bin/sh

EMONCMS_DIR=/var/www/html/emoncms
printf "\nUpdate emoncms....\n"
git -C $EMONCMS_DIR pull
for M in $EMONCMS_DIR/Modules/*
  do
    if [ -d "$M/.git" ]; then
      printf "\nUpdate emoncms/$(basename $M)....\n"
      git -C $M pull
    fi
  done

this same test could probably be used in PHP I’m guessing, I haven’t tried implementing anything yet, it’s on one of my many todo list’s but if @jon or @Andy_Taylor fancing giving it a go, please do.

I was thinking we could just see a “Modules” entry in the emoncms section below the current “Version” entry, just highlighting the additional Modules in bold would work well eg

admin . app . dashboard . device . eventp . feed . graph . input . myip . process . schedule . time . user . vis

Another approach would be to just hardcode the core modules into the function as my method of testing for a .git can be tripped up by manually added modules, in my example above “myip” incorrectly appears to be a core Module because it has no .git sub-folder, an alternative test to check for an entry in emoncms/.gitignore might also work but may also complicate things.

[Andy_Taylor]

PoC code has been written, seems to work ok, and due to doing ALL of it in PHP, it should also work on Windows hosts too (but this will need testing).

I’ll do a PR for in now.

[Andy_Taylor]

PR submitted - please make sure that this does what you want (more information in the PR) and that its tested on Windows servers - since its all written in PHP it should work perfectly on Windows also.

[Jon (Jon Murphy)]

Andy

I entered this command to grab the update for emoncms modules:

git pull https://github.com/AndyTaylorTweet/emoncms.git master

but I see this error:

Modules	
Notice: Undefined variable: emoncms_modules in /var/www/emoncms/Modules/admin/admin_main_view.php on line 247

 

EDIT: did I enter the correct command to grab your changes?

[Andy_Taylor]

Not sure how I managed that but I managed to miss a line of code out in my code I checked into GIT.
Have another try.

[Paul (Paul Reed)]

Now merged into the master git branch.

Although it should work fine in Windows as it’s written in PHP, we haven’t been able to test on that platform (I only deleted a Windows test install last night!!!)

Could a Windows user please confirm

Paul

[joyrider3774]

@Paul

Could a Windows user please confirm

works fine in windows as well although i don’t have redis so it isn’t displaying buffer / writer due to that i guess

[pb66 (Paul)]

Thanks for the speedy turnaround, I wasn’t expecting that.

I hope don’t mind, I have made a couple of changes after pulling in your changes, I have made all “Additionally installed Modules” bold as they are optional and most likely the focus of any interest in this info, one would hope the core modules (the ones installed automatically from the emoncms/emoncms repo) are all there but as modules become included or broken out to their own repo, errors can occur, so for support I think a full module list is more informative. as it doesn’t hide Modules that don’t have a .git folder.

[Andy_Taylor]

@pb66 - well this is what happens when you share a good idea…

Speaking of sharing, you want to share your changes? This was your idea after all, maybe we should look to include your version rather than mine…

[pb66 (Paul)]

For speed (and lack of php skills) I’m afraid I just hacked your code to fit, here’s the diff. I am only too happy to share, but it isn’t up to PR grade and wanted your thoughts before suggesting changes to what you submitted.

diff --git a/Modules/admin/admin_main_view.php b/Modules/admin/admin_main_view.php
index 593a67b..03e833b 100644
--- a/Modules/admin/admin_main_view.php
+++ b/Modules/admin/admin_main_view.php
@@ -125,12 +125,12 @@
   }

 $emoncmsModulesPath = substr($_SERVER['SCRIPT_FILENAME'], 0, strrpos($_SERVER['SCRIPT_FILENAME'], '/')).'/Modules';
-$emoncmsModuleFolders = glob("$emoncmsModulesPath/*/.git", GLOB_BRACE);                                                 // find all the subfolders containing a .git file
+$emoncmsModuleFolders = glob("$emoncmsModulesPath/*", GLOB_BRACE);                                                 // find all the subfolders containing a .git file
         foreach($emoncmsModuleFolders as $emoncmsModuleFolder) {                                                        // loop through the folders
         $emoncmsModuleFolder = str_replace($emoncmsModulesPath."/", '', $emoncmsModuleFolder);                          // clean up the formatting, removing the path from the $emoncmsModulesPath variable
-        $emoncmsModuleFolder = str_replace("/.git", '', $emoncmsModuleFolder);                                          // more clean up to remove /.git from the end of the output
-        if ($emoncmsModuleFolder == 'app' || $emoncmsModuleFolder == 'config' || $emoncmsModuleFolder == 'dashboard' || $emoncmsModuleFolder == 'graph' || $emoncmsModuleFolder == 'wifi') {$emoncmsModuleFolder = "<b>".$emoncmsModuleFolder."</b>";}
-        if (isset($emoncms_modules)) { $emoncms_modules = $emoncms_modules.", ".$emoncmsModuleFolder; } else {$emoncms_modules = $emoncmsModuleFolder;} // add the commas as appropriate
+//        $emoncmsModuleFolder = str_replace("/.git", '', $emoncmsModuleFolder);                                          // more clean up to remove /.git from the end of the output
+        if ($emoncmsModuleFolder != 'admin' && $emoncmsModuleFolder != 'eventp' && $emoncmsModuleFolder != 'feed' && $emoncmsModuleFolder != 'input' && $emoncmsModuleFolder != 'process' && $emoncmsModuleFolder != 'schedule' && $emoncmsModuleFolder != 'time' && $emoncmsModuleFolder != 'user' && $emoncmsModuleFolder != 'vis') {$emoncmsModuleFolder = "<b>".$emoncmsModuleFolder."</b>";}
+        if (isset($emoncms_modules)) { $emoncms_modules = $emoncms_modules.', '.$emoncmsModuleFolder; } else {$emoncms_modules = $emoncmsModuleFolder;} // add the commas as appropriate
         }

 ?>

I still think using the existence of a .git sub-folder could still have some value, in my screenshot example I think myip should appear differently, perhaps in bold/italics to stir interest when seen on the admin page.

For example the recent transition to a separate dashboard module caused a little confusion and being able to tell if the dashboard module folder was present or not, and if so whether it was a git repo or not would have been handy.

But it’s a small use case and probably not worthy of a lot of effort, just to italic the occasional non-core and non-git folder.

[Andy_Taylor]

So the question is - how do we want this to work…

I use the bone stock setup, since it does all I want, so I don’t have any modules installed - I guess what we need is some feed back from everyone about what exactly they want to show here;

Every folder with a .git
Every Folder
What should be bold / not bold / Italic / not italic

[joyrider3774]

about .git folders what happens if users use the download function on github and not git checkout ? are .git folders included as well then ? I’m guessing that they are not in the repo so it will only display the modules if users used git to get emoncms and their (optional) modules and not when using the download link like https://github.com/emoncms/emoncms/archive/master.zip

edit: just checked it does not contain them. So in my opinion it should display all folders and not check on .git folders to be compatible with the download function …

about bold: you could create a (hardcoded) list of core modules or optional modules and depending on the list verify the directory name with the one for the hardcoded values and display the optional ones (graph, dashboard etc) in bold. Or you can do it the otherway around and keep a list of core modules and display all others in bold

another edit: What about using color codes ? like
green = emoncms core modules (…)
blue = emonpi core or related modules (…)
orange = optional emoncms modules (graph, dashboard)

or if using color codes and displaying all folders use green to display as installed and red or orange as not installed. italic can also be used for this if wanting to use color codes as above. this would only work though if you keep a full list hardcoded in the code to compare against

[joyrider3774]

@Andy_Taylor there is a bug in the code … It is possible that is not set here producing this error below. It needs an isset verification i guess … this happend because my dev install doesn’t have any optional modules installed (has no .git dirs) at least i guess havent looked into it in detail …

( ! ) Notice: Undefined variable: emoncms_modules in C:\wamp\www\emoncmsdev\Modules\admin\admin_main_view.php on line 249
Call Stack
#	Time	Memory	Function	Location
1	0.0013	171600	{main}( )	...\index.php:0
2	0.0165	416272	controller( )	...\index.php:156
3	0.0178	445472	admin_controller( )	...\core.php:64
4	0.0179	446056	view( )	...\admin_controller.php:26
5	0.0193	535760	include( 'C:\wamp\www\emoncmsdev\Modules\admin\admin_main_view.php' )	...\core.php:74

[Andy_Taylor]

@joyrider3774

emoncms_modules

PR submitted: Bugfix for emoncms modules by AndyTaylorTweet · Pull Request #585 · emoncms/emoncms · GitHub

[joyrider3774]

thanks only noticed it by accident though if i had not used the dev build i was running i wouldn’t have noticed it

[pb66 (Paul)]

Hi Andy, I hadn’t answered straight away, so as to see what comments other users made.

@Andy_Taylor

I use the bone stock setup, since it does all I want, so I don’t have any modules installed

I assume you have an emonPi or an emonSD image, is that right? if so that would actually include the app, config, dashboard and wifi additional modules, installed by default. If you ran my version on your system you should see

admin, app, config, dashboard, eventp, feed, graph, input, process, schedule, time, user, vis, wifi

This helps users understand the relationship between emoncms and the “optional but pre-installed modules” so that questions like “do you have the apps module installed?” make more sense and don’t attract answers like “No, I have not installed any additional modules” without previously declaring they have an emonPi/emonSD and therefore it is included.

Whether the “dashboard” is bold or not tells us whether the transition from an in-built module to a separate module has been made on your system, although most users are past this point now it would have been useful at that time and demonstrates it’s usefulness in the future when modules get moved/altered or new ones appear.

Also as Joyrider has mentioned, the absence of .git sub-folders in folders downloaded and unzipped into place could also cause confusion if they are not seen, seeing “apps” not in bold would tell us that this was installed by unzipping a download and at a glance answer questions like why it won’t update via a git pull.

It’s also not so long ago that users were still installing the raspberrypi module which is no longer supported and if installed will cause issues for emonhub over accessing the serial port. Also users of the events module or packetgen may not be aware they have been retired and be so used to just cloning them when installing a fresh emoncms they do not visit the repo’s to find out not to use them. I would like to see all these stand out in bold (or bold italic if downloaded) so it is easier to spot the errors, IMO highlighting the “stock” and most unlikely to be at fault folders, distracts attention from the ones we need to check.

IMO making app, graph and dashboard bold on every single emoncms install regardless of build just because they are some of the default modules included on the emonSD card, even if not using the emonSD has no value and highlights an irreverent difference that could even cause confusion.

The use of color as suggested by Joyrider, isn’t a bad idea, but it sounds like a lot of work and may not translate well if we expand the admin “status” page to a download log type file, both bold and bold italics could still be used in a txt file.

However the main thing is that all emoncms/Modules contents are listed, the formatting is less important and perhaps even remove it all together and keep with plain text if it causes confusion.

[Andy_Taylor]

@pb66 - after taking another look at this - and reading through your post; it looks like the most useful option is to list all modules - regardless of a .git file. Identifying stock modules seems to be useful too - so;

My current work makes stock modules normal text and adds bold to modules not part of stock emoncms.

[pb66 (Paul)]

Sounds good to me!
From your screenshot I can see symlinked modules are another case that wouldn’t have a .git sub-folder, so your “backup” and “postprocess” modules would not have previously been listed.

[borpin (Brian Orpin)]

Perhaps one of the things about the modules is what are essential and what are optional. Also remember that not everyone runs an EmonPi so wifi is not essential . Including the installed version would have benefits when trying to offer support.

I also wonder if this feature could be extended to simplify upgrading.

If this feature (or an upgrading feature) could bring in a list of modules available on GitHub, enable the user to select the branch (stable master etc) per module, and then offer to install the module or inform the user if there is an update and then allow the user to do the pull from that interface, it would encourage users to keep modules and the core up to date.

I think one of the blocks to users updating (and thus keeping the majority on a similar version - which in itself would have support benefits) is that it is never that clear what needs updating along with a lack of knowledge of GitHub. Whilst I understand version control (and rolled my own nearly 20 years ago using PERL), how to actually use GitHub has, as yet, eluded me!

It also requires some developer discipline. It seems ‘stable’ has been rather unstable recently