ISC Konstanz summer contributions

Hi guys,

it’s been a while and I finally found the means to make this contributions, as annoyingly some of the changes require each other and getting everything to a point that its comfortably installable and documented took longer than I liked.
But we simply need to contribute our stuff in a hopefully not too big flood, as getting our fork up-to-date is getting increasingle merge-conflict-prone^^
“We”, thats the ISC Konstanz e.V., a small private solar research institute in southern germany that’s heavily using emoncms since 2015 (as a quick reminder :wink:).

I’ll use this topic as a summary in several sections, as our contributions consist of 5 different repositories by now.

As I tried to push the emonmuc (a module for OpenMUC) project once a while ago, but it simply was not very setup-friendly, I was increadibly happy for the new EmonScripts, to now optionally include the emonmuc installation. For which I also created a pull request. Our separate isc branch provides an installation of all isc-related modules for which I am creating separate pull-requests. I hope this eases the review n stuff a little :slightly_smiling_face:

The installation process with this specific fork branch is shown in the emonmuc readme.
But in the end it mostly promts to run

wget -O - https://raw.githubusercontent.com/isc-konstanz/EmonScripts/isc/install/init.sh | bash

The readme also includes a few guides to explore the main features of the emonmuc framework once installed:

Emoncms

The emoncms contributions consist of mainly small changes like fixed indents and systax for the new input_view.php/.js and feedlist_view.php. Generally, I tried to improve the javascript code consistency a little to use more camelCase, as conventions encourage. I additionally changed the collapse-indicators to be chevron icons, as I simply preferred those visually.
Also, I improved the themes’ style sheet functionalities a little. E.g. to make less use of !important (still a lot though) and include a few things such as the color of top affix background color in the new views.

As we added a total of 3 new menu entries, I increased the menu order values of most modules, as I arrogantly didn’t like the input menu order => 0 :slight_smile:

Further, the sidebar now allows to be differently colored, depending on the theme used:

At last, I updated the MysqlTimeseries to include some new features, as we use those feeds as a shared resource e.g. for forecasts and EMS a lot.
To improve this, we simply added the possibility to optionally use a separate database for the mysql feeds, configurable via the settings.php:

$feed_settings = array(
    'mysql'=>array(
        'prefix' => 'feed_',
        'generic' => true,
        'database' => 'emondata',
        'username' => 'emondata',
        'password' => '<password>'
    )
);

If nothing is configured, the engine will behave as before.
Additionally, a different prefix can be configured. Alternatively, by setting generic and optionally prefix false, the feed table name may be configured in the inputs process configuration dialog.

App Module

As shown in the pictures above, we added a new feedin app, that primarily caters to the needs of german EEG feed-in tariffs.

But doing so, I added some new object oriented graph and data cache functionalities that try to be fast and resource efficient.

  • Modules/app/Lib/feed.js is now a class that holds the apikey and provides async calls for all api paths.
  • Modules/app/Lib/data.js provides a cache for requested feed data and allows the aggregated iteration of several feeds at once. It also allows and aggregates the values of unevenly spaced engines like PHPTIMESERIES.
  • Modules/app/Lib/graph.js is the complete graph shown in the images above and could be used and enhanced in other apps.

The feed selection in the configuration view now also groups feeds by tags - and hence allows the existence of several feeds with the same name but different tags.

light.js now also includes a light-colored configuration UI. Any app that includes the light style sheet hence will be shown with a less dark config:

Device Module

The major new feature here is the thing functionality, as already once mentioned in an older forum thread.

The thing view adds an OpenHAB-like thing/item/channel functionality for devices, providing an entry-point to control e.g. Smartplugs or any communication over ModBus, M-Bus, DLMS/COSEM, IEC 61850 (generally smartmeters or EVs) or whatever, with only emoncms as a central GUI.

This can be achieved with modules providing a *_thing.php file and an additional “items” tag in templates:

"items": [
    {
         "id": "enable",
           "type": "Switch",
           "label": "Enable",
           "mapping": {
               "ON": { "channel": "foo_enable", "value": true },
               "OFF": { "channel": "foo_enable", "value": false }
           },
           "input": "foo_state"
     }
]

At the moment the necessary function public function set_item($itemid, $mapping) in device_thing.php is only a placeholder that directly writes a value into an input, but I thought this could be filled e.g. with a more general MQTT command usable for EVSE or Smartplugs. I’m not very experienced with MQTT though^^
The $mapping argument will be anything configured in the item section though and is completely customizable. A MQTT item could e.g. get something passed like { "path": "uri/to/state", "json": "{value_to_set:true}" } if configured in the template.

We use the template functionality especially to configure OpenMUC devices and channels, as this adds a quite large layer of complexity and configuration-possibilities that can be just bothersome.
Device templates hence configure a variety of Metering devices and stuff we use regularly like for example RPi output pin control.
Hence, I arbitrarily updated the grouping scheme of the device sidebar, to have a more generic category system.

Graph Module

The graph module contains just very small visual adjustments and an increased menu order value.
For the light sidebar it was necessary to adjust some small lines in the style sheet and I further changed the header buttons to use glyphicons and the text to be bold, as I personally thought those prettier.

Also, I removed the style="margin-top:1px" of all checkboxes of the graph view, as all checkboxes were slightly offcentered for me.




Well, I hope I didn’t forget too much and I am terribly sorry for this being such big pull-requests as I know this does not make reviewing anything easier.

If you like the general concept, I’d still be inclined to change the main projects repository to something more public/oem related like https://github.com/emoncms/emonmuc :wink:

1 Like

This looks epic @Adminde! I will look through your different pull requests and get back to you :slight_smile:

Yes, looks very impressive work.

I’m interested to learn more about emonMuc, are many meters IEC 62056-21 compatible? It’s really awesome how you can extract consumption & solar PV data directly from the meter:

Are IEC 62056-21 compatible meters easy to come by? Do you have a list of compatible meters? Do you have a source for the IR reader/writer sensor?

To be honest, I do not have the slightest idea :slightly_smiling_face:
For other contries as Germany at least^^ here it is prescribed by law for the operators to use electricity meters with an optical “INFO” interface with either an IEC62056-21 or a SML (Smart Meter Language) protocol available. It still can be quite a hassle though, as both protocols may have alternating baud rates configured etc.
Which is why I was hoping to expand the list of available device templates for that :wink:

But as an example, the local grid operator in my hometown uses mainly two differend manufacturers and for “normal” residential buildings these two models:

1 Like

And to be honest, last time we just ordered a bunch of these here from amazon.
I don’t know how this will be in the UK, but Smart Meter IR Probe should be something to search for. Here, the optical RS485 interface is alternatively called D0 but I don’t know how about other countries