Emoncms self contained docker container and HomeAssistant Addon

Hi,

I have a pipeline producing docker images with the complete stack onboard : mariadb, redis, apache, php, mosquitto and the main modules (sync, postprocess, backup)

So it is a kind of virtual twin for the physical emonpi, but without emonhub…

for an ephemeral test :

docker pull alexjunk/emoncms
docker run --rm -it alexjunk/emoncms

To get the container IP address : docker network inspect bridge

To send mqtt datas from the host if 172.17.0.2 is the container IP : mosquitto_pub -h 172.17.0.2 -u "emonpi" -P "emonpimqtt2016" -t 'emon/test/t3' -m 15

To run on a local network, use docker run -p :

docker run --rm -it -p 8080:80 alexjunk/emoncms

if 192.168.1.13 is the host IP, then emoncms can be reached via http://192.168.1.13:8080

The previous commands use the latest tag but you can find various images on the HUB

It is easy to use a client container to run a client app using emoncms databases or sending datas to it.

You just have to share the same network namespace :

docker run --rm --name=emoncms -it alexjunk/emoncms

the Dockerfile for a python client could be something like that :

ARG BUILD_FROM=alpine:3.16

FROM $BUILD_FROM

RUN set -x;\
    apk update && apk upgrade;\
    apk add --no-cache python3 py3-pip;\
    python3 -m pip install --no-cache-dir mysql-connector-python redis paho-mqtt==1.6.1

RUN set -x;\
    wget https://raw.githubusercontent.com/dromotherm/sandbox/master/emoncms/client_dockerfile/test.py

build :

docker build -t emoncms_client .

use :

docker run --rm --network=container:emoncms -it emoncms_client

Once connected :

python3 test.py

More on https://docker-emoncms.github.io

[Revised on request of @alexandrecuer – Moderator (RW)]

5 Likes

Nice! thanks for sharing this @alexandrecuer! can see that being useful

2 Likes

(post deleted by author)

1 Like

This is potentially really useful. More so than each part running in separate containers. Not for me though, as I’m a PVE user not Docker.

1 Like

(post deleted by author)

4 Likes

Here is a link to an application server giving the ability to launch emoncms containers :slight_smile:

Easy to test new version of modules…

I think this single container is the way to go for the HomeAssistant add-on. The exception I’d suggest is the MQTT Broker as very often, those using HA have a separate Broker running.

I’m sure there was a discussion about this but I can’t seem to find it. @glyn.hudson did an initial add-on IIRC.

Just discovering HA, add-ons seem to be nothing more than containers…

HA uses the s6 overlay to run multiple services in the same container, i think it is a cleaner approach than supervisord

1 Like

@TrystanLea @borpin : I did some more work on the container approach.
I’ve explored the s6-overlay to run multiple services in the same container, which seems inspiring.

I went for multiarch build with alpine 3.16 and it is working quite well. Alpine is a bit different so you cannot use the debian tools like phpenmod, a2enmod, a2ensite. Moreover there is no www-data user only an apache one, and I decided to run all the workers (feedwriter, service-runner, emoncms_mqtt) with the apache user, dont know what you think of that ?

After everything is a matter of rights but finally the Dockerfile is quite simple, aside it written in my own style :wink:

I have to spend more time on the data persistence (none for the moment), which can be solved (I think) with a oneshot s6 service and will also be the occasion to solve the problem of the database initialisation, that I treated in a rather dirty way

so if you want to test :

docker pull alexjunk/emoncms:alpine3.16

it will work with aarch64/amd64/armv7

I’ve tested with homeassistant on a pi3, I could managed to make it work as an addon, with a very simple config.yml file, giving an image name so that the home-assistant fetches on docker and does not build locally :

name: emoncms
description: The webapp for energy monitoring and more
version: alpine3.16
image: alexjunk/emoncms
slug: emoncms
init: false
arch:
  - armv7
  - aarch64
  - amd64
ports:
  80/tcp: 8002
  1883/tcp: 9883

I’ve crafted a small repo with some Dockerfiles and github actions configured to produced the multiarch images…

1 Like

That sounds really promising. Being able to run as an Add-On to HA would be magic.

Not sure what you mean by this :slight_smile:

1 Like

Actually the docker solution to achieve data persistence is to use volumes.

So if you create volumes it is OK, one for mariadb, one for the timeseries datadirs…volumes are initially fed from the container file tree : If you start a container which creates a new volume, and the container has files or directories in the directory to be mounted such as /app/, Docker copies the directory’s contents into the volume.

But in home-assistant add-ons, there is no support for volume from what I’ve understood…

So I’ve read how they proceed for the mariadb addon :

They use the data folder :

and :

and also :

Yes, thought they might.

As I say, a standalone addon would be magic! With the ability to automate the backup, it would become a very attractive option. Potentially, I’d consider getting rid of my Proxmox setup.

There really isn’t any other option for collecting longer term data in HA, other than Grafana really which is a bit of a pest to then use (in comparison to emoncms IMHO - other opinions are available), so it could become quite popular (@TrystanLea @glyn.hudson).

2 Likes

You mean automating backup with docker tools and not use the backup module, that’s it ?

Actually, I could make work the import script of the module without major changes, just removing all the trailing sudo, but export script is not yet working

No, with HA tools either an automation or another addon such as Google Drive Backup (which I use).

It should be working fine now.

To install, add the https://github.com/Open-Building-Management/emoncms to the repositories of the add-on store of Home-Assistant :

then install :

@TrystanLea : I had to modify some of the modules scripts, in order they work on alpine, very light mods, mostly removing sudo :
cf https://github.com/Open-Building-Management/emoncms/blob/5c8fa84c10eab4d3aa40782310f1e7faed7f85a2/Dockerfile#L132C2-L138C71

3 Likes

That looks great! thanks @alexandrecuer!

@alexandrecuer - taken the plunge and installed it.

First thing is MQTT. Whilst strictly for a ‘standalone’ system it is required, in reality, I want to be able to point emoncms at a different broker.

Can the MQTT IP Address be configurable?

Any thoughts on exposing the settings file? For instance Log Level?

[edit]
Configure for SSL - I run my HA on SSL locally :frowning:

Yes it possible I think
We have to add first an option MQTT_HOST in the emoncms/config.yaml at main · Open-Building-Management/emoncms · GitHub
Then it requires to modify in consequence emoncms_pre.sh, when it opens /data/options.json (HA creates this file to gather all the options of the add-on)
emoncms/emoncms_pre.sh at 5c8fa84c10eab4d3aa40782310f1e7faed7f85a2 · Open-Building-Management/emoncms · GitHub
and add a line to inject host = '$MQTT_HOST' in the settings.ini generated at runtime

To modify log level is possible also, same manner

For SSL, I did it once adding crt_file, key_file, crtchain in the container at buildtime, but I guess it must be different approach in HA…must understand first how they do…

1 Like

@alexandrecuer - where are the data files created?

It is in the /data folder of the host. From what I’ve understood, you don’t have access to the whole system when you connect to HA in ssh. As they say, SSH access through the SSH add-on (which will give you SSH access through port 22) will not provide you with all the necessary privileges

Debugging the Home Assistant Operating System | Home Assistant Developer Docs

Haven’t tried all this for the moment…

1 Like