Community
OpenEnergyMonitor

Community

Setup Sync with Docker

Tags: #<Tag:0x00007f6e043e5de0> #<Tag:0x00007f6e043e5d18>

Hello,

Working on migrating my data from emoncms.org to a docker container on my nas. I have the container up and running with docker compose (https://github.com/emoncms/emoncms-docker), but I dont see it having the sync module. What would be the steps to get this added in?

Thank you in advance for any help.

Regards,

Dan

It looks as if @beaylott hasn’t noticed this post yet. He should now get an email notificaton, so hopefully he’ll be along soon.

Thanks @Robert.Wall , I dont monitor this forum very closely.

The solution is to add a line in the Dockerfile to clone the sync repo e.g.

… then rebuild the Docker image for emoncms using that modified Dockerfile by whatever means.

Hi Ben, the sync module is not as staightforward to install as the graph or dashboard modules. The Sync module requires cloning to a location outside of the /var/www path and then a sub-folder of that repo is symlinked into emoncms/Modules. There are also some other requirements to make this module work such as a “service-runner.sh” script (part of the emonpi repo not currently standalone) and a cron job to constantly check for emoncms flags to trigger that service-runner.sh.

See the module repo for further info

and also there are a few posts here on the forum discussing getting the sync module installed and working, but it’s spread out across several threads https://community.openenergymonitor.org/search?q=sync%20module rather than in one place.

Just thought I should mention it because if @flemingdp tries what you suggest, it definitely will not work.

ok in that case the solution is probably going to require use of crond and/or supervisord in the container in order to use the service-runner.sh . Or it needs a separate container. All the Docker stuff needs overhauling anyway so I might have a look when I try that.

1 Like

Would love to see the containers consolidated, but that may be challenging. Happy to test. Let me know. Thank you.

Hi @beaylott Is there anything I can do to help with this? I would love to start sycing my historical data down.

After spending two nights trying to get this working for myself and trying to wrap my head around docker, this seems to be the simplest solution:

  1. Edit the Dockerfile and add following lines (in place of that one line that @beaylott suggested):
RUN git clone https://github.com/emoncms/sync.git /home/pi/sync

RUN chgrp www-data /home/pi
RUN chgrp -R www-data /home/pi/sync
RUN ln -s /var/www/html /var/www/emoncms
RUN ln -s /home/pi/sync/sync-module /var/www/emoncms/Modules/sync

The reason I decided to place it in /home/pi even though there is no pi user is because that is the default location for $HOMEDIR on which the sync depends and it will save one step of modifying the settings.php. The other reason was that putting in /root did not seem right and would require modifying access rights to /root.

  1. after you save the file, you will need to rebuild the container:
sudo docker-compose up -d --build
  1. reload the webpage. At least use Ctrl+F5 if not close and reopen the tab.

  2. Before you try to use Sync. Go to Setup->Administration and do Update&Check to update the database.

  3. Follow the instructions on Sync git page on how to configure it.

Now comes the biggest caveat: I spend about 6 hours researching how to implement cron in docker. And although it is possible, it would over complicate things beyond simple instructions if implemented correctly. That leaves two options:

  1. if you are just trying to transfer data from emoncms.org and this a one time deal, you can run the download manually by getting into the container using
sudo docker exec -t -i emoncmsdocker_web_1 /bin/bash

Once in there you need to run

php /home/pi/sync/sync_run.php
  1. if you want to do this on schedule, you can create a cron job on your host that executes that script inside the container. I have not done it myself but the crontab should look something like this
* * * * * docker run --rm emoncmsdocker_web_1 /home/pi/sync/emoncms-sync.sh

I hope that helps

1 Like