Emonhub: inactive dead

m2ts · 27 March 2021 12:29

Hello

Situation: I would like to use emoncms with 2x SDM120 and 1xSDM630. Those are MID certified modbus kWh meters. At the moment I use a simple setup with 1x SDM120, connected to a socket and where I can plug in a lamp or … At the moment I also use a laptop (Lubuntu 20.04.2 LTS) and a RS485 USB adapter connected to it.

The steps seem to be emoncms, emonhub (to be able to use the SDM120 interfacer) and reading modbus. For emoncms I tried

The docker way. The setup of emoncms was very fast, but as far as I understand: emonhub is not included and I don’t know much on docker, so I left that path.
The emonscripts way. There happens só much that I left that path too. I reinstalled my Lubuntu, to be sure that nothing got in the way.
The step-by-step approach. I needed to change some things, but I managed and got a working emoncms where I could log in. This seems to be my favorite approach :-).

Next step was to install emonhub. I followed these steps, but I can’t get it to work:

http://localhost/emoncms/ > admin > services: emonhub is not mentioned.
Executed ‘sudo systemctl enable emonhub.service’, no errors.
Reload admin-page: now emonhub is mentioned, but ‘inactive dead’.

And now I’m stuck. I executed ‘git clone GitHub - openenergymonitor/emonhub: Python service linking and decoding input to MQTT & Emoncms’ in my $HOME-folder. Could that be a reason? Hopefully somebody has some ideas…

borpin · 27 March 2021 18:00

I’m not really clear how you are setting this up. Is it in a container? I don’t think anyone has tried to setup emoncms on Lubuntu before.

Note the step by step is archived and should not be used.

There are instructions for installing on Ubuntu within emonscripts https://github.com/openenergymonitor/EmonScripts/blob/master/install/readme.md#ubuntu

Are you trying to run emonhub in a container or native?

Again, I am not sure if anyone has done it on a laptop before.

Did you run the install script for emonhub?

Is there anything in syslog?

[edit]

This does not start a service, it simply enables it at boot.

m2ts · 29 March 2021 18:38

Thank you @borpin! It’s running native and not in a container. I did the emonscripts-way, but it didn’t work and there happens so much that I found debugging difficult. Then I tried “the archived way” and I found it more easy to understand what happens… with the exception of emonhub .

http://localhost/emoncms/ works and I can logon.
sudo systemctl status emonhub → Unit emonhub.service could not be found.
sudo systemctl start emonhub → Failed to start emonhub.service: Unit emonhub.service not found.
cat /etc/systemd/system/emonhub.service → see below
sudo /usr/local/bin/emonhub/emonhub.py --config-file=/etc/emonhub/emonhub.conf --logfile=/var/log/emonhub/emonhub.log → command from the emonhub.service
cat /var/log/emonhub/emonhub.log → see below.
http://localhost/emoncms/admin/view doesn’t mention emonhub as one of its services (only redis-server and mosquitto).

As far as I understand I can start emonhub “manually” (see number 5), but it’s not registered as a service in Ubuntu and/or emoncms. But if that’s the case, I don’t understand how I can fix it.

The emonhub.service:

$ cat /etc/systemd/system/emonhub.service 
[Unit]
Description=emonHub data multiplexer
# The config file lives in /etc/emonhub/emonhub.conf
# The log file lives in /var/log/emonhub/emonhub.log
Requires=var-log.mount
After=var-log.mount network.target

[Service]
User=emonhub
ExecStart=/usr/local/bin/emonhub/emonhub.py --config-file=/etc/emonhub/emonhub.conf --logfile=/var/log/emonhub/emonhub.log
PermissionsStartOnly=true
ExecStartPre=/bin/mkdir -p -m 0775 /var/log/emonhub/
ExecStartPre=/bin/chgrp -R emonhub /var/log/emonhub/
Type=exec
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target

The log:

$ cat /var/log/emonhub/emonhub.log
2021-03-29 20:12:02,803 INFO     MainThread EmonHub emonHub (emon-pi variant) v2.1.5
2021-03-29 20:12:02,804 INFO     MainThread Opening hub...
2021-03-29 20:12:02,804 ERROR    MainThread [Errno 2] could not open port /dev/ttyAMA0: [Errno 2] No such file or directory: '/dev/ttyAMA0'
2021-03-29 20:12:02,804 ERROR    MainThread Unable to create 'RFM2Pi' interfacer: 'bool' object has no attribute 'write'
2021-03-29 20:12:02,805 WARNING  MainThread Setting emoncmsorg apikey: obscured

borpin · 29 March 2021 21:43

You could have asked.

Likely a lot of the paths etc are wrong as that has all changed.

Assuming you are trying to use the serial port, you need to modify this as well.

I’m not exactly sure what data source you are trying to read with emonhub.

The standard configuration file will not work.

m2ts · 30 March 2021 04:08

As a first step, I don’t think anything is wrong with emoncms, but only with the emonhub service. I think I need that service to integrate 2x SDM120 and 1xSDM630. Those are MID certified modbus kWh meters.

For me, the probleem seem to be:

Strange, because it can be found at /etc/systemd/system/emonhub.service. And all the paths seems ok to me.

borpin · 30 March 2021 09:05

Try and start the service then check syslog.

Did you run the emonhub installer (the install script in the repo)?

[edit]
It may not start as the config is not correct - refers to a serial port that does not exist.

[edit2]
If you did not run the installer, you will need to explain precisely what commands you executed.

A better way will be to git clone to /opt/openenergymonitor/emonhub/ and run the install.sh file.

m2ts · 30 March 2021 17:47

I installed everything from scratch again by using my previous steps, so that I have an up-and-running emoncms. But I didn’t install emonhub yet. Like I have written in my first post, last time I followed these instructions to install emonhub. Namely:

sudo apt-get update
sudo apt-get install -y mosquitto
sudo nano /etc/mosquitto/mosquitto.conf --> to set `persistence` to false
git clone https://github.com/openenergymonitor/emonhub.git
cd emonhub
git checkout stable
sudo ./install.sh
journalctl -f -u emonhub --> to see the log

What should I better do in a different way to install emonhub this time? Does my working directory need to be /opt/openenergymonitor/emonhub/? Last time I did the git clone command from my $HOME.

borpin · 30 March 2021 22:11

Did you check the syslog as I suggested (as you start the service)?

journalctl -f -u emonhub

m2ts · 31 March 2021 04:20

Yes, I did check syslog and I removed the [[RFM2Pi]]-interfacer completely from /etc/emonhub/emonhub.conf. But to no reveal. Your last suggestion was to git clone to… and that’s why I started from scratch again. In that way I was sure that I could do a clean install again.

So, at the moment I didn’t do a emonhub install, as I wondered if my posted install steps are ok. Does my working directory need to be /opt/openenergymonitor/emonhub/ (I don’t think it matters, but I’m not sure)? Last time I did the git clone command from my $HOME.

I also found the topic ‘Installing Emonhub on a PC’ where @thierry could install emonhub on a pc after some tweaking. Unfortunately the tweaking itself isn’t mentioned.

borpin · 31 March 2021 07:22

Ok, so ideally yes create and use that target directory. Those seem to be the install instructions on GitHub so give it a go.

It might be a permissions thing.

When you start it, check syslog.

m2ts · 31 March 2021 14:43

The steps I did:

sudo apt-get update
sudo apt-get install -y mosquitto
sudo nano /etc/mosquitto/mosquitto.conf --> to set `persistence` to false
sudo mkdir /opt/openenergymonitor
cd /opt/openenergymonitor
sudo git clone https://github.com/openenergymonitor/emonhub.git
cd emonhub
sudo git checkout stable
sudo ./install.sh

Some strange things (details further on):

ERROR: launchpadlib 1.10.13 requires testresources, which is not installed.
Installing collected packages: paho-mqtt, pybluez, py-sds011, six, pymodbus, sdm-modbus
  Attempting uninstall: six
    Found existing installation: six 1.14.0
    Not uninstalling six at /usr/lib/python3/dist-packages, outside environment /usr
    Can't uninstall 'six'. No files were found to uninstall.
  Attempting uninstall: pymodbus
    Found existing installation: pymodbus 2.1.0
    Not uninstalling pymodbus at /usr/lib/python3/dist-packages, outside environment /usr
    Can't uninstall 'pymodbus'. No files were found to uninstall.
Failed to restart emonhub.service: Unit var-log.mount not found.

Detailed output from sudo ./install.sh
(I just hit ‘enter’ on the raspberrypi question)

sudo ./install.sh
Apply raspberrypi serial configuration? 1=yes, 0=no: 
You entered 

Geraakt:1 http://archive.ubuntu.com/ubuntu focal InRelease
(...etc...)
De volgende extra pakketten zullen geïnstalleerd worden:
  libexpat1-dev libpython3-dev libpython3.8-dev python-pip-whl python3-attr python3-automat python3-constantly python3-dev
  python3-distutils python3-hamcrest python3-hyperlink python3-incremental python3-lib2to3 python3-openssl python3-pyasn1
  python3-pyasn1-modules python3-serial-asyncio python3-service-identity python3-setuptools python3-twisted python3-twisted-bin
  python3-wheel python3-zope.interface python3.8-dev zlib1g-dev
Voorgestelde pakketten:
  pkg-config python-attr-doc python-configobj-doc python-openssl-doc python3-openssl-dbg python3-wxgtk3.0 | python3-wxgtk
  python-setuptools-doc python3-tk python3-pampy python3-qt4 python3-wxgtk2.8 python3-twisted-bin-dbg
De volgende NIEUWE pakketten zullen geïnstalleerd worden:
  bluetooth libbluetooth-dev libexpat1-dev libpython3-dev libpython3.8-dev python-pip-whl python3-attr python3-automat python3-configobj
  python3-constantly python3-dev python3-distutils python3-hamcrest python3-hyperlink python3-incremental python3-lib2to3 python3-openssl
  python3-pip python3-pyasn1 python3-pyasn1-modules python3-pymodbus python3-serial python3-serial-asyncio python3-service-identity
  python3-setuptools python3-twisted python3-twisted-bin python3-wheel python3-zope.interface python3.8-dev zlib1g-dev
0 opgewaardeerd, 31 nieuw geïnstalleerd, 0 te verwijderen en 2 niet opgewaardeerd.
Er moeten 10,0 MB aan archieven opgehaald worden.
Na deze bewerking zal er 46,3 MB extra schijfruimte gebruikt worden.
Ophalen:1 http://archive.ubuntu.com/ubuntu focal/universe amd64 bluetooth all 5.53-0ubuntu3 [3.148 B]
(...etc...)
Extraheren van sjablonen uit pakketten: 100%
Voorheen niet geselecteerd pakket bluetooth wordt geselecteerd.
(Database wordt ingelezen ... 281764 bestanden en mappen momenteel geïnstalleerd.)
Uitpakken van .../00-bluetooth_5.53-0ubuntu3_all.deb wordt voorbereid...
(...etc....)
Instellen van python3-attr (19.3.0-2) ...
(...etc...)
Bezig met afhandelen van triggers voor man-db (2.9.1-1) ...
Collecting paho-mqtt
  Downloading paho-mqtt-1.5.1.tar.gz (101 kB)
     |████████████████████████████████| 101 kB 2.9 MB/s 
Requirement already satisfied: requests in /usr/lib/python3/dist-packages (2.22.0)
Collecting pybluez
  Downloading PyBluez-0.23.tar.gz (97 kB)
     |████████████████████████████████| 97 kB 3.1 MB/s 
Collecting py-sds011
  Downloading py_sds011-0.9-py3-none-any.whl (3.8 kB)
Collecting sdm_modbus
  Downloading sdm_modbus-0.4.4-py3-none-any.whl (8.1 kB)
Requirement already satisfied: pyserial in /usr/lib/python3/dist-packages (from py-sds011) (3.4)
Collecting pymodbus>=2.3.0
  Downloading pymodbus-2.5.0-py2.py3-none-any.whl (154 kB)
     |████████████████████████████████| 154 kB 5.0 MB/s 
Collecting six>=1.15.0
  Downloading six-1.15.0-py2.py3-none-any.whl (10 kB)
Building wheels for collected packages: paho-mqtt, pybluez
  Building wheel for paho-mqtt (setup.py) ... done
  Created wheel for paho-mqtt: filename=paho_mqtt-1.5.1-py3-none-any.whl size=61543 sha256=e94679eaa017ee6089ce6e2ca1ea20ad500760bd28e3668e454c3ee158f20c5f
  Stored in directory: /root/.cache/pip/wheels/78/ec/85/b657c03288b6e363c90369c96ee04eea9abb9be78cadb7855b
  Building wheel for pybluez (setup.py) ... done
  Created wheel for pybluez: filename=PyBluez-0.23-cp38-cp38-linux_x86_64.whl size=139138 sha256=8d71bced211d990a44596a11174bed24f72283dbd35e16a3c6e1292118bdd13a
  Stored in directory: /root/.cache/pip/wheels/05/b9/3d/36ace9a7ee80a5c01b1d4910723e0c593d1f3031664e2c9907
Successfully built paho-mqtt pybluez
ERROR: launchpadlib 1.10.13 requires testresources, which is not installed.
Installing collected packages: paho-mqtt, pybluez, py-sds011, six, pymodbus, sdm-modbus
  Attempting uninstall: six
    Found existing installation: six 1.14.0
    Not uninstalling six at /usr/lib/python3/dist-packages, outside environment /usr
    Can't uninstall 'six'. No files were found to uninstall.
  Attempting uninstall: pymodbus
    Found existing installation: pymodbus 2.1.0
    Not uninstalling pymodbus at /usr/lib/python3/dist-packages, outside environment /usr
    Can't uninstall 'pymodbus'. No files were found to uninstall.
Successfully installed paho-mqtt-1.5.1 py-sds011-0.9 pybluez-0.23 pymodbus-2.5.0 sdm-modbus-0.4.4 six-1.15.0
- installing emonhub.service
Created symlink /etc/systemd/system/multi-user.target.wants/emonhub.service → /opt/openenergymonitor/emonhub/service/emonhub.service.
Created symlink /etc/systemd/system/emonhub.service → /opt/openenergymonitor/emonhub/service/emonhub.service.
Failed to restart emonhub.service: Unit var-log.mount not found.
- Service ActiveState=inactive

The tail -f /var/log/syslog during the sudo ./install.sh:

Mar 31 16:21:57 slaptop dbus-daemon[579]: [system] Activating via systemd: service name='org.freedesktop.PackageKit' unit='packagekit.service' requested by ':1.70' (uid=0 pid=2818 comm="/usr/bin/gdbus call --system --dest org.freedeskto" label="unconfined")
Mar 31 16:21:57 slaptop systemd[1]: Starting PackageKit Daemon...
Mar 31 16:21:57 slaptop PackageKit: daemon start
Mar 31 16:21:57 slaptop dbus-daemon[579]: [system] Successfully activated service 'org.freedesktop.PackageKit'
Mar 31 16:21:57 slaptop systemd[1]: Started PackageKit Daemon.
Mar 31 16:22:32 slaptop dbus-daemon[3674]: [session uid=0 pid=3672] AppArmor D-Bus mediation is enabled
Mar 31 16:22:32 slaptop dbus-daemon[3674]: [session uid=0 pid=3672] Activating service name='org.kde.kwalletd5' requested by ':1.0' (uid=0 pid=3668 comm="/usr/bin/python3 /usr/bin/pip3 install paho-mqtt r" label="unconfined")
Mar 31 16:22:32 slaptop org.kde.kwalletd5[3676]: QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
Mar 31 16:22:32 slaptop dbus-daemon[3674]: [session uid=0 pid=3672] Successfully activated service 'org.kde.kwalletd5'
Mar 31 16:22:32 slaptop dbus-daemon[3674]: [session uid=0 pid=3672] Activating service name='org.freedesktop.secrets' requested by ':1.0' (uid=0 pid=3668 comm="/usr/bin/python3 /usr/bin/pip3 install paho-mqtt r" label="unconfined")
Mar 31 16:22:32 slaptop org.freedesktop.secrets[3680]: GNOME_KEYRING_CONTROL=/root/.cache/keyring-N74200
Mar 31 16:22:32 slaptop dbus-daemon[3674]: [session uid=0 pid=3672] Successfully activated service 'org.freedesktop.secrets'
Mar 31 16:22:41 slaptop systemd[1]: Reloading.
Mar 31 16:22:41 slaptop systemd[1]: /lib/systemd/system/dbus.socket:5: ListenStream= references a path below legacy directory /var/run/, updating /var/run/dbus/system_bus_socket → /run/dbus/system_bus_socket; please update the unit file accordingly.
Mar 31 16:23:52 slaptop kernel: [ 1194.098076] perf: interrupt took too long (4974 > 4961), lowering kernel.perf_event_max_sample_rate to 40000

Conclusion
I guess the line
Failed to restart emonhub.service: Unit var-log.mount not found.
needs some attention as this var-log.mount is mentioned in
cat /etc/systemd/system/emonhub.service as:

Requires=var-log.mount
After=var-log.mount network.target

borpin · 31 March 2021 16:38

I’m not sure where the package six comes from! It could be a problem with pybluez.

You can just comment out the var-log.mount for the moment but be aware that will prevent Git updates.

A better way is to create a drop in.

sudo systemctl edit emonhub.service

put the following in

[Unit]
Requires=
After=network.target

save.

you then need to reload the daemon

sudo systemctl daemon-reload

Then restart.

(edited as I had missed out the [Unit])

m2ts · 31 March 2021 17:30

@borpin I guess your solution worked. What I did (first edit command with the lines you suggested):

sudo systemctl edit emonhub.service
sudo systemctl daemon-reload
sudo systemctl status emonhub
● emonhub.service - emonHub data multiplexer
     Loaded: loaded (/opt/openenergymonitor/emonhub/service/emonhub.service; enabled; vendor preset: enabled)
    Drop-In: /etc/systemd/system/emonhub.service.d
             └─override.conf
     Active: active (running) since Wed 2021-03-31 19:05:48 CEST; 10min ago
   Main PID: 713 (python3)
      Tasks: 3 (limit: 4608)
     Memory: 32.2M
     CGroup: /system.slice/emonhub.service
             └─713 python3 /usr/local/bin/emonhub/emonhub.py --config-file=/etc/emonhub/emonhub.conf --logfile=/var/log/emonhub/emonhub.log

mrt 31 19:05:48 slaptop systemd[1]: Starting emonHub data multiplexer...
mrt 31 19:05:48 slaptop systemd[1]: Started emonHub data multiplexer.
mrt 31 19:13:06 slaptop systemd[1]: /etc/systemd/system/emonhub.service.d/override.conf:1: Assignment outside of section. Ignoring.
mrt 31 19:13:06 slaptop systemd[1]: /etc/systemd/system/emonhub.service.d/override.conf:2: Assignment outside of section. Ignoring.
mrt 31 19:14:14 slaptop systemd[1]: /etc/systemd/system/emonhub.service.d/override.conf:1: Assignment outside of section. Ignoring.
mrt 31 19:14:14 slaptop systemd[1]: /etc/systemd/system/emonhub.service.d/override.conf:2: Assignment outside of section. Ignoring.
mrt 31 19:16:21 slaptop systemd[1]: /etc/systemd/system/emonhub.service.d/override.conf:1: Assignment outside of section. Ignoring.
mrt 31 19:16:21 slaptop systemd[1]: /etc/systemd/system/emonhub.service.d/override.conf:2: Assignment outside of section. Ignoring.

Then I checked http://localhost/emoncms/admin/view and now not only the redis-server and mosquitto is running, but also emonhub. Thank you!!! So I guess I can try the SDM120 emonhub interfacer. The guide talks about “Login to the local copy of Emoncms running on the emonPi/emonBase and navigate to Setup > EmonHub. Click on ‘Edit Config’” The problem is that I only have Inputs, Feeds, Visualisation, Schedule and Admin in the setup menu. But I guess I can edit /etc/emonhub/emonhub.conf and reload the emonhub service.

borpin · 1 April 2021 06:59

Sorry the correct instruction should have been (corrected above)

sudo systemctl edit emonhub.service

put the following in

[Unit]
Requires=
After=network.target

m2ts · 16 April 2021 14:56

FYI. As borpin already pointed out I used “archived steps” and they shouldn’t be used. And I need to let go of the ‘sudo git’ habit. So I did: I installed everything from scratch again. My notes are in a different topic.