Mosquitto and emonhub not starting anymore after update

Emoncms
1

I did a full update and nothing worked anymore
Then I did a apt-get update and upgrade and most of the services started to work
except mosquitto (1.5.7-1+deb10u1) and emonhub
I think emonhub won’t work because of mosquitto ( I use it in emonhub)

The problem is I don’t see anything wrong in the logs

-- The unit mosquitto.service has entered the 'failed' state with result 'exit-code'.
Aug 13 21:20:02 emonpi systemd[1]: Failed to start Mosquitto MQTT v3.1/v3.1.1 Broker.
-- Subject: A start job for unit mosquitto.service has failed
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- A start job for unit mosquitto.service has finished with a failure.

what should I check?
should I downgrade mosquitto?

2

root@emonpi:/var/log/apt# mosquitto
mosquitto: symbol lookup error: /usr/lib/arm-linux-gnueabihf/libuv.so.1: undefined symbol: uv__udp_try_sentuv_uDp_re#v]stqst

3

[edit]
You will need to tell us your HW and OS as a starting point. If a Pi,

Is this an emonPi or an emonBase? What base image?

4
Server Information

Server Information

Services

  • emonhub :- Failed Failed
  • emoncms_mqtt :- Active Running
  • feedwriter :- Active Running - sleep 300s 0 feed points pending write
  • service-runner :- Active Running
  • emonPiLCD :- Failed Failed
  • redis-server :- Active Running
  • mosquitto :- Failed Failed
  • demandshaper :- Active Running

Emoncms

Server

  • OS :- Linux 5.4.51-v7l+
  • Host :- emonpi | emonpi | (192.168.12.43)
  • Date :- 2020-08-14 08:45:08 BST
  • Uptime :- 08:45:08 up 11:31, 0 users, load average: 0.00, 0.05, 0.02

Memory

  • RAM :- Used: 17.18%
    • Total :- 904.58 MB
    • Used :- 155.41 MB
    • Free :- 749.17 MB
  • Swap :- Used: 0.25%
    • Total :- 100 MB
    • Used :- 256 KB
    • Free :- 99.75 MB
      Write Load Period

Disk

  • / :- Used: 58.90%
    • Total :- 3.92 GB
    • Used :- 2.31 GB
    • Free :- 1.42 GB
    • Write Load :- 7.64 KB/s (11 hours 29 mins)
  • /var/opt/emoncms :- Used: 10.66%
    • Total :- 9.98 GB
    • Used :- 1.06 GB
    • Free :- 8.41 GB
    • Write Load :- 6.43 KB/s (11 hours 29 mins)
  • /boot :- Used: 21.10%
    • Total :- 252.05 MB
    • Used :- 53.19 MB
    • Free :- 198.86 MB
    • Write Load :- 0 B/s (11 hours 29 mins)
  • /var/log :- Used: 45.70%
    • Total :- 50 MB
    • Used :- 22.85 MB
    • Free :- 27.15 MB
    • Write Load :- n/a

HTTP

  • Server :- Apache/2.4.38 (Raspbian) HTTP/1.1 CGI/1.1 8543

MySQL

  • Version :- 5.5.5-10.3.23-MariaDB-0+deb10u1
  • Host :- localhost:6379 (127.0.0.1)
  • Date :- 2020-08-14 08:45:08 (UTC 01:00‌​)
  • Stats :- Uptime: 41479 Threads: 11 Questions: 13204 Slow queries: 0 Opens: 60 Flush tables: 1 Open tables: 54 Queries per second avg: 0.318

Redis

  • Version :-
    • Redis Server :- 5.0.3
    • PHP Redis :- 5.0.2
  • Host :- localhost:6379
  • Size :- 172 keys (807.65K)
  • Uptime :- 0 days

MQTT Server

  • Version :- Mosquitto
  • Host :- localhost:1883 (127.0.0.1)

PHP

  • Version :- 7.3.19-1~deb10u1 (Zend Version 3.3.19)
  • Modules :- apache2handlercalendar Core ctype curl date dom v20031129exif fileinfo filter ftp gd gettext hash iconv json v1.7.0libxml mbstring mosquitto v0.4.0mysqli mysqlnd vmysqlnd 5.0.12-dev - 20150407 - $Id: 7cc7cc96e675f6d72e5cf0f267f48e167c2abb23 $openssl pcre PDO pdo_mysql Phar posix readline redis v5.0.2Reflection session shmop SimpleXML sockets sodium SPL standard sysvmsg sysvsem sysvshm tokenizer wddx xml xmlreader xmlwriter xsl Zend OPcache zlib

Pi

  • Model :- Raspberry Pi 4 Model B Rev 1.1 - 1GB (Sony UK)

  • Serial num. :- 10000000CBD94558

  • CPU Temperature :- 55.50°C

  • GPU Temperature :- 55.0°C

  • emonpiRelease :- emonSD-17Oct19

  • File-system :- read-write

Client Information

Client Information

HTTP

  • Browser :- Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36
  • Language :- en-US,en;q=0.9,ro;q=0.8

Window

  • Size :- 1841 x 960

Screen

  • Resolution :- 1920 x 1080
5

Also please click on copy as markdown
image
and paste it here; no further formatting required.

6

ok, I edited the post

1 Like
7

I still need this answer

Is this an emonPi or an emonBase?

8

what does

systemctl status {mosquitto.service,emonhub.service}

say?

When posting code or bash output, please put in 3 ‘backticks’ (found at the top left of the keyboard normally) on a line on their own, then the code, then 3 more backticks on a line following the code.

    ```
    code
    ```

If it is something like php you can add a language identifier that after the first 3 backticks so ```php

9

Did you do a dist-upgrade?

10

No, I did not

1 Like
11

it’s running on raspberry pi, so it’s emonPI
based on emonSD-17Oct19 (Stable)

12

Please provide this info.

13 
● mosquitto.service - Mosquitto MQTT v3.1/v3.1.1 Broker
   Loaded: loaded (/lib/systemd/system/mosquitto.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Fri 2020-08-14 11:29:25 BST; 19s ago
     Docs: man:mosquitto.conf(5)
           man:mosquitto(8)
  Process: 1289 ExecStart=/usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf (code=exited, status=127)
 Main PID: 1289 (code=exited, status=127)

Aug 14 11:29:25 emonpi systemd[1]: mosquitto.service: Service RestartSec=100ms expired, scheduling restart.
Aug 14 11:29:25 emonpi systemd[1]: mosquitto.service: Scheduled restart job, restart counter is at 5.
Aug 14 11:29:25 emonpi systemd[1]: Stopped Mosquitto MQTT v3.1/v3.1.1 Broker.
Aug 14 11:29:25 emonpi systemd[1]: mosquitto.service: Start request repeated too quickly.
Aug 14 11:29:25 emonpi systemd[1]: mosquitto.service: Failed with result 'exit-code'.
Aug 14 11:29:25 emonpi systemd[1]: Failed to start Mosquitto MQTT v3.1/v3.1.1 Broker.

● emonhub.service - emonHub service description
   Loaded: loaded (/opt/openenergymonitor/emonhub/service/emonhub.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Fri 2020-08-14 11:22:38 BST; 7min ago
 Main PID: 492 (code=exited, status=1/FAILURE)

Aug 14 11:22:38 emonpi systemd[1]: emonhub.service: Service RestartSec=100ms expired, scheduling restart.
Aug 14 11:22:38 emonpi systemd[1]: emonhub.service: Scheduled restart job, restart counter is at 5.
Aug 14 11:22:38 emonpi systemd[1]: Stopped emonHub service description.
Aug 14 11:22:38 emonpi systemd[1]: emonhub.service: Start request repeated too quickly.
Aug 14 11:22:38 emonpi systemd[1]: emonhub.service: Failed with result 'exit-code'.
Aug 14 11:22:38 emonpi systemd[1]: Failed to start emonHub service description.
Aug 14 11:29:23 emonpi systemd[1]: /opt/openenergymonitor/emonhub/service/emonhub.service:10: PIDFile= references path below legacy directory /var/run/, updating /var/run/emonhub.pid → /run/emonhub.pid; please update the unit file
Aug 14 11:29:23 emonpi systemd[1]: /opt/openenergymonitor/emonhub/service/emonhub.service:10: PIDFile= references path below legacy directory /var/run/, updating /var/run/emonhub.pid → /run/emonhub.pid; please update the unit file
lines 1-28/28 (END)
14 
root@emonpi:~# /usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf
/usr/sbin/mosquitto: symbol lookup error: /usr/lib/arm-linux-gnueabihf/libuv.so.1: undefined symbol: uv__udp_try_sentuv_uDp_re#v]stqst
15

Can you please post the result of

cat /etc/mosquitto/mosquitto.conf

Please stop the mosquitto service

sudo systemctl stop mosquitto

Try and start it manually please

/usr/sbin/mosquitto -v -c /etc/mosquitto/mosquitto.conf

and post any error messages

16 
root@emonpi:~# cat /etc/mosquitto/mosquitto.conf
# Place your local configuration in /etc/mosquitto/conf.d/
#
# A full description of the configuration file is at
# /usr/share/doc/mosquitto/examples/mosquitto.conf.example

pid_file /var/run/mosquitto.pid

persistence false
persistence_location /var/lib/mosquitto/

log_dest file /var/log/mosquitto/mosquitto.log

include_dir /etc/mosquitto/conf.d
allow_anonymous false
password_file /etc/mosquitto/passwd
log_type error

root@emonpi:~# sudo systemctl stop mosquitto
root@emonpi:~# /usr/sbin/mosquitto -v -c /etc/mosquitto/mosquitto.conf
/usr/sbin/mosquitto: symbol lookup error: /usr/lib/arm-linux-gnueabihf/libuv.so.1: undefined symbol: uv__udp_try_sentuv_uDp_re#v]stqst
root@emonpi:~#
17

the problem seems to be caused by the symbol lookup error

the lib is part of gcc, I think
I already tried apt-get reinstall gcc-arm-linux-gnueabihf
it did not seem to fix the problem though

18

I have just checked my emonPi, and I am on OS: Linux 4.19.118-v7+ which is quite different.

Try reinstalling the mosquitto package.

19

apt-get reinstall mosquitto did not work
however I removed mosquitto, then apt autoremove and I reinstalled mosquitto and not it works.

however, emonhub still does NOT work

systemctl status emonhub.service
● emonhub.service - emonHub service description
   Loaded: loaded (/opt/openenergymonitor/emonhub/service/emonhub.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Fri 2020-08-14 12:31:42 BST; 2min 9s ago
  Process: 488 ExecStartPre=/bin/mkdir -p /var/log/emonhub/ (code=exited, status=0/SUCCESS)
  Process: 489 ExecStartPre=/bin/chgrp -R emonhub /var/log/emonhub/ (code=exited, status=0/SUCCESS)
  Process: 490 ExecStartPre=/bin/chmod 775 /var/log/emonhub/ (code=exited, status=0/SUCCESS)
  Process: 491 ExecStart=/usr/local/bin/emonhub/emonhub.py --config-file=/etc/emonhub/emonhub.conf --logfile=/var/log/emonhub/emonhub.log (code=exited, status=1/FAILURE)
 Main PID: 491 (code=exited, status=1/FAILURE)

Aug 14 12:31:42 emonpi systemd[1]: emonhub.service: Service RestartSec=100ms expired, scheduling restart.
Aug 14 12:31:42 emonpi systemd[1]: emonhub.service: Scheduled restart job, restart counter is at 5.
Aug 14 12:31:42 emonpi systemd[1]: Stopped emonHub service description.
Aug 14 12:31:42 emonpi systemd[1]: emonhub.service: Start request repeated too quickly.
Aug 14 12:31:42 emonpi systemd[1]: emonhub.service: Failed with result 'exit-code'.
Aug 14 12:31:42 emonpi systemd[1]: Failed to start emonHub service description.

/usr/local/bin/emonhub/emonhub.py --config-file=/etc/emonhub/emonhub.conf --logfile=/var/log/emonhub/emonhub.log
Traceback (most recent call last):
  File "/usr/local/bin/emonhub/emonhub.py", line 25, in <module>
    import emonhub_interfacer as ehi
  File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 14, in <module>
    import requests
  File "/usr/lib/python3/dist-packages/requests/__init__.py", line 115, in <module>
    from .api import request, get, head, post, patch, put, delete, options
  File "/usr/lib/python3/dist-packages/requests/api.py", line 13, in <module>
    from . import sessions
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 27, in <module>
    from .adapters import HTTPAdapter
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 44, in <module>
    from urllib3.contrib.socks import SOCKSProxyManager
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 724, in exec_module
  File "<frozen importlib._bootstrap_external>", line 857, in get_code
  File "<frozen importlib._bootstrap_external>", line 525, in _compile_bytecode
ValueError: bad marshal data (unknown type code)
20

how do I reinstall emonhub?