MQTT change now nothing works even after returning to defaults

dodegkr · 27 June 2020 10:55

Hi All,

I just adjusted my emonhub config to use a different broker by literally changing the ip and username password in emonhubs config, sadly then all of my inputs went N/A NULL
so I changed the config back to the default, saved and rebooted, but still N/A NULL
The reason for changing is to use the same broker for home assistant to publish emon data to HA

Any ideas?

[[MQTT]]

Type = EmonHubMqttInterfacer
[[[init_settings]]]
    mqtt_host = 127.0.0.1
    mqtt_port = 1883
    mqtt_user = emonpi
    mqtt_passwd = emonpimqtt2016

dodegkr · 27 June 2020 11:00

Server Information

Services

emonhub :- Active Running
emoncms_mqtt :- Active Running
feedwriter :- Active Running - sleep 300s 57 feed points pending write
service-runner :- Active Running
emonPiLCD :- Inactive Dead
redis-server :- Active Running
mosquitto :- Active Running
demandshaper :- Active Running

Emoncms

Version :- low-write 10.2.3
Modules :- Administration | App v2.1.6 | Backup v2.2.3 | EmonHub Config v2.0.5 | Dashboard v2.0.7 | DemandShaper v1.2.5 | Device v2.0.5 | EventProcesses | Feed | Graph v2.0.9 | Input | Postprocess v2.1.4 | CoreProcess | Schedule | Network Setup v1.0.0 | sync | Time | User | Visualisation | WiFi v2.0.2
Git :-
- URL :- GitHub - emoncms/emoncms: Web-app for processing, logging and visualising energy, temperature and other environmental data
- Branch :- * stable
- Describe :- 10.2.3

Server

OS :- Linux 4.19.75-v7+
Host :- emonpi | emonpi | (192.168.0.57)
Date :- 2020-06-27 11:59:41 BST
Uptime :- 11:59:41 up 19 min, 0 users, load average: 1.11, 1.06, 0.81

Memory

RAM :- Used: 16.88%
- Total :- 975.62 MB
- Used :- 164.7 MB
- Free :- 810.92 MB
Swap :- Used: 0.00%
- Total :- 100 MB
- Used :- 0 B
- Free :- 100 MB
  Write Load Period

Disk

/ :- Used: 59.84%
- Total :- 3.92 GB
- Used :- 2.35 GB
- Free :- 1.38 GB
- Write Load :- 225.88 B/s (18 mins)
/var/opt/emoncms :- Used: 5.48%
- Total :- 9.98 GB
- Used :- 559.66 MB
- Free :- 8.93 GB
- Write Load :- 48.94 B/s (18 mins)
/boot :- Used: 20.90%
- Total :- 252.05 MB
- Used :- 52.68 MB
- Free :- 199.37 MB
- Write Load :- 0 B/s (18 mins)
/var/log :- Used: 98.23%
- Total :- 50 MB
- Used :- 49.12 MB
- Free :- 904 KB
- Write Load :- n/a

HTTP

Server :- Apache/2.4.38 (Raspbian) HTTP/1.1 CGI/1.1 80

MySQL

Version :- 5.5.5-10.3.17-MariaDB-0+deb10u1
Host :- localhost:6379 (127.0.0.1)
Date :- 2020-06-27 11:59:41 (UTC 01:00‌)
Stats :- Uptime: 1165 Threads: 14 Questions: 2843 Slow queries: 0 Opens: 46 Flush tables: 1 Open tables: 40 Queries per second avg: 2.440

Redis

Version :-
- Redis Server :- 5.0.3
- PHP Redis :- 5.0.2
Host :- localhost:6379
Size :- 91 keys (793.43K)
Uptime :- 0 days

MQTT Server

Version :- Mosquitto 1.5.7
Host :- localhost:1883 (127.0.0.1)

PHP

Version :- 7.3.9-1~deb10u1 (Zend Version 3.3.9)
Modules :- apache2handlercalendar Core ctype curl date dom v20031129exif fileinfo filter ftp gd gettext hash iconv json v1.7.0libxml mbstring mosquitto v0.4.0mysqli mysqlnd vmysqlnd 5.0.12-dev - 20150407 - $Id: 7cc7cc96e675f6d72e5cf0f267f48e167c2abb23 $openssl pcre PDO pdo_mysql Phar posix readline redis v5.0.2Reflection session shmop SimpleXML sockets sodium SPL standard sysvmsg sysvsem sysvshm tokenizer wddx xml xmlreader xmlwriter xsl Zend OPcache zlib

Pi

Model :- Raspberry Pi 3 Model B+ Rev 1.3 - 1GB (Sony UK)
Serial num. :- 51417E17
CPU Temperature :- 60.69°C
GPU Temperature :- 61.2°C
emonpiRelease :- emonSD-17Oct19
File-system :- read-write

Client Information

HTTP

Browser :- Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.1 Safari/605.1.15
Language :- en-gb
Resolution :- 768 x 1024

LAST ENTRIES ON THE LOG FILE

borpin · 27 June 2020 17:22

You need to change the settings for emoncms in the /var/www/emoncms/settings.ini to read the values from the new broker.

emonhub is sending the values to a Broker. emoncms needs to read the values from the same broker.

dodegkr · 28 June 2020 07:54

Thanks for that does one add a line of say server = x.x.x.x or set it to false instead of true.
as it has no server target in settings.ini

dodegkr · 28 June 2020 10:26

seems somehow that emonhub is broken, the log is empty.

the config comes from

https://raw.githubusercontent.com/openenergymonitor/emonhub/emon-pi/conf/emonpi.default.emonhub.conf

I don’t understand why the log is empty, the service is showing as failed.

borpin · 28 June 2020 11:54

There is a file in that folder called 'default.settings.ini` - these are the default do not edit this.

Copy the mosquitto section into settings.ini to override the default settings as rquired - keep the section heading.

First check that the log folder is not full (you can see this from the admin page).

If that is OK then there was a problem with an update so try doing a full update.

dodegkr · 29 June 2020 14:30

log has plenty of space and updated but still no joy, I guess I will have to start again when I get home, thanks for trying to help

borpin · 30 June 2020 06:38

Have you changed the settings.ini?

dodegkr · 30 June 2020 15:57

copied the default to a new settings.ini
modified Mqtt and mysql.

but it now breaks the guy

Fatal error : Uncaught Error: Call to a member function get() on bool in /opt/emoncms/modules/demandshaper/demandshaper-module/demandshaper_model.php:33 Stack trace: #0 /opt/emoncms/modules/demandshaper/demandshaper-module/demandshaper_menu.php(20): DemandShaper->get_list(Object(Device), 1) #1 /var/www/emoncms/core.php(235): require(’/opt/emoncms/mo…’) #2 /var/www/emoncms/index.php(350): load_menu() #3 {main} thrown in /opt/emoncms/modules/demandshaper/demandshaper-module/demandshaper_model.php on line 33

borpin · 30 June 2020 19:35

Well of course it does because that is not what I said to do. Some of the settings in the settings.ini are different to the default settings. The computer is, though, doing exactly what you asked it to do.

If you had followed my instructions and copied across the MQTT bit, you’d probably be fine!

Copy the default contents from here EmonScripts/defaults/emoncms/emonpi.settings.ini at master · openenergymonitor/EmonScripts · GitHub

then add in

; The 'subscriber' topic format is rx/* - where * is the emoncms input node number.
; The 'publisher' topic format is user selectable from the 'Publish to MQTT' input process, for example power/solar
; Activate MQTT by changing to true
host = 'localhost'
port = 1883
basetopic = 'emon'
client_id = 'emoncms'

UNDER the [mqtt] section LEAVING the other items there.

Amend so the host, user, and password match whatever you have set on your broker and then

sudo systemctl restart emoncms_mqtt.service

dodegkr · 1 July 2020 12:00

Ok, thank you for your patience.
I have done that the gui now loads but still blank in emon hub log and the inputs are still borked.

borpin · 1 July 2020 12:16

emonhub has stopped. Try running the update again - might fix it.

borpin · 1 July 2020 12:17

Of course, if what you had wanted to do was send the data to another broker as well as emonpi, you could just have added in a second MQTT Interfacer or bridged the 2 Brokers.

dodegkr · 2 July 2020 10:27

something is wrong.
if you press full update now it flashed up
service-runner trigger sent, and that’s it, blank log window.
perhaps the SD is dying

dodegkr · 2 July 2020 10:28

all the apps are empty and the dashboards don’t load.

TrystanLea · 6 July 2020 18:34

Hello @dodegkr, I can see that you posted an error suggesting that emoncms cant read from feed id 61 together with the demandshaper error higher up suggests that there are quite a few things broken there? It might be easier to recover following the new SD card upgrade path documented here and then importing your existing data from this SD card Update & Upgrade - Guide | OpenEnergyMonitor I know thats a bit of a pain to do but can save trawling through SSH for a long time.

TrystanLea · 6 July 2020 18:35

One other thought is that it might be worth trying to flush redis, you can do this from the admin user interface or via SSH:

$ redis-cli flushall

That will in the case of the error you are seeing for feed 61, clear the associated caches and force a reload of all the feed information, that error could have arisen from a mismatch between the cache and the database…

dodegkr · 7 July 2020 12:26

Thank you, I tried the command and the log windows now has output but the output is

/bin/sh: 1: OPENENERGYMONITOR_DIR/emonpi/service-runner-update.sh: not found

the emoncms log is

Server Information

LAST ENTRIES ON THE LOG FILE

TrystanLea · 8 July 2020 08:32

Hello @dodegkr it looks like something has gone wrong with the settings. OPENENERGYMONITOR_DIR should be set in /var/www/emoncms/settings.ini. This is what this file should look like:

emoncms_dir = "/opt/emoncms"
openenergymonitor_dir = "/opt/openenergymonitor"

[sql]
server = "127.0.0.1"
database = "emoncms"
username = "emoncms"
password = "emonpiemoncmsmysql2016"
; Skip database setup test - set to false once database has been setup.
dbtest   = true

[redis]
enabled = true
prefix = ''

[mqtt]
enabled = true
host = 'localhost'
port = 1883
user = 'emonpi'
password = 'emonpimqtt2016'
basetopic = 'emon'
client_id = 'emoncms'

[feed]
engines_hidden = [0,6,10]
redisbuffer[enabled] = true
redisbuffer[sleep] = 300
phpfina[datadir] = '/var/opt/emoncms/phpfina/'
phptimeseries[datadir] = '/var/opt/emoncms/phptimeseries/'

[interface]
enable_admin_ui = true
feedviewpath = "graph/"
favicon = "favicon_emonpi.png"

[log]
; Log Level: 1=INFO, 2=WARN, 3=ERROR
level = 2

borpin · 8 July 2020 08:35

Except that he needs to add in his custom MQTT settings which is where we started…

He copied the entire default file.