EmonESP Custom MQTT Port

ian · 1 October 2020 15:00

I have uploaded latest version to a sonoff.
I cannot get mqtt to connect.

My port is 2082. What concerns me is that if I connect to the sonoff via its URL even though I have saved my mqtt parameters the port number always reverts to 1883 if I refresh the page.

Have you tested none standard port numbers?

glyn.hudson · 1 October 2020 15:34

Yes, I have tested on a Sonoff smartplug.

Apologies, you’re totally correct. I thought custom MQTT ports had been implemented. However, it looks like even though a field for custom MQTT ports has been added to the UI the port entered is not actually saved:

github.com

openenergymonitor/EmonESP/blob/master/src/web_server.cpp#L322

    
      
                                request->arg(F("server")),

                                request->arg(F("path")),

                                request->arg(F("node")),

                                request->arg(F("apikey")),

                                request->arg(F("fingerprint")));

          

            char tmpStr[200];

            snprintf_P(tmpStr, sizeof(tmpStr), PSTR("Saved: %s %s %s %s %s"),

                       emoncms_server.c_str(),

                       emoncms_path.c_str(),

                       emoncms_node.c_str(),

                       emoncms_apikey.c_str(),

                       emoncms_fingerprint.c_str());

            DBUGLN(tmpStr);

          

            response->setCode(200);

            response->print(tmpStr);

            request->send(response);

          }

          

          // -------------------------------------------------------------------

I’ve opened an issue to get this fixed:

github.com/openenergymonitor/EmonESP

Non standard MQTT port not saving

opened 03:28PM - 01 Oct 20 UTC

closed 12:10PM - 02 Oct 20 UTC

glynhudson

bug

The UI supports entering a non-standard MQTT port, however the value entered thi…s custom port field is not actually saved: https://github.com/openenergymonitor/EmonESP/blob/master/src/web_server.cpp#L322 ![](https://community.openenergymonitor.org/uploads/default/original/2X/1/196d2aeea453acc67b52b53e922579e1af6cc70e.png) A user has reported unable to get MQTT to connect on a non-standard port: https://community.openenergymonitor.org/t/emonesp-v3-1-0-significant-update-no-more-spiffs/15544/12?u=glyn.hudson

Also, HTTP update does not yet work on the S20 Smartplug since the ESP in the plug only has 1Mb of flash (compared to 4Mb on the Huzzah), this means there is nowhere for the update to be stored before it’s applied. There is a chance we may be able to fix this by further shrinking the FW.

glyn.hudson · 2 October 2020 12:19

Hi @ian,

This issue should now be fixed:

See attached for S20 smartplug pre-compiled FW which includes the fix. Please could you confirm if it works for you? You should be able to apply the update via the HTTP interface.

smartplug-dev.bin (825.7 KB)

borpin · 3 October 2020 05:08

@ian, why not just use Tasmota?

ian · 3 October 2020 08:04

Can you use Demand Shaper to control an ESP with Tasmota firmware?

borpin · 3 October 2020 08:46

You would just need to set the appropriate MQTT topics and possibly a Tasmota rule.

I don’t use DemandShaper at all and I’m not sure if you can specify what topic to post the on/off command to. If you can then yes to your question. If no, then DemandShaper needs to be more flexible .

TDMGR GitHub - jziolkowski/tdm: GUI application to discover and monitor devices flashed with https://github.com/arendst/Sonoff-Tasmota is a really great tool for seeing what is going on with all your Tasmota devices.

ian · 5 October 2020 08:41

Hi @glyn.hudson

Still an issue.

Firstly I could not update OTA. failed message

I then uploaded firmware and received this message.

I the erased ESP and tried again.

This time MQTT config saved OK but no connection.

On investigation there is a problem with the server address.

I have my server address 192.168.1.28 saved:-

espurl

Which is fine. But when I look in the log the address is different 192.168.1.22 and not surprisingly connection fails.

ESPLog

So it looks as if the server address is also not being saved correctly.

glyn.hudson · 6 October 2020 00:17

The OTA update is now fixed, you should not be able to OTA update from the version your running onwards.

I’ve setup an MQTT server on port 2082 to test and it connected fine for me:

Thanks for reporting the debug issue, this is a bug the debug window shows the incorrect IP. You can check the correct IP is being used by looking at http://192.168.1.90/config

To double-check your running the FW with these fixed try updating to the firmware attached to this post via the HTTP update then check the custom port again.

firmware.bin (825.7 KB)

ian · 6 October 2020 09:28

Some progress.

mqtt is now connecting but only for a short period and then drops out. I see it briefly in Demand Shaper asking to allow connect.

http://192.168.1.90/config returns incorrect mqtt url

{"espflash":1048576,"version":"3.1.0","node_description":"WiFi Relay","node_type":"wifirelay","ssid":"studiomesh","pass":"_DUMMY_PASSWORD","www_username":"","www_password":"","hostname":"wifirelay6590","emoncms_server":"","emoncms_path":"","emoncms_node":"","emoncms_apikey":"","emoncms_fingerprint":"","mqtt_server":"192.168.1.22","mqtt_port":2082,"mqtt_topic":"emon","mqtt_user":"bpa","mqtt_pass":"_DUMMY_PASSWORD","mqtt_feed_prefix":"","timer_start1":0,"timer_stop1":0,"timer_start2":0,"timer_stop2":0,"time_offset":0,"voltage_output":0,"ctrl_mode":"Off","flags":2,"emoncms_enabled":false,"mqtt_enabled":true,"ctrl_update":false,"ctrl_state":false}

I don’t have any trouble connecting other ESP devices to MQTT so I have no idea what is causing the issue.

I have just tried new firmware on a spare huzzah I have with similar result.

glyn.hudson · 6 October 2020 15:54

Mmm that’s progress, at least the custom MQTT port is now working. It’s strange /config returns the incorrect URL for you. I can’t replicate that, /config always returns the same URL as shown in the input field in the web UI once it’s been saved.

I notice your wifi name is ‘studiomesh’, I’ve head reports from other users with mesh networks that the ESP has trouble connecting to specific ports over a mesh network. Have you got a non-mesh WiFi AP you could use to test the connection? Or maybe try moving the ESP to be on the same mesh node as the MQTT server.

I’m not sure what else to suggest, I’m unable to replicate the issue.

I’ve just done an official V3.1.1 release which includes the latest changes: Release V3.1.1 · openenergymonitor/EmonESP · GitHub

ian · 6 October 2020 18:47

Delete superseded by later information.

ian · 7 October 2020 09:15

Hi @glyn.hudson

I had an Eureka moment.

Whenever I lose the MQTT connection the MQTT server has reverted to 192.168.1.22.

I suddenly realised that is the emoncms url. So how the hell did the esp know that url?

I think the answer is demand shaper.

This is when I first setup the ESP

Config is:-

{“espflash”:1048576,“version”:“3.1.1”,“node_description”:“Sonoff Smartplug”,“node_type”:“smartplug”,“ssid”:“studiomesh”,“pass”:"_DUMMY_PASSWORD",“www_username”:"",“www_password”:"",“hostname”:“smartplug6590”,“emoncms_server”:"",“emoncms_path”:"",“emoncms_node”:"",“emoncms_apikey”:"",“emoncms_fingerprint”:"",“mqtt_server”:“192.168.1.28”,“mqtt_port”:2082,“mqtt_topic”:"",“mqtt_user”:“bpa”,“mqtt_pass”:"_DUMMY_PASSWORD",“mqtt_feed_prefix”:"",“timer_start1”:65535,“timer_stop1”:65535,“timer_start2”:65535,“timer_stop2”:65535,“time_offset”:65535,“voltage_output”:65535,“ctrl_mode”:“Off”,“flags”:2,“emoncms_enabled”:false,“mqtt_enabled”:true,“ctrl_update”:false,“ctrl_state”:false}

I go into emoncms and go to demandshaper.
I get this

I click Allow.

I think at this point the ESP loses the MQTT connection. I managed to capture it in the log:-

GET http://192.168.1.90/lastvalues
GET http://192.168.1.90/status
GET http://192.168.1.90/lastvalues
GET http://192.168.1.90/status
GET http://192.168.1.90/lastvalues
GET http://192.168.1.90/status
GET http://192.168.1.90/lastvalues
Received 12 bytes from 192.168.1.22, port 42090
UDP packet contents: emonpi.local
mqtt_server changed
MQTT Server Updated
Fetching MQTT Auth
192.168.1.22
Connecting to http://192.168.1.22:80/emoncms/device/auth/request.json
Authentication request registered for IP 192.168.1.90
Disconnecting MQTT
MQTT Connecting to…192.168.1.22
MQTT failed: -2
GET http://192.168.1.90/status
GET http://192.168.1.90/lastvalues
Free memory 23608 - diff 4048 8800
GET http://192.168.1.90/status
GET http://192.168.1.90/lastvalues

Clearly the UDP packet is changing something.

Now I don’t know if the problem is the demandshaper message contains something it shouldn’t or that the ESP is reacting to the message in a way it shouldn’t.

When I go back to the ESP I see this:-

And config now shows this:-

{“espflash”:1048576,“version”:“3.1.1”,“node_description”:“Sonoff Smartplug”,“node_type”:“smartplug”,“ssid”:“studiomesh”,“pass”:"_DUMMY_PASSWORD",“www_username”:"",“www_password”:"",“hostname”:“smartplug6590”,“emoncms_server”:"",“emoncms_path”:"",“emoncms_node”:"",“emoncms_apikey”:"",“emoncms_fingerprint”:"",“mqtt_server”:“192.168.1.22”,“mqtt_port”:2082,“mqtt_topic”:“emon/SPtest”,“mqtt_user”:“bpa”,“mqtt_pass”:"_DUMMY_PASSWORD",“mqtt_feed_prefix”:“studio2studio2”,“timer_start1”:65535,“timer_stop1”:65535,“timer_start2”:65535,“timer_stop2”:65535,“time_offset”:65535,“voltage_output”:65535,“ctrl_mode”:“Off”,“flags”:2,“emoncms_enabled”:false,“mqtt_enabled”:true,“ctrl_update”:false,“ctrl_state”:false}

The only way you can replicate this is to set up an instance of Emoncms on one pi with a mosquitto server on another pi in the same way I have.

ian · 14 October 2020 13:48

@TrystanLea

@glyn.hudson

Any chance of looking at this issue?

I would really like to add more sonoff plugs to demandshaper and this is a complete showstopper at the moment.

I can confirm that my test plug has remained MQTT connected for many days now but if I try to add to DS it immediately fails as documented above.

TrystanLea · 14 October 2020 16:12

Hello @ian, I will get the latest version on a smart plug here and see if I can replicate.
I’m still running v3.0.0 at the moment.

@jeremypoulter can you see what is going wrong here?

jeremypoulter · 14 October 2020 16:31

That is the EmonCMS auto config thing is it not? @glyn.hudson/@TrystanLea not totally sure how it is supposed to work so not sure if I broke anything, or should it just be removed now?

ian · 19 October 2020 16:09

@glyn.hudson
@TrystanLea

Is there any progress on this? Is there anything a user can do or is the feature Jeremy referred to embedded in the software? Really keen to get a number of plugs under DS control to make best use of Octopus Agile tariff.

TrystanLea · 19 October 2020 16:16

No I still need to look at this.

You should still be able to configure the smartplug manually. You need to put in all the MQTT details yourself so that it connects to the emonBase/Pi. Then once it pops up in the Emoncms input list, click on the cog icon to bring up the ‘Configure Device’ dialog and select the Control > MQTT > Smartplug device template. Click save to complete, it should then be visible in the demandshaper.

ian · 20 October 2020 08:59

Hi @TrystanLea

That worked so I can add new plugs. Unfortunately this has highlighted the issue I describe in this thread

I will add comments there.

borpin · 20 October 2020 09:18

I think this could probably be made to easily work with Tasmota rather than trying to write your own code for the ESP.

Much better use of your time IMHO.

Technically, what MQTT input does DS expect. What output does it generate. Tasmota has a powerful rules system that can do almost anything and easily templated. Would then work for any type of ESP8266 plug. Is there any documentation for DS?

There is also an argument here to integrate it with Home Assistant by publishing an MQTT message HA can process.