The idea is still to simplify the interface between the EmonTXx and other systems via Wi-Fi rather than RFM69. This is done by using the ESPHome Core and adding on a component specifically for the EmonTX. This approach has a number of advantages:
The C++ code base required to be maintained by OEM is reduced resulting in bug fixes and security updates to the base libraries being easily passed onto the ESP device.
Updates & configuration changes can be done OTA
The interface to HomeAssistant would be simplified and the device work independently from EmonCMS.
ESPHome has changed significantly since I first suggested this, with the core developer now part of the HA family and the integration to HA tighter and the support on a more professional footing.
@awjlogan - if you are looking at this, I’d suggest two components
A component to read the data from the TX Serial
A component to send the data to emoncms. Using a Lambda to send the data by MQTT is horribly clunky. Could provide an option for MQTT or HTTP API then it could go to emoncms.org.
@borpin - apologies, I’ve only just had a chance to have a look properly at this. Looks great - I’ll start having a play with it!
On the todo items, the emonPi3/Tx++ can easily do the OneWire interface and there’s an expander on board for I2C and I’ve left the temperature API agnostic to the actual sensor being used. We can skip the SPI interface and directly interface with the UART (using the existing header) - I put JSON support in to match the Tx4/Pi2 firmware, and this is clearly where it came from!
As it interfaces with the UART on the Pi3, if ESPHome supports commands being sent to a destination, it could configure it directly.
Currently, I have a USB-C header on the ESP board for programming - this seems a bit clunky (as there’s already a USB-C connector on the Pi3), so I’ll also look at programming the board through the Pi3 as well (I’m not too familiar with Espressif’s stuff yet).
In the meantime, I can make you an ESP board up (probably, have to check my part stash) and after I’m happy it works, could send you one to use as well?
Quick skim of the ESPHome documentation suggests that the UART Component would be a reasonable place to start. It does support transmit from the ESPHome as well, so could be used to configure the Tx/Pi in situ.
Looking at Brian’s Tx4 module, one question I have would be how to generalise it so the JSON decoding is not static. That is, the Pi3 can support 3 digital (OneWire or pulse) or 1 extra analog and 2 digital (OneWire or pulse) inputs. The number of temperature sensors is not defined but is detected at start up. There is a defined maximum for static allocation, but unlikely to be used in practice so you’d end up with a lot of empty entries (“T5”: 30000, “T6”: 30000 …). See what I can come up with - I also need to set up HA so I can test things
Is there still some “demand” for an official ESPHome component to fully support EmontxV4 and other similar devices ?
If yes, I can take what already exists (which repo should I take, the original or the fork ?) and make it to the end with documentation, … and ask the ESPHome team to integrate it officially.
First, I would concentrate on reading the measurements… and if possible, I would implement the possibility to fully configure EmonTx through HA.
Hi @FredM67 - yes, I think it would be useful. I’ve got a design made up with an ESP32-C3 that just fits on to the normal Raspberry Pi header with power and UART. I haven’t had time to put much effort into the ESPHome setup for it, so it’s rather languished (and honestly I don’t really enjoy working with frameworks!).
There are two options for the Tx5 and the incoming Tx6 - key:value data, or JSON. I was hoping to use ESPHome to digest the JSON output, seemed most straightforward. Configuration of the Tx is all through the UART.
In terms of documentation, probably the best thing to look at is the source code. For the Tx4/5, it’s here for 6 CT inputs. Line 28 describes the formats. Configuration options are here.
For the upcoming Tx6/Pi3, the output will be the same and you can see the configuration options on line 936 here.
Let me know if you would like one of the ESP32 boards and if you have any other queries/suggestions
Yes, that’d be great to have one of the ESP32 boards.
I don’t really have the hardware needed for that.
A couple of weeks ago, I’ve implemented an integration for Robin Emley’s mk2pvrouter. That works very fine. This integration not only reads the data (power, diverted energy, …) but it can control the router too (override, stop diversion, …). This integration is in the queue for review to be integrated in ESPHome.
I’ll keep you inform about the progress and for sure, I’ll need a couple of testers
Since the number of output values is very “dynamic” (single phase, three-phase, CTs between 0 and 12, …), I’d go rather with something similar to the mk2pvrouter component which is itself similar to the component Teleinfo.
Here, the user has to select which sensors he wants to read. Providing a gist to give him a complete list of all available sensors, he’ll have to simply comment/uncomment what he wants or what he doesn’t want.
The other way would be to predefine all the sensors, but well because of the huge amount of possibilities, the user would probably have a lot of sensors he doesn’t want or that are simple not available. The method is, I think, much more appropriate for a device which provides a finite well-defined list of sensors.
I don’t know enough about the HA integration side to comment too much on the data that’s available and how it’s parsed. For example, the emonPi3 will always output all the voltage and CT channels, even if they are inactive, as 0 is still a valid value. If the HA integration can filter out values that are known to be unused, that would be best - is that the first option you presented?
Can we assume that the JSON format is always going to be the same @TrystanLea?
If so, then it will only be the sensors themselves that will change. If the Component has a mechanism so you can tell it to look for CT1 and then creates HA sensors called Mains Supplyfor voltage, power etc (as it does now - badly) that should be flexible to cope with any number of inputs. You can then also look for Temperature as well.
One thing to consider is that we also want the component to be able to communicate with emoncms - the OEM software. This communication can be done locally via MQTT or by HTTP (locally and directly to emoncms.org).
Ideally the same data could be sent to multiple end points (HA, MQTT & HTTP).
Well my idea is to have something fully flexible. The user decides what he wants to see, what he wants to track.
This way, physical unconnected sensor, which show with a value of “0” can simply be skipped.
The user could also track just one part of a sensor. Let say, CT1 monitors my induction plate. If the user don’t care about the instant power, he would simply define in the yaml file the sensor for energy.
It would look something like this:
I would provide a gist with all available sensors, and the user simply have to:
copy-paste
comment/uncomment what he doesn’t need/what he needs.
If the user makes a mistake and defines a sensor that does not exist (like “E13”), the sensor would simply appear as “Not available” or something similar.
What do you think about that ?
BTW, I would also let the ESP32 takes control of the temperature sensors.
I think this is good - as 0 is a valid value, you can’t rely on that to indicate a data source’s status.
Less keen on this - the OEM systems typically use 1Wire temperature sensors, so you’d have to expand the adaptor board to either include external connectors or connect to the 1Wire pin on the Pi header - you’d then need a branch of the firmware to not activate the 1Wire pins. I would strongly suggest sticking with the data made available. The Pi3(/Tx6) will also have the facility to use I2C sensors within the unit.
@borpin - I don’t expect the JSON format to change, and if it does, does it matter? It’s just key/value pairs so it will always be available to decode. Sorry if I’ve missed something here?
Yes, definitely Is this possible with ESPHome? Although, if you have emonCMS running, can you not just use the HA integration for that? Again, sorry if I’m missing something there!
Indeed, a value of 0 could be simply an installed CT on a “dead” wire, or not installed at all.
It’s only a possibility. On EmonTxv4, I know the expansion board can take control of the Dallas sensors and even the RF module.
As long as the JSON stays “flat” (no sub-element, …), the component will work.
I did already try a couple of months ago to “forward” sensor values to IFTTT, through a REST-API. It worked.
So, it should be possible to do that for MQTT too.
BTW, this part won’t be to be implemented in the component itself. That’s pure ESPHome stuff.
To ease all of this, we could provide a gist with a sample config for MQTT and HTTP.
Got it - is there any advantage to nesting it? For example, you could represent it as:
{ "CT": [45.4, 56.5] }
In some ways, that is nicer than just padding the K/V format. As far as I know, there are no consumers for the JSON format just now - it was added for use the ESPHome. If you’d prefer it nested, I can do it for the Pi3…
Yes - the onboard microcontroller will handle those when there’s no Raspberry Pi present. All the data and configuration just comes through the UART port.
My idea is that the ESPHome Component isn’t simply to ease connection to HA, but can also be used as the means of connecting the ESP to emoncms as well. This would negate the need for OEM’s ESP firmware.
This would need it to be written so it can simply be onboarded when first booted. I’ve seen some examples of this type of deployment.
No. Let’s keep it simple!
As the data is already coming out on the UART in the JSON, there is no need.
I found this really tricky to do and took a lot of the ESPs cycle time. I think doing this within the component a bit more would be easier.
I never managed to get the HTTP to connect to the emoncms API. The raw JSON can simply be sent to emoncms, I have just never worked out how to make it available to send
Something must be wrong in the sources.
On my old Emontx v4 (3 phases), I receive {"MSG":14004,"V1":232.00,"V2":232.12,"V3":232.15,"P1":0,"P2":0,"P3":0,"P4":0,"P5":0,"P6":0,"E1":1,"E2":-3,"E3":-3,"E4":0,"E5":-3,"E6":-4}
Ah, sorry @FredM67 - the comment says Vrms as that’s the packed format for the RFM transmission. Your readings are the correct ones. The Pi3 also sends temperature and pulse data - this could potentially be put into the Tx4/5 firmware but it’s already tight for cycles…
