And unfortunately, it might not even be on our user’s network - it could be someone else’s, a long way away in the opposite direction to the receiver, far enough for our receiver not to hear it but close enough for our transmitter to.
While that’s certainly a possibility, given the restrictions on transmitter power, channel usage
limits, and the vast majority of devices in the ISM bands are not connected to well-placed,
efficient antennas, the odds of that happening are fairly low.
Unfortunately, the Law of Natural Perversity doesn’t obey odds. If it’s possible for it to happen, it’s going to appear here as a problem, sooner or later.
I don’t know exactly where it was introduced in JeeLib, relating to the fact OEM is using older versions since PIO was introduced and therefore fixed library versions. But IIRC JeeLib was improved to abort reception at an earlier point if the preamble and group didn’t match, so using a later version of JeeLib and recompiling the base/reciever FW might improve the clashing e.g. problem as the receiver should be freed up sooner.
Regarding avoiding clashes where we know it could occur, eg starting multiple emontx’s at once (power outage etc). Would it be worth adding a nedeid x 10millis delay at start-up and also use the nodeid to set the resend delay to try and avoid 2 resends trying to resend together again, the higher nodeid would get delayed slightly more.
Hopefully, the introduction of the RFM69 “native” mode, which will be necessary when the emonPiCM becomes available wherein all reception processing is done inside the RFM69, will alleviate the problem.
But the fact remains that the basic operating principle of the emonTx & emonTH ensures that collisions will occur, and relies on probability in the hope that the consequences won’t be serious.
The sketch that Chris is using is based around emonLibCM, so it transmits at rigorously controlled intervals based on the processor clock. However, the r.f. library switches to ‘receive’ to check that the channel is clear before switching to transmit to send the data. If a signal above a set level is detected, it waits for it to cease, up to a preset maximum time, before transmitting anyway. In my test, this did seem to offer a significant improvement with two transmitters working to the same base.
1 Like
Hi Paul
Thanks for the cable suggestion. Thought I would document here (for when I inevitably break something and want to refer back…) what I did to “wire” my two emonTx’s to the emonPi:
-
Get two of the cables you linked above, 2x male-female USB extenders, and connect straight to the UART ports of the emonTx’s and USB ports of the emonPi
-
Check they show up:
pi@emonpi:~ $ lsusb
Bus 001 Device 005: ID 0403:6001 Future Technology Devices International, Ltd FT232 Serial (UART) IC
Bus 001 Device 004: ID 0403:6001 Future Technology Devices International, Ltd FT232 Serial (UART) IC
- Look for serial number details of the two devices
pi@emonpi:~ $ lsusb -v | grep Serial
Bus 001 Device 005: ID 0403:6001 Future Technology Devices International, Ltd FT232 Serial (UART) IC
idProduct 0x6001 FT232 Serial (UART) IC
iSerial 3 AD0K0RW1
Bus 001 Device 004: ID 0403:6001 Future Technology Devices International, Ltd FT232 Serial (UART) IC
idProduct 0x6001 FT232 Serial (UART) IC
iSerial 3 AQ00MFGH
- Create a new udev rule. I think writing udev rules is a dark art and I had to play around a bit
pi@emonpi:~ $ cat /etc/udev/rules.d/9-emon.rules
SUBSYSTEM=="tty", ATTRS{idProduct}=="6001", ATTRS{idVendor}=="0403", ATTRS{serial}=="AQ00MFGH", SYMLINK+="emonTX-AQ00MFGH"
SUBSYSTEM=="tty", ATTRS{idProduct}=="6001", ATTRS{idVendor}=="0403", ATTRS{serial}=="AD0K0RW1", SYMLINK+="emonTX-AD0K0RW1"
- Reload the rules and trigger them:
pi@emonpi:~ $ sudo udevadm control --reload
pi@emonpi:~ $ sudo udevadm trigger
- Check the symlinks have been created:
pi@emonpi:~ $ ls -al /dev/emon*
lrwxrwxrwx 1 root root 7 Nov 21 22:36 /dev/emonTX-AD0K0RW1 -> ttyUSB1
lrwxrwxrwx 1 root root 7 Nov 21 22:36 /dev/emonTX-AQ00MFGH -> ttyUSB0
- Run the following to check data can be read off the symlinked devices:
#!/usr/bin/env python
import time
import serial
ser = serial.Serial(port='/dev/emonTX-AQ00MFGH',baudrate = 115200,parity=serial.PARITY_NONE,stopbits=serial.STOPBITS_ONE,bytesize=serial.EIGHTBITS,timeout=1)
while 1:
x=ser.readline()
print x
- Messages are read every 10 secs so assume working. Now need to change emonhub configuration, so added to the interfacers section:
[[TxN1]]
Type = EmonHubTx3eInterfacer
[[[init_settings]]]
com_port= /dev/emonTX-AQ00MFGH
com_baud = 115200
[[[runtimesettings]]]
pubchannels = ToEmonCMS,
[[TxN2]]
Type = EmonHubTx3eInterfacer
[[[init_settings]]]
com_port= /dev/emonTX-AD0K0RW1
com_baud = 115200
[[[runtimesettings]]]
nodeoffset = 1
pubchannels = ToEmonCMS,
- Give the whole thing a reboot! Now I can see two new inputs in EmonCMS, catchily named ‘0’ and ‘1’, which seem to be receiving the same data as is still arriving over radio from the emonTx’s. Happy days!
Do people generally disable radio on “wired” emonTx’s, or just leave them alone?
Thanks
Chris
2 Likes
Good stuff.
You could just connect with miniterm.
As your issues were radio collisions, I’d disable it (do that through miniterm) - @pb66 - can you do that in emonhub config?
2 other suggestions -
You could use nodename for a user friendly name.
[[TxN2]]
Type = EmonHubTx3eInterfacer
[[[init_settings]]]
com_port= /dev/emonTX-AD0K0RW1
com_baud = 115200
[[[runtimesettings]]]
nodename = Serial_PiZ
If you are using MQTT, I’d also suggest using the JSON version of the MQTT data (keeps it all together) and timestamp it - MQTT Runtime settings.
[[[runtimesettings]]]
subchannels = ToEmonCMS,
timestamped = True
# Disable other formats
node_format_enable = 0
nodevar_format_enable = 0
# Enable JSON format
node_JSON_enable = 1
node_JSON_basetopic = emon/
If nobody is listening to the radio, it won’t concern @coltcc. However, if the radio will never be used again, I’d certainly turn it off, if only to not broadcast your power consumption to anybody who cares to listen.
There’s a serial command w0 that will turn the radio off. That needs to make its way into the emonTx by whatever route you now have that’s convenient.
At the moment, there’s nothing in emonHub apart from the USA switch that goes in the reverse direction.
Thanks for writing up and sharing how you did this, I’m sure it will be most useful for other users. I’m gonna bookmark the post so I can point users to it in future when this pops up again.
I only mentioned disabling RFM because of the option to wire just one emontx, which would not of solved the collisions unless you stopped the rf transmitted parallel data. Since you have wired both, what happens in the rf domain won’t effect your serial data.
Having said that, the rfm transmissions can make a bit of a divot in the voltage waveform if you are still powering the emontx’s via the AC adapter, so if you wanted to improve the accuracy by a pretty minor amount you could stop the rfm. I do however suspect you are now powering the emonTx’s via the 5v pin of the FTDI connector, ie from the emonPi USB ports? If so have you removed the internal jumper so the AC is only used for reference measurements and not to power the emonTx? I suggest you check the no load noise levels (eg what do the powers read at rest?) and check again after removing the jumpers, Robert has experienced noise entering the 5v rail when power via USB walwart, but in my experience, the supply from the RPi USB is pretty sound.
No, as Robert says there isn’t yet any way to adjust the config via emonhub, the JeeInterfacer only does Jee command format eg 4b, 5i, 0s etc and the Tx3e interfacer doesn’t offer any commands to my knowledge.
However, if the RFM2Pi (JeeInterfacer) is disabled in emonhub.conf then the RFM packets will not be processed and passed on to emoncms.
Do you mean the USA command is reversed? in relation to which format? I thought it was 1p which is the same way round as the Jee commands for nodeid, group, freq, quiet and version etc.
No. My meaning was: the “USA” command is the only information that travels in the direction of towards the emonTx. Everything else travels from the emonTx.
Are you sure, or am I still not understanding you correctly??
The Tx3e interfacer doesn’t issue any commands including the USA command and the Jee Interfacer send all the ones I mentioned above, and also uses the s command to transmit data, but the Jee interfacer isn’t connected to the serial emonTx’s.
Are you saying the Tx3e interfacer can send a USA command in Jee format (eg 1p) despite never being used for a emonpi? Or have I missunderstood?
I’m assuming that an emonTx serially connected to a RPi will behave in exactly the same way as an emonPi does, given that the front end there is almost identical to the emonTx and is serially connected to the Pi part.
The objective is to be able to send the appropriate configuration command to the emonTx’s via the FTDI serial port. I consider the easiest, most straightforward and obvious way to do that is to disconnect whatever the working connection is, replace it with a programmer connected to a computer with a keyboard and screen, and send the command. Then restore everything.
Trystan is working on a way to do that for the emonPiCM via his version of emonHub. I don’t know exactly the state of progress. In principle, that will also work for a serial-connected emonTx. There’s no intention to extend it to “over the air”.
You threw me with this statement
I couldn’t place how or where that could be true since the Tx3e interfacer used here in this thread to connect to the serially connected emontx’s is not very well named since the protocol used is actually whet was introduced with the emonESP, the emontx serial output was previously space separated variables, the emontx FW was changed to use key:value pairs to accommodate the emonESP and the Tx3e interfacer doesn’t output any commands to my knowledge.
Where as the Jee Interfacer will output many commands (ie not just the USA command as per your statement) leaving me confused as to what you were refering to.
Maybe in the future, it’s what I’ve wanted for some time. But currently no. The emonTx outputs key:value pairs to serial where as the emonpi outputs space seperated values, and the emonpi has many commands that can be issued by emonhub for the RFM control and transmission too, but the emonTx is just a transmitter with no reciever/base station capability so the Jee commands that the emonpi understands mean nothing to the emonTx.
This will of course all change now with the CM FW on the emonTx but as yet there is no provision in emonhub for the CM FW. I read about Trystan’s “all in one” interfacer that “replaces” all the other interfacers and that couldn’t be further away from the architecture of emonhub, so I have no idea how that can work. If there is one interfacer that accommodates all emon devices including Jee command, space separated values, key:value pairs and the new CM format commands too, emonhub will become something else with the same name.
Sorry if I’ve confused you (and anyone else). My problem is I’ve never found the time to chase down the code and find out how it works, and that goes for any documentation as well. So I don’t have more than superficial knowledge of emonHub and what happens inside the Pi. I’ve no idea what the formal rules are for the data that’s sent - whichever format it’s in: What’s acceptable as a name (character set), what’s acceptable as a number, are text values accepted, what delimiters are required, how many variables, what the limit on the message length is, etc?
LOL
If there were some, the problems with “special characters” in passwords, variable names etc wouldn’t arise. I think some call it being professional.
Having the emonTx unilaterally send data is prone to multiple failure modes.
-
If you have multiple emonTx units their transmissions can collide (similar to the “thundering herd” problem in “push”-based distributed systems)
-
If there is any collision or other interference a transmission can be lost.
It seems that switching to a one-wire connection would suffer from (1)? The UART->USB cables seem pretty pricey.
A more reliable approach would be to have the master (emonPi/emonBase) poll for the data. This pull-based approach avoids any collisions and allows you to retry a failed poll. Has this approach been considered?
You need to know the history.
The emonTx was conceived as a small, battery-powered unit for monitoring whole-house consumption in the form of current. Often, in the UK, there is no socket available near the meter and “consumer unit”. If a mains supply was available locally, then a voltage input and a separate 5 V USB power supply could be added. The RPi didn’t exist, the NanodeRF was used as a base station sending to emoncms.org via a wired Ethernet connection to your router. Clearly, it would have been very difficult to arrange for emoncms.org to poll every potential user, so the Nanode would have needed to poll the emonTx, and then forward the data to emonCMS.
Under battery operation, it’s necessary to operate the RFM as a receiver continuously, or at least until the first poll request is received, and then at least for a small window when a poll request is expected. Obviously, it’s possible to envisage complicated schemes to reduce the battery load.
This is still a major consideration in the design of the emonTH.
This is my interpretation as best I know the facts. If you want to know the why from the horse’s mouth, you’d better ask @glyn.hudson or @TrystanLea.
Yes that’s correct, the battery operated nature of the original configuration is the reason this approach stuck. I guess while packet collisions is an issue it’s not been a significant enough issue to warrant the development work to get a good polling system to work when considered against all of the different development activities in the project, not to say that that prioritisation is necessarily correct.
One relatively simple firmware change that could reduce the change of collisions for multiple EmonTx units would be to include a small randomised delay at initial startup, so that after power cuts where all units come on at the same time, they start transmitting at slightly different times.
Roberts recent additions are also very helpfull for this issue:
Almost that idea is in the sketches being developed to go with the emonPiCM: the start-up is delayed by 20 ms per NodeID. It won’t of course prevent the higher NodeID catching up over the next however long it takes, but it should prevent the initial rush if everything powers up at the same instant.
1 Like