Unreliable content with RFM69C and emonBase

franck102 · 2019-01-19T09:11:30.300Z

Hi all,

I am trying to use an RFM69CW to communicate with my emonBase, using the JeeLib library in compatibility mode, and I cannot figure out what I am doing wrong. I already have a node working with the same setup, but smaller payloads. As soon as my payload gets large enough the RFM69pi discards all my packets with an “unreliable content” log entry (the working node sends 3 longs, so 12 bytes + node ID + crc).
If anyone can spot something obvious I missed I would really appreciate it, this is beginning to drive me crazy…

I have attached the complete files, below are some relevant excerpts:

The node code:

typedef struct
{
    uint8_t ltarf;
    uint32_t east;
    uint32_t easf01;
    uint32_t easf02;
    uint32_t eait;
    uint16_t urms1;
    uint16_t urms2;
    uint16_t urms3;
    uint8_t pref;
    uint8_t pcoup;
    uint16_t sinsts;
    uint16_t sinsts1;
    uint16_t sinsts2;
    uint16_t sinsts3;
    uint16_t sinsti;
    uint16_t smaxin;

} TIC;
TIC tic = {
        0, 100,200,300,0,231,232,234,12,13,4200,1200,1400,1600,0,0
};
...
#define RF69_COMPAT 1   
#include <JeeLib.h> 
rf12_initialize(nodeId, RF_freq, networkGroup);
rf12_sendNow(0u, &tic, sizeof(TIC));
rf12_sendWait(0);
delay(200);

The emonHub log entries:
2019-01-19 08:55:39,016 DEBUG RFM2Pi Discarding RX frame ‘unreliable content’? 20 0 100 0 0 0 200 0 0 0 44 1 0 0 0 0 0 0 231 0 232 (-42)
2019-01-19 08:55:49,280 DEBUG RFM2Pi Discarding RX frame ‘unreliable content’? 20 0 101 0 0 0 201 0 0 0 45 1 0 0 0 0 0 0 231 0 232 (-39)

FranckemonRadio.ino (1.9 KB)
emonRadio.zip (1.8 KB)

franck102 · 2019-01-19T10:13:13.703Z

After looking into the JeeLib code, it turns out that the rf69 compatibility code is completely broken in the version I had - and I am pretty sure that was the master version of the github repo:
https://github.com/jeelabs/jeelib

After browsing the JeeLib forums for a while it appears that the RFXConsole branch is actively maintained, and using that branch instead sending works out of the box
https://github.com/jeelabs/jeelib/tree/RFxConsole-RSSI

Franck

Robert.Wall · 2019-01-19T18:06:10.980Z

This could be related or relevant:

EmonGLCD sketch issues Hardware

Bingo! I was using the recommended Jeelib v12 on my GLCD sketches. Using JeeLib from 10th Sep 2015 GitHub - jeelabs/jeelib at f097c0039c926881d80a74bec7a7aa020de610ee in my Arduino library instead has fixed it and the display has come to life! Happy Bunny - now to send proper data rather than dummy data!

franck102 · 2019-01-19T18:37:01.592Z

Frankly I came very close to opening my emonBase to see what it would take to go RFM69 & LowPowerLab library all around

If anyone has plans around that I am willing to contribute help…

Franck

Robert.Wall · 2019-01-19T20:04:25.439Z

To the best of my knowledge, LowPowerLabs format is completely different to JeeLib’s, so it would be all or nothing. If you changed the sketch in the RFM69Pi in your emonBase, you’d need to remember when you went to update it to pretend that it was an emonPi so that the upload to the RFM69Pi would fail, thereby leaving your LowPowerLabs sketch intact. And I’m guessing that emonHub would need changing too, at a minimum so as to not issue the wrong commands to the RFM69Pi

franck102 · 2019-01-20T07:02:49.681Z

I understand, and I probably won’t go there unless there is a desire to officially support this at the emonBase/emonPi level.

I am pretty sure I can create a fork of the LowPowerLabs library that could decode both native LPL packets and JeeLib packets, if that helps.

Franck

pb66 · 2019-01-20T11:28:12.679Z

I think @Lycle_Park may have an emonPi using the LowPowerLabs library. See Undefined node list up and How to emonhub full reset the emonpi?

As for emonHub supporting a LPL native packet format, that is do’able and probably preferable to trying to get one entirely backwards compatible protocol working ie running LPL in JeeLib compat mode in RF12 compat mode for a rfm69!

franck102 · 2019-01-20T13:43:37.357Z

It is probably easier, but the LPL library seems more solid, has more features, and it fully leverages the RFM69 hardware…

Franck

pb66 · 2019-01-20T13:59:49.788Z

franck102:

it fully leverages the RFM69 hardware…

but to be fair, the whole reason we still use JeeLib is because it runs in RF12 compat mode. It seems really unfair to cling on to a certain part of a Lib to retain compatibility with rfm12b and then jump ship to a lib that has additional functionality because it doesn’t maintain the backwards compatibility we demand.

I’m not biased towards JeeLib, I’m just questioning the comparison of JeeLib in compat mode (by our choice) with native rf69 in LPL. Having said that, if we want to go full rfm69 (drop rf12 compat) switching libs purely to compartmentalize rf12 compat from rf69, that might go someway to prevent confusion between different rfm69 implementations (ie rf12 compat or not). If it’s LPL it’s full rf69, if it’s JeeLib it’s rf12(compat) with no interchangeability between them.

franck102 · 2019-01-20T19:32:44.721Z

Confusing topic, not sure I followed everything
Of course the rf12 nodes out there must remain supported. What I was thinking specifically is having an emonBase with an RFM69 module running a yet-to-be-written modified LPL library that can receive both LPL and JeeLib packets.
This would fully support existing nodes, and newer nodes using RFM69 and the LPL would benefit from acks, retries, better performance, etc…

But maybe I am missing something?
Franck

pb66 · 2019-01-20T19:38:28.839Z

franck102:

But maybe I am missing something?

Indeed you might be.

All OEM rfm69cw nodes are being driven with rf12 “compatibility” code from JeeLib so that rfm12b and rfm69cw can share the same network. If the rfm69cw’s are driven with a proper rf69 lib (Jeelib or LPL AFAIK) they will not be able share the same network as an rfm12. This is nothing to do with the format of the payload, this is about the way the transmissions are made and received regardless of what the ~60bytes of payload are.

Long story short, you cannot have all the rfm69 goodies and rfm12 compatibility together, you need to choose one or the other.

franck102 · 2019-01-20T19:51:29.496Z

Hmmm, I believe I can put an RFM69 module with a modified LPL library on my existing emon network and successfully receive and decode the emon packets.
I do understand that that will require low-level modifications inside the LPL library and that this isn’t just a payload issue.

Franck

pb66 · 2019-01-20T20:16:24.596Z

perhaps we have different understanding of “rf69 features”, the acks and retries are both currently available on rf69 already even in rf12 compat mode as rfm12bs can use acks and retries too.

Things like hardware encryption cannot possibly work on rf12 because it doesn’t have it, there are physical differences between the modules that cannot be overcome in software. But if you do manage it, I’m sure JeLabs would like to hear how as a large part of JeeLib is attempting to get rf12 and rf69 working together, namely rf12 compat mode.

I don’t want to put you off trying though.

The point I make is still valid if you want to ditch JeeLib because it’s limited, and then dumb down LPL so it can communicate with rf12’s too, you will end up in a similar position as it is the rfm12b modules are limiting the feature set, through physical differences, not lib differences.

But please do prove me wrong, it would be great to move forward AND retain compatibility, it’s the compatibility bit that prevents advancement in that area.

There again this need becomes weaker as the years roll by, how many rfm12b devices are there out there that belong to networks that will be upgraded to full rf69?

For some time now, even the emonPi upgrade procedure ignores any possibility that users may still have rfm12b based rfm2pi’s, in fact worse than that, it assumes there definitely isn’t any and over writes the rf12 FW with rf69 FW causing the device to stop responding and the user has to manually re-flash the old rfm12b FW (and remember not to do the same again in the future).

In short, is it still worth holding back progress and features to the bulk of users to retain compatibility for a diminishing minority user base that may or may not even want to change their rfm network at all?

I’m only debating the options, I’m not advocating dropping support for rfm12’s, likewise I’m not looking to block moving forward either, just weighing things up, out loud . .

Robert.Wall · 2019-01-20T21:12:25.215Z

franck102:

I believe I can put an RFM69 module with a modified LPL library on my existing emon network and successfully receive and decode the emon packets.
I do understand that that will require low-level modifications inside the LPL library and that this isn’t just a payload issue.

I know nothing of the LPL library, and only a little about the receive side of JeeLib, but my feeling is you must learn exactly how both work, then design something that will be able to determine which protocol it is using. At that point, you should be able to use large parts of each library to decode the message. The big advantage you have with the RFM69 is the internal buffer, which means that if your total message will fit inside it, you can process it as slowly as you like. The RFM12B must have each incoming byte extracted and stored as it arrives, which puts a heavy and a priority load on the processor. Unfortunately, the way that OEM connects the RFM69CW to the processor means that some useful features of the the RFM are not available.

franck102 · 2019-01-20T21:15:36.501Z

What I was proposing is easier than what JeeLib does. The modified LPL lib would only support RFM69 hardware, but it could receive and process packets from any remote node.

By using it on the base (receiver side) we would get a purely native (lib and hardware) solution for newer nodes with a native RF69 setup (unmodified LPL + RFM69), and backward compatibility for rf12 nodes.

Franck

franck102 · 2019-01-25T17:34:05.929Z

I just wasted 3h again today trying to chase down hardware problems with RFM69 modules, when it was in fact more JeeLib library bugs.

Even using the RFXConsole branch the library misses the send complete interrupt (or mis-processes it) and remains stuck in rx_state = TX_SEND - i.e. rf12_sendWait(0) never returns.

The same project works flawlessly using the LPL library. The effort to support multiple radio hardwares in one library is commendable, but as far as I am concerned it has failed here and I will either purchase a few RF12 modules, or look into using the LPL to generate JeeLib packets.

I am attaching the whole source in case someone wants to have a look… but I am done trying

Franck

Radio.zip (1.6 KB)

Robert.Wall · 2019-01-25T18:09:01.521Z

Do you see the same problem with our Github version?

GitHub

GitHub - openenergymonitor/jeelib: JeeLib for Arduino IDE: Ports, RF12, and...

JeeLib for Arduino IDE: Ports, RF12, and RF69 drivers from JeeLabs - GitHub - openenergymonitor/jeelib: JeeLib for Arduino IDE: Ports, RF12, and RF69 drivers from JeeLabs

franck102:

I will either purchase a few RF12 modules…

Just a warning that, if you were thinking of using the 3-phase PLL sketch, in order to be able to transmit successfully from that sketch, it was necessary to extract the transmit code from JeeLib and modify it to work only on the RFM69CW. Therefore, you cannot use the 3-phase PLL sketch with an RFM12B.

franck102 · 2019-01-25T18:50:39.851Z

Good question, I will try to get the lib from openenergy github… once the frustration has died down

Franck

franck102 · 2019-01-26T05:32:29.214Z

Sending works with the oem version of JeeLib. The code in sendstart_compat is radically different from the one in the RxConsole branch (older).
It is very similar (maybe identical) to the code in the master JeeLib branch, however in oem’s version the variable rf12_len (a define) below indeed takes the value of the len argument passed to the function. In the master branch’s version it didn’t, leading to problems with larger payloads:

void RF69::sendStart_compat (uint8_t hdr, const void* ptr, uint8_t len) {
rf12_buf[2] = len;
// From here on for some reason the function uses rf12_len for the
// payload length
rxstate = - (2 + rf12_len); // preamble and SYN1/SYN2 are sent by hardware

Probably some include/define problem in the master branch… thanks for the suggestion, I will use the oem version going forward.
Franck

Robert.Wall · 2019-01-26T12:41:04.487Z

Thank you for looking at that. I’ll notify the administrators so that our version does not get accidentally “upgraded” to one that doesn’t work correctly.