Oh, and by the way I’m happy to help on the emonGLCD side, as I think I’m the last person who touched it.
Oddly my emonGLCDs came with RFM69 so I’m not sure if there were some shipped with RF12 originally.
Anyway, the emonGLCD suggestion above does did not work on my RFM69 based board, it failing to transmit when it was applied (not surprising in fairness).
I’m happy to workup new RF code for it as needed (20 minutes playing and I’ve got the sketch transmitting with rf69.h based code).
That is the title of the Topic, but also where I was not sure what exactly, in some places, the subject was, because of the rather frequent use of “it”.
I have had nothing to do with that. If you check the references, it is completely down to the work of JCW and his associates. Some of the r.f. properties of the transmission are subtly different, so a careful study of the differences between the JeeLib and the RF69 libraries might be fruitful.
I think you’re absolutely correct there. I reported the RSSI bug when I found it - the response is still awaited. Also note there are hints that some of the RFM69 settings were less than optimal - again I’m not aware of any update. So I’m not holding on to any hope of support from that direction.
As far as I’m aware, the emonGLCD is obsolete =, and has been unavailable as a self-build kit for a good while (regrettably, but I fully understand the reason it was withdrawn). Converting it to use the RFM69 Native library (for transmission, if you require the extended capabilities) and the RF69 for reception, or the RF69 for both, should be simple, I considered it not worth the effort for the few systems that might, only might, still be running and needing to be converted.
I know I soldered up two emonGLCD kits, both had RFM12Bs, I also remember a post emonGLCD as an RF signal strength meter | Archived Forum that took a wire from the RFM12B to recover signal strength. Until I read this, I was under the impression that no emonGLCD kits included the RFM69CW, and all had included the RFM12B. @glyn.hudson or @TrystanLea are likely to be the only ones to know just how many of each were shipped, and I think they also must be drawn in to give their opinion on your contribution.
I think we largely concur.
The rfm69 emonglcd are an easy conversion and I’ll get that done. I guess the thing we need to agree on is the sending api. I’m not sure many folks use it. A few do but I think small numbers. The fact it was broken for a few years in the most recent release of emonhub (after the python3 conversion) leads me to that conclusion.
Maybe it’s time to revisit the topic as you say. Especially for emonpi2 which uses another avr which I have no familiarity with. Or perhaps makes sense to have the bridge between the two defined and then it can be uC independent.
I’ve also never used low power lab but it kind of seems nice to have a common ecosystem. Although the rf69.h simplicity and portability are nice. Essentially any device that can drive SPI can send and receive super simply.
I’ll take a look at low power lab some more to further my own knowledge
Out of curiosity do you know if all emonth shipped with rf69 ?
Maybe we are lucky and the vast majority of oem devices are rfm69 based ?
It’s not the frequency settings is the issue it really is down to buggy code in jeelib rf12 compatibility layer.
Sorry to drag you back into this world.
Oh and can you confirm that your rf12 based emonglcd in rf69 mode worked (at least for it to tx it’s temp every 30 secs) ?
I looked very hard before deciding that it might be a case of “The devil you know…” A serious worry is the 1% limit on band occupancy and how it interacts with the very low data rate of LPL, which clearly isn’t a problem with environmental sensors where things change slowly, but which could be restrictive where energy monitoring and the much faster rates of change are encountered. I might be wrong, but from memory LPL won’t handle the RFM12B.
The real bugbear was Hope’s decision to bake into hardware a message format that was just two data bytes transposed from JeeLabs’. Most inconsiderate 
Again, most certainly not. The original was RFµ with a RFM12B, then the RM69CW
Yes, a decision was made to switch completely to the RFM69CW, I’m not sure when as a date, but around the time of the launch of the V3.4. But I have a V3.4 with a RFM12B - maybe it was an early one for testing. The V3.2 used the RFµ328 and a RFM12B, the V2 also always had a RFM12B.
No - I can’t remember, and I don’t want to side-track even further (though the hot weather precludes logical thought - as you might have noticed.)
I too am not impressed by the patch to JeeLib, it’s implementation (forced by the Arduino IDE’s concept of a library) means editing source code to swap? Why can’t a library have a path? Utterly and totally wrong and the worst possible hack, but I recognise it might have had a certain appeal. It’s why rfm69nTxLib exists, and I don’t propose to do anything to implement a receive version. My view is the future lies with the Hope data format and the RFM69CW. To me as a systems person, it makes complete sense to hand over the entire transmission and reception path to a subsystem: put a message in at one end and get it out at the other. Handling one byte at a time was the only way with the very limited capabilities of the RFM12B, but the basic concept was flawed - because it was pushed too far. It worked with emonLib because the processor was doing nothing for 99% of the time, so it could respond to interrupts on each byte sent or received.
I am now more enlightened on jeelib and rfm69nTxLib.
I’ve just compared the code for the send functions and rfm69nTxLib is, well, elegant.
I can easily see the data been sent in 2byte hunks via SPI.transfer where in jeelib, it’s a bit of a mess. I think there is a bug in jeelib around a memcpy when RF12_COMPAT is enabled, but damned if I can find it. That makes sense with the behaviour I’m seeing, and as the packet is 2bytes longer with RF12_COMPAT, I think there is a bug there somewhere. But I digress.
What I’ve learnt is worth gold the latest being just why rfm69nTxLib exists. It is much cleaner than jeelib and it is possible to get a RF12 to send RFM69 native packets reliably using it. We just can’t RX using it (yet).
I just need to put pieces of the jigsaw together now… so I’m going to shut up a while as I’ve a lot to be going on with the information you’ve given me, which again has furthered my understanding yet further.
I’ll be in touch and thank you again.
Alan
For what its worth, when I tried to use JeeLib receiving with the emonLibCM library running (i.e. doing what the emonPi has to do - measure and receive from another node), the success rate was not good - from memory about 20% of incoming data got through. It seemed pointless to continue.
Nor won’t – for the very simple reason that the processor (the '328P at least) can’t avoid dropping samples or incoming bytes when it has to handle each incoming character separately, along with sampling 30 - 40 samples per 20 ms cycle across the 3 ADC channels of the emonPi, and process and send the data onwards to wherever.
And that’s ultimately why, in the absence of the chip shortages and as they mentioned, G&T were intending to go with a STM32 solution for the emonTx V2 and emonPi2, rather than being forced into the AVR-DB in order to have a product to sell.
It’s all making sense now.
Because as you said it has to poll and suck data out 1 byte at a time from the RF12, if it’s busy doing other things, then tough… it’ll miss all the data. Which is why now I understand the RF12 sucks for this application. As you said earlier, you want all of this TX/RX offloaded to the hardware and the AVR deals with it when it’s ready.
Good thing there is no emonpi that has a RF12 afaik.
This leaves the cases of emonglcd that needs to RX and, well, being frank, my rollerblind controller jeenodes, which at this stage I’m tempted to rip off the RF12 and replace with RFM69… but where would the fun in that be.
Since your last messages I’ve been staring at the jeelib code trying to figure out where this transmit is going wrong… and I’m not further… Bed time now though…
I’ll keep you posted as I loose more hair.
Back again.
Looking at the emonhub code, it seems someone has already created a new interfacer to talk using the new command format used in your CM code. It’s called EmonHubOEMInterfacer.
However there are small issues with it - both in the send functionality which has not been updated at all and still uses the byte based approach with a ‘s’ suffix for send. I’ll take a look at that today.
You may want to add a note to your original posting and emonPiFronEndCM code where you talk about editing emonhub.conf to edit it as follows :
[[RFM2Pi]]
Type = EmonHubOEMInterfacer
[[[init_settings]]]
com_port = /dev/ttyAMA0
com_baud = 38400 # 9600 for old RFM12Pi
[[[runtimesettings]]]
pubchannels = ToEmonCMS,
subchannels = ToRFM12, emonCMS
group = 210
frequency = 433
baseid = 5 # emonPi / emonBase nodeID
quiet = true # Report incomplete RF packets (no implemented on emonPi)
calibration = 230V # (UK/EU: 230V, US: 110V)
# interval = 20 # Interval to transmit time to emonGLCD (seconds)
Again, I’ll take a look at getting send working again, but we are a lot closer than I thought.
FWIW, I’ve made some progress debugging JeeLib sending with RF12_COMPAT. It seems to only be a problem in rfdemo sketch as it somehow corrupts the groupid and increments it by one when the PROGMEM help text is enabled. It’s very weird but I’m putting this down to a corner case in this codebase for now. If the bug is indeed confined to RFDEMO and not Jeelib RF12 then we have a way to send and receive RFM69-format packets using RF12 as JCW intended, however it is not suitable for CM applications due to the interrupt-driven, slow nature of both send and receive. But it does allow a way forward on the older RF12 OEM devices that may have RF12 that need to send and receive (emonGLCD for example).
[Formatted by Mod]
True (I think) but that gives the wrong impression. The RFM69CW in the emonPi NOT running the CM code uses the RFM69 as if it were a RFM12B, i.e. with a 1-byte message buffer that is emptied as a result of the interrupt generated when the byte arrives. So it might just as well be a RFM12B, which is why the emonPi won’t do CM until you swap all the software in your system and adopt the Native message format, which allows the RFM69CW to handle reception and assembly of the message into a string in its buffer all on its own.
Absolutely.
That’s the end point I’m trying to get to:
emonPi - RFM69 - RF69 Native Packets - CM firmware
EmonGLCD - RFM69 - RF69 Native Packets - rf69.h library
EmonGLCD - RF12 - RF69 Packets - jeelib library (RF12_Compat)
I think you’ve the other cases covered pretty well (emonth, emonTx).
I’m making good progress on it.
The good news is we don’t need no stinkin’ reverse polish anymore - EmonHubOEMInterfacer.py was easily modified.
I’m just working on the last little bugs in the sending and should be there shortly. The good news is it required minimal changes in your emonPiCM firmware.
When I’m done, hopefully later on, I’ll get it on GitHub so you can have a look.
Then, we should be well on our way to getting the CM firmware adopted as the ‘default’ on emonPi (well, emonPi2 at least, with an option for emonPi1 users to do it).
I presume you realise that emonLibCM for the AVR-DB will be a lot different to emonLibCM for the '328P.
I didn’t realise that (doh!) … but as long as the serial API is the same between the PI and AVR and you continue to use EmonHubOEMInterfacer.py we should all be good.
It’s not as if you are going back to using a RF12 in emonPi2, so I assume the intent to stay with RFM69 packets.
It’s quite obvious that it will change dramatically, from handling 1 voltage and 5 currents on a single phase to 3 voltages and 12 currents across three phases or a split phase.
The API will stay as close as possible to the existing - but that’s as far as it will go. For example, each c.t. will need to know which voltage or voltages its load is connected to.
Note, there’s nothing in emonLibCM (old or new, or indeed in emonLib) that interacts directly with the radios, or any other means of communication. It’s the sketch that handles extracting the data that the library provides, formatting it and despatching it.
In that sense, operation is exactly analogous to the RFM69 Native mode: the sketch polls the device via its library (RFM69…lib or emonLib…) and if new data is present, retrieves it and does whatever is necessary with it.
Indeed, obvious now you’ve spelt it out 
Those changes may (and likely) will necessitate changes in EmonHubOEMInterfacer.py, or even provision of another interfacer to talk to it. And that’s all fine.
As long as you try and preserve the sending function, and still spit received data out, all good.
Anyway… GOOD news.
I have literally, this second just got my emonGLCD (with RFM69, using rf69.h) sending and receiving native packets through your emonPICM firmware.
There were a few subtle changes required (in essence the 2 extra bytes (header + srcId) now being part of the payload), but easily handled.
Small code changes in the emonPiCM firmware - contained in only the handling of input after cmd T issued and addition of destID in the send function (as opposed to only 0 for broadcast).
I’ll get all of this on GitHub shortly for you to look over … then we’ll have to figure out how to handle RF12 based emonGLCD as I don’t have one, and ideally be nice to have a common codebase.
If you tell me what your suggested changes are, I’ll look at them. But I won’t be backtracking to test anything with an emonGLCD any time soon. I’ve had to stop work on the -DB due to the temperatures, and won’t be restarting until into next week, then I’ll be trying to make up for the lost time.
Here you go:
That is your code with my small mods added for your review.
If testing, you will also need the 2 interfacer files in the same repo - these both were broken as the send functionality was not tested after the python3 move. Please do review and consider adding the changes proposed so emonGLCD can work.
This is updated emonGLCD code for RFM69 based only. I include it so you can see how it sends and receives data. I do plan to make this adaptable to support both RF12 and RFM69. At this stage it is here to prove the entire end to end TX & RX stack using emonPiCM and emonGLCD works.
Let me know your thoughts.
That doesn’t help - can you attach or PM the files - with the change notes of course.
Hi Robert
You’re really should get onto Github, it makes collaboration with you much easier, and brings a multitude of collaboration features that posting code on a forum doesn’t. Your choice I guess, but it makes it harder for others to collaborate. And this way it’s easily merged into the emon codebase as Glen and co can also easily merge changes. With your blessing I’m hoping to make a GitHub branch that includes all your + my changes that Glyn can just merge in, making it an easy choice which F/W people can run on emonPi. Certainly going to get the interfacers committed as they are just broken at present.
Here is a zip of your emonPICM code. The only things I changed really were the handling of ‘T’ command so it is byte-wise to match how it’s always been done and what emonhub expects and the’ RF Data handler - outbound’ routine to accommodate destination node ID. Having everything with a dest of ‘0’ (broadcast’) creates a lot of extra traffic on the AVRs that RFM69 can filter out now. Both changes documented in the GitHub readme.md, but I’d advise a quick diff to ensure nothing else snuck in.
Again, please let me know your thoughts.
emonPiFrontEndCM.zip (64.1 KB)