No inputs after emonpi new SD card and CM firmware update

[rupert]

Looks like the LPL changes were carried across to the emonPi_DS_jeelib_classic firmware perhaps?

My emonPi has the LPL firmware. My emonHub node entry for the emonPi (node5 usually) is shown below. There are two entries.

The one with the # on every line is commented out (not used), and was the original jeelib_classic one before the LPL upgrade.

The one below is the working one for the LPL upgrade.

I don’t know if it will work for you …

#[[5]]
#    nodename = emonpi
#    [[[rx]]]
#        names = power1,power2,power1pluspower2,vrms,t1,t2,t3,t4,t5,t6,pulsecount
#        datacodes = h, h, h, h, h, h, h, h, h, h, L
#        scales = 1,1,1,0.01,0.1,0.1,0.1,0.1,0.1,0.1,1
#        units = W,W,W,V,C,C,C,C,C,C,p

[[5]]
    nodename = emonpiCM
    [[[rx]]]
        names = Msg, power1,power2,power1pluspower2,vrms,t1,t2,t3,t4,t5,t6,pulse1count,pulse2count,E1,E2
        datacodes = L, h, h, h, h, h, h, h, h, h, h, L, L, l, l
        scales = 1, 1,1,1, 0.01, 0.01,0.01,0.01,0.01,0.01,0.01, 1, 1, 1,1
        units = W,W,W, V, C,C,C,C,C,C, p, p, Wh,Wh

[PeteF (Peter Forrow)]

Yes that emonpiCM node configuration works, thanks @rupert !

But obviously the data is going to a new node, so am I right to say I can rename it to ‘emonpi’ in the config (as it was before) and the data will be processed as before and everything will be fine, albeit with a ~24hr gap in data?

Also, I still have no emontx or emnth data inputs working, presumably a similar problem with frame mismatches although they’re not showing up in the emonhub log at all.

I am now rather confused (a) how I got to here and more importantly right now (b) how to fully recover, especially wrt the emontxs and emonths…

[rupert]

Yes, I think renaming node 5 to emonPi will send the data to the previous feeds. I take it that the emonPi power etc measurements are appearing ok now?

At the moment I can’t think why the 433MHz JeeLib Classic data from your emonTx3s and emonTHs is not being received and appearing in the emonHub log via ttyAMA0. As it was all working up until the update (!) I think it’s unlikely that there’s a hardware problem.

My LPL emonPi interfacer looks like the following. Don’t know if this helps!

[[emonpi]]
        Type = EmonHubOEMInterfacer
        [[[init_settings]]]
            com_port = /dev/ttyAMA0
            com_baud = 115200
        [[[runtimesettings]]]
            pubchannels = ToEmonCMS,
            subchannels = ToRFM12,   

        group = 210
        frequency = 433
        baseid = 5                              # emonPi / emonBase nodeID
        calibration = 230V                      # (UK/EU: 230V, US: 110V)
        quiet = true                            # Disable quite mode (default enabled) to enable RF packet debugging, show packets which fail crc
        # interval =  300                         # Interval to transmit time to emonGLCD (seconds)  

[PeteF (Peter Forrow)]

Aha. Mine is the same except for
Type = EmonHubJeeInterfacer

I think that might be rather significant, but I haven’t changed it because I’m officially out of my depth now.

I have renamed the input and the feed is working ok, although there are some weird things I’ve had to reconfigure, it seems as though the order of the data in the frames has changed? Might be just me though…

[rupert]

My emonPi interfacer used to be called RFM2Pi and used the Type = EmonHubJeeInterfacer …

I think that the introduction of the emonTx4 has meant various changes to the infrastructure …

[PeteF (Peter Forrow)]

Crossed messages! So what caused yours to change?

[rupert]

To be honest, I can’t remember why I changed it - the reason may be somewhere in all the threads that I referenced. But as I said in the crossed message, I think that the introduction of the emonTx4 has meant some changes to the infrastructure …
For example, the emonTx4 with the 6 channel extension board, can measure 12 power channels. The data this produces will not fit into the RFM69 buffer, so it has to be sent as two separate packets and then recombined in the emonPi. I think (not sure?) that the EmonHubOEMInterfacer does this …

[rupert]

Have you tried the EmonHubOEMInterfacer to see what happens?

I think it’s supposed to be more flexible than the EmonHubJeeInterfacer.

This link (more reading, sorry) explains the interfacers:

https://docs.openenergymonitor.org/emonhub/emonhub-interfacers.html#list-of-interfacers-links-to-github

Although I’m not sure if the EmonHubJeeInterfacer worked for you before, why it doesn’t now?

[PeteF (Peter Forrow)]

Yes I have tried it now, appears to be working although I have these errors on restart:

2023-10-25 13:01:58,063 DEBUG    RFM2Pi     ---------------------------------------------------------------------
2023-10-25 13:01:59,063 DEBUG    RFM2Pi     Config format: new
2023-10-25 13:02:02,166 ERROR    RFM2Pi     CONFIG FAIL: group cmd: g210 (no reply)
2023-10-25 13:02:03,167 ERROR    RFM2Pi     CONFIG FAIL: frequency cmd: b433 (no reply)
2023-10-25 13:02:04,168 ERROR    RFM2Pi     CONFIG FAIL: baseid cmd: i5 (no reply)
2023-10-25 13:02:04,168 DEBUG    RFM2Pi     ---------------------------------------------------------------------

I have no idea why these are coming up, my RFM2Pi interfacer is the same as yours and the emonpi inputs and feeds appear to be ok.

Any idea what I need to do to get my emontxs and emonths working? The config for the emontx, for example is:

[[15]]
    nodename = emontx3cm15
    [[[rx]]]
        names = MSG, Vrms, P1, P2, P3, P4, E1, E2, E3, E4, T1, T2, T3, pulse
        datacodes = L,h,h,h,h,h,L,L,L,L,h,h,h,L
        scales = 1,0.01,1,1,1,1,1,1,1,1,0.01,0.01,0.01,1
        units = n,V,W,W,W,W,Wh,Wh,Wh,Wh,C,C,C,p
        whitening = 1

This is as it was (working) using emonSD-24Jul20 and emoncms 11.3.0 yesterday before I updated to emonSD-10Nov22 and emoncms 11.3.22.
After all, you wouldn’t have thought this was an unusual or compatibility-breaking update?

Perhaps @Robert can help? I haven’t updated the emontx3 firmware since I helped with testing your CM firmware several years ago Robert. I can’t recall if there is an easy way to find out which version the emontx3s are running…

[rupert]

from the docs link in my previous post:

OEM Interfacer

Replaces EmonHubJeeInterfacer with a more flexible implementation that can accept a range of different formats from connected devices, including OpenEnergyMonitor devices. Here are a number of supported data formats:

    Decimal space seperate representation of RFM binary data e.g OK 5 0 0 0 0 (-0)
    KEY:VALUE format e.g power1:100,power2:200
    JSON format e.g {"power1":100,"power2":200}

Example configuration:

    [[OEM]]
        Type = EmonHubOEMInterfacer
        [[[init_settings]]]
            com_port = /dev/ttyAMA0
            com_baud = 115200
        [[[runtimesettings]]]
            pubchannels = ToEmonCMS,
            

so perhaps the OEM Interfacer doesn’t need the following commands, that cause the errors?

group = 210
        frequency = 433
        baseid = 5                              # emonPi / emonBase nodeID
        calibration = 230V                      # (UK/EU: 230V, US: 110V)
        quiet = true                            # Disable quite mode (default enabled) to enable RF packet debugging, show packets which fail crc
        # interval =  300                         # Interval to transmit time to emonGLCD (seconds)  ype or paste code here

and if it did cause errors here I might not have noticed them …

[rupert]

My node 15 configuration looks the same as yours:

[[15]]
    nodename = emontx3cm15
    [[[rx]]]
       names = MSG, Vrms, P1, P2, P3, P4, E1, E2, E3, E4, T1, T2, T3, pulse
       datacodes = L,h,h,h,h,h,l,l,l,l,h,h,h,L
       scales = 1,0.01,1,1,1,1,1,1,1,1,0.01,0.01,0.01,1
       units = n,V,W,W,W,W,Wh,Wh,Wh,Wh,C,C,C,p
       whitening = 1

However, not wishing to confuse the issue here, but for full disclosure …
In my case I did the following:

Upgraded my emonPi firmware to emonPi_CM_LPL_1_1_4.hex
Edited my emonP interfacer to EmonHubOEMInterfacer (pointed at ttyAMA0) and node 5 config to suit
Which meant I got working CM and LPL on my Pi
But I couldn’t receive Jeelib format from my emonTx3 any more, unless I updated it
So I added a JeeLink USB module (which can receive the Jeelib protocol)
Added an EmonHubJeeInterfacer interfacer (pointed at USB)

So my emonPi now has two receivers and can receive both JeeLib and LPL, Although I have had some data dropout problems, which can be resolved by restarting emonHub. Still working on that!

I haven’t ever tried to return my emonPi atmega328 to JeeLib Classic.

[Robert.Wall]

@PeteF

Perhaps @Robert can help?

I’ve looked at my crib spreadsheet and I’ve only got that configuration for all the CM, so it should be correct.
What does emonHub log tell you about the emonTxs (there’s 40 bytes expected by that Node 15 config.) and the emonTHs?

[rupert]

@PeteF
Are your emonTx3s still running Robert’s CM firmware? If so they might be using a third (!) RF format called JeeLib Native - which is not compatible with either LPL or JeeLib classic …

Similarly with your emonTHs …

If they are, it means that you would have had CM JeeLib Native firmware in your emonPi atmega328 to receive it …

[rupert]

@PeteF
I think that to find out what firmware your emonTx3s are running you would need to put a serial to USB adapter on the emonTx3 serial port, and connect the USB to a computer running a terminal program (e.g. putty). Then look at the terminal to see what the start up text says when the emonTx3 is turned on.

see:
https://docs.openenergymonitor.org/emontx3/direct.html

[PeteF (Peter Forrow)]

@Robert.Wall

What does emonHub log tell you about the emonTxs (there’s 40 bytes expected by that Node 15 config.) and the emonTHs?

Absolutely nothing. No entries from any of the emontx or emonth nodes.

[PeteF (Peter Forrow)]

@rupert

I think that to find out what firmware your emonTx3s are running you would need to put a serial to USB adapter on the emonTx3 serial port, and connect the USB to a computer running a terminal program (e.g. putty). Then look at the terminal to see what the start up text says when the emonTx3 is turned on.

Yes I can do that, as I have updated the firmware a few years back. Just have to dig out the old iMac and cables!

[PeteF (Peter Forrow)]

@rupert

Are your emonTx3s still running Robert’s CM firmware? If so they might be using a third (!) RF format called JeeLib Native - which is not compatible with either LPL or JeeLib classic …

Yes they are still running it. I will connect one up and find out what version the emontxs are running this afternoon.

@rupert

If they are, it means that you would have had CM JeeLib Native firmware in your emonPi atmega328 to receive it …

Entirely possible I guess. Any way to find out?

[rupert]

Well the upgrade of the emonPi atmega328 to emonPi_CM_LPL_1_1_4.hex will have erased the previous firmware, so it’s gone and I don’t think you can’t find out. Possibly there might be some information in the Raspberry Pi logs if there are still any from before the update, but that’s just an ill-informed guess!

Robert’s CM firmware was written to allow CM on the emonPi instead of the discreet sampling method

@Robert.Wall

@rupert

To help your choices - I don’t think that you can get Continuous Monitoring on the emonPi if you use the JeeLib Classic radio format.

This is correct, you absolutely cannot have CM on the emonPi with JeeLib Classic, and this is exactly the reason why I developed emonLibCM and the ‘Native’ format, because it unloads a lot of the radio processing to the RFM68CW, which the RFM12B (for which JeeLib Classic was made), can’t do.

but this required a new RF Format called JeeLib Native on the emonPi. As the emonPi is the data logger for the system, Robert wrote JeeLib Native updates for the other members of the OEM family such as the emonTx3 and the emonTH, so that they could send JeeLib Native format data to the CM emonPi. It seems this is what’s in your emonTx3 and emonTH and was in your emonPi until the update.

I think you will have to choose what radio format you want to use in the future and whether you want CM in your emonPi.

If you want CM in the emonPi you can’t use Jeelib Classic. You will need JeeLib Native or LPL.

New emonTx4s and emonTHs are supplied from the OEM shop with a choice of firmware:

1. Standard (RF encryption) This is LPL
2. Existing hardware compatibility (<2022) This is JeeLib Classic

The new emonBase is supplied with LPL by default:
Note: by default the new version of the emonBase with the RFMSPI is NOT compatible with older hardware e.g emonTx V3. Please contact us if you would like a version which is compatible with older hardware.

Robert has details on using JeeLib Native at:

(https://community.openenergymonitor.org/t/emonlibcm-version-2-2-2/9241)

One more caveat - if you have a Mk2 Router from Robin Emley, it only uses the Jeelibs Classic format.

A tangled web!

You can update an emonTx3 by connecting it to your emonPi. I’m haven’t checked what formats are available - perhaps Jeelib classic and LPL:

(https://docs.openenergymonitor.org/emontx3/firmware.html#updating-firmware-using-an-emonpi-emonbase)

Similarly with an emonTH2:

(https://docs.openenergymonitor.org/emonth2/firmware.html#updating-firmware-using-an-emonpi-emonbase)

Hope this helps

[PeteF (Peter Forrow)]

Yes that helps a lot. I hadn’t realised that so much had changed with regards to the RF side.
To be honest, I was pretty happy before I started updating, only a few minor-ish considerations that prompted me to try and get up-to-date.
My tx3’s did/still do CM, thanks to Robert’s past efforts, and that’s where the majority of my inputs enter the system. The emonpi didn’t, but it never caused me too much real pain. CM doesn’t matter on the emonth’s for my application of them - slowly changing sources.

So what I’d prefer is to get the emonpi/atmega328 back to CM JeeLib Native firmware, on the assumption that this gets me back to where I was but with up-to-date emoncms. I’m not proficient enough to find out if that’s possible or how to do it, since I don’t think it’s an option from the Admin firmware update drop-downs. Any ideas?

One days I’ll perhaps order a set of new hardware, but my concience pricks me to use the ‘old’ stuff for a while longer…

[rupert]

Is the emonPi CM Jeelib Native firmware file still on your iMac …

And perhaps the instructions on how to use it …