Logging kWh using newer EmonTX CM sketch

Hi,

I’ve recently added some more EmonTX units to my installation which carry the newer CM sketch. The outputs from this have changed from my existing units such that they now output 4 power, 4 energy, 3 temp, pulse, voltage, msg and rssi.

My previous units didn’t output the energy, instead, when logging the power, I would also do a Power to kWh process and log that separately. This is in line with the guide here 2. Log Locally - Guide | OpenEnergyMonitor

Is there a different way that this should be logged now using the 4 energy outputs from the newer EmonTX sketch? If so, can someone confirm what it should be, is it as simple as logging the energy output to it’s own feed? If so, I noticed that when the EmonTX is rebooted that these values revert to 0, does this cause a problem?

Thanks

You can use whichever method you like, or both. The E(nergy) values were added after the Guide was published, and it’s not been updated.

Yes, using the energy values is as simple as logging to a feed - obviously one that is separate from the one resulting from the power-to-energy process step.

The advantages and disadvantages are:
If data is lost in transit, the power values stop and the power to energy process in emonCMS can’t happen, whereas provided the emonTx carried on working, the energy will be totalled at source and the true value recorded again by emonCMS when communication is restored.

If the emonTx resets, even momentarily, the energy value restarts from zero as you’ve observed. Unfortunately, due to the limited life of the EEPROM, the last value can’t be saved so it cannot carry on from where it left off. It shouldn’t be difficult to create a process within emonCMS that remembers the last value, and if the next is close to zero and the change has been sufficiently great, to add the saved value to each subsequent reading - the big problem will most likely be deciding when to apply the ‘correction’. It’s likely the message number will be useful for this - that too will reset to zero when the emonTx resets or powers up. Whether Trystan intends to do this, I can’t say.

The message number allows emonCMS (or you, if you log them) to see where the communications failed and how many messages were lost. I think it’s not yet used for anything.

The EEPROM is spec’d at 100,000 writes.
Sure sounds like that’d be enough to last for more than just a few months…

Since the emontx never knows which value will be it’s last, it has to write every value. 100000 × 10s / 86400s = 11.6 days.

Speaking strictly about an integer kWh total, (which might change very slowly or quite rapidly, or anywhere in between) the value wouldn’t need to be written if it hasn’t changed. Reading an article that mentioned the EEPROM update function brought that to mind. The function reads the memory location and skips the write if the “new” value is the same as the “old” value. Ref: https://www.arduino.cc/en/Reference/EEPROMUpdate

The 100,000 write limit is for each byte, so if only half of the EEPROM were used,
then 11.6 days x 512 = 5939 days which is a bit more than 16 years.
Odd as it sounds, an EEPROM wear leveling library actually exists.
Wear leveling on a microcontroller's EEPROM - Electrical Engineering Stack Exchange

Microchip appnote about high endurance EEPROM storage:
https://www.microchip.com//wwwAppNotes/AppNotes.aspx?appnote=en592140

A big issue would be is there enough time between calculations to do any of this?
(the EEPROM write itself takes 3.3ms)
Another would be is enough flash memory available?

There are probably as many possible algorithms as members here, and whichever is chosen, someone is going to like it and someone else will call it total rubbish. I did think about it when writing the library, but decided that the library was the wrong place for something like that.

It would be nice to know what sort of memory commercial meters use for this, and what and when it’s updated.

The GE kv2c meter that feeds my house uses EEPROM for data storage.

Excerpt from the manual:

1.3.1.4 Nonvolatile Memory

The kV2c meter is equipped with two nonvolatile memory devices. All data values and program parameters are stored in semiconductor nonvolatile memory, battery is not required for data storage. EEPROM memory stores programmed operating parameters and meter data. Flash memory is used for special program provisions and memory intensive data requirements. Meter data quantities are updated at each power fail event. Stored data is constantly checked to prevent errors.

That’s the pertinent bit. I don’t think there’s enough time between sensing a mains fail and the supply dying, but it might be worth trying. I’ll add it to the jobs queue - at the present rate, some time next year is when I’ll get around to it, if it doesn’t get pushed down by more important things.

Makes me wonder if they use a supercap in the DC power supply to give them the
time to get things done before it dies completely.

Hi,

Thanks for the feedback thus far. The drawbacks of the total energy consumption being lost following an emonTX power fail are too significant I think to outweigh the benefits for me currently.

With that said, would the following process not work? Seems a bit long winded, but reads ok to me? Also, can the WhAccumulator be used in this scenario? Do I need anything special to use this?

The difference is, the meter is a law unto itself, whereas we depend on a certified a.c. adapter and can’t drag too much current for fear of putting a dint into the waveform we’re measuring. So they can easily charge a supercap - or even a big electrolytic. The emonTx can’t survive a second missing mains cycle if it’s starting from minimum supply voltage.

Seems my process doesn’t work, the feed value recorded is yoyo’ing between the input value and a doubled up value…

I’m not convinced by that maths. But I’m not gonna argue as I’m not an expert, I do know that the idea of recording energy readings in an emonTx used to come up a lot and was always discounted as unfeasible/unsustainable.

Of that I have never had any doubt

As do many

That is the trick, whilst wear levelling and or cleverly minimising the writes will of course extend the life significantly. Neither comes remotely close to having HW purposely designed to facilitate and prioritise the eeprom writes at power off that negates the intermediate writes altogether, an eeprom could last forever if the power supply never goes down. Even if wear levelling and minimal write were applied to an emonTx with a rarely changing energy value, it’s life is still eroding every time there is change, even if the supply was indefinitely perfect and never failed, which is a highly unlikely combination, in fact it would most probably be a pointless monitoring project if there so little activity as to cause the eeprom to last.

I’m not saying it’s not possible, coz’ we know many devices do it, just not the emonTx IMO.

Try using the rather snazzily named “pulse_count_to_pulse_increment” process, ignore the “pulse” bit and regard it as a “count to increment” process and follow it with an accumulator process (not a (k)waccumulator, it works much better than the (k)waccumulator.