Emontx3-continuous - New continuous monitoring firmware

Pull downs did the trick for the most part, IRMS2 and IRMS3 were reporting small-ish values (~0.15), therefore, I also set the ICAL value for the unused inputs to 0.00000.

@WhichWayWazzit, very nice, congratulations. This is rewarding for me as well since it is the first successful deployment outside of my own house! :slight_smile:

I do have on the list to have an “ignore” capability. I’m working on another all-consuming project just now, but will come back to this firmware.

Craig

@Robert.Wall, how does one interrogate the Dallas temperature sensor and not lose samples? If I’m doing my math right, a OneWire 64-bit scratchpad transaction takes something like 88 bits, and at 15.4 kbps, this would take about 5.7 ms. This is somewhere between a quarter and a third of a 50-60 Hz mains cycle spent busy-waiting. I assume there is a ring buffer of some substantial size?
Thanks,
Craig

I’ve not yet had time to look at your code to see how you do it, but using the library for everything isn’t the way. What I do is fire off the request to convert in advance of the time I need the result, then come back and retrieve the answer just before the due time to report the temperature (along with everything else). In other words, you don’t hang around waiting for the sensor to do the conversion. That lead time is variable, depending on the resolution you ask for.

Thanks, understood. I was actually asking about just the OneWire transaction itself. The temperature conversion inside the device takes something like 750 msec, which is obviously very long and impossible to busy-wait for.

But my question was the basic OneWire request for the 64-bit scratchpad value. By my calculations the transaction time (request scratchpad + 64-bit reply) is actually a significant fraction of a 50 Hz (or 60 Hz) mains cycle.

Craig

I can’t remember now. I’m fairly certain there was no observable interruption to sampling though.
If I can find time tomorrow, I’ll try to measure it, but “a significant fraction of a 50 Hz (or 60 Hz) mains cycle” sounds wrong to me.

Found what I was looking for. On page 5 of AN 937 “Book of iButton Standards”

“Commands and data are sent bit by bit to make bytes,
starting with the least significant bit. The synchroniza-
tion of master and slave is based on the sharp slope that
the master generates by pulling the data line low. A cer-
tain time after this slope, depending on data direction,
either the master or the slave samples the voltage on
the data line to get one bit of information. This method of
operation is called data transfer in time slots. Each time
slot is independently timed so that communication
pauses can occur between bits if necessary, without
causing errors. Figure 1–5 illustrates the general char-
acteristics of this communication.”

There’s a picture of how long it takes to read the scratchpad relative to a mains cycle here.

The typical bit-banging OneWire librarires sit in a busy loop with interrupts disabled for:

. 70 usecs for the initial reset pulse
. 10 usecs to write a ‘1’ bit
. 65 usecs to write a ‘0’ bit
. 13 usecs to read a bit

then you can take as long as you like before you do the next bit. I’m not sure what speed you run the ADC at, but if it’s at 110 usecs, then I’d expect some lost interrupts during the 480 usecs reset pulse (which starts each one wire transaction - but only once at the beginning of the entire transaction, you can see it in the scope trace above).

If you offload it all to a uart as per Tutorial 214 then there’s no timing critical bits for the cpu at all, and interrupts are never disabled to do OneWire transactions.

@dBc, thanks. The value of 15.4 kbps for bit speed seems to be somewhat of a “standard” (which is 65 us per bit). Even ignoring the reset pulse, I was doing the math as
65 us x (2 request bytes + 9 scratchpad reply bytes) x 8 bits/byte = 5.7 ms
Add to that the 480 us reset pulse and then it’s more like 6.2 ms total transaction time, at least. This appears to agree with your chart that shows ~7.3 ms. This is just for the scratchpad request, ignoring the actual conversion time, which can be up to 750 ms. But the 6.2-7.3 ms transaction time is definitely a significant fraction of mains cycle.

The DallasTemperature library, which is used by the emontx3 (standard) firmware uses, does all transactions in a busy-wait loop. For a scratchpad read, the reset, request bytes and reply bytes are all performed in a blocking manner (i.e. waits using delay()). Looking at the lower level library, OneWire, it also uses busy waits to perform bit-level and byte-level transactions, and leaves interrupts disabled for up to 70usec.

The current published version of emonLibCM does not read the temperature sensor. None of the popular libraries I can find are able to perform non-blocking OneWire transactions. So I was trying to figure out how the new emonLibCM does it, and whether I could learn from it. Thanks,

Craig

When I discovered that that version has serious defects and gives wrong values, I asked for it to be withdrawn, but it never was. It should not be used.

OK, I was not advocating to use the current published version of emonLibCM.

(emphasis added)

I realised you probably knew (else you wouldn’t have reinvented it), but I put the warning in just in case somebody who hadn’t read the history saw the mention of it and thought “That looks like a good idea, I’ll use that.” It was a good idea, but it suffered a few serious errors during conversion from sketch to library, so they’d get badly wrong answers for some values.

I was going to say 480 usecs, but you’re right, they’re not disabled that long during the reset() sequence. I’ve corrected my claim above that they are. I’m not familiar with the emon library code under discussion so can’t really contribute further on the details.

OK, I’m back at base now and can give a bit more of an informed answer…

Yes, looks like the ds18b20 can do anywhere from 60 to 120 usec per bit. The uart offload solution I use uses a bit time of 87 usecs (uart set to 115200 baud and each uart byte representing one OneWire bit) which explains why mine is coming in a bit slower than your calculations.

And I think that assumes only one sensor per bus. If you’ve got multiple sensors per bus then 2 request bytes becomes 10 request bytes because you need to include the ROM code of the device whose scratchpad you want to read. There’s a pic of that here. Using my timings that takes ~13 msecs (not including the reset). The Green and Yellow traces are two different OneWire buses running simultaneously.

Were you able to find it anywhere?

Good points!

Nope, @Robert.Wall has not given any information yet.

@WhichWayWazzit, there is a new firmware release (version 1006) which adds the ability to ignore some current transformer channels. It also fixes a bug regarding pulse counting. Available from the github page listed in the top post.

Just for reference, Robert Wall’s EmonLibCM library was publicly released in November 2018.
Forum here: EmonLibCM - Version 2.2.1 (Released 30/10/2021)
Github here: GitHub - openenergymonitor/EmonLibCM: Continuous Monitoring alternative to EmonLib

EmonLibCM samples at half the rate that Emontx3-continuous samples, so it will be about 38 samples per 50 Hz AC waveform (or 32 samples per 60 Hz AC waveform).

EmonLibCM allows multiple Dallas OneWire style temperature sensors. Following up about our interest about how it handles busy-waits, it appears that EmonLibCM accumulates power measurements within the interrupt handler for the datalogging interval. So a busy-wait within the DallasTemperature library will in principle be interrupted by ADC sample measurements.

In practice however, DallasTemperature leaves interrupts disabled for up to 70 usec during its bus transactions. Since EmonLibCM samples with a prescaler of 128, the ADC sample interrupts occur every 104 usec (using system clock of 16 MHz, prescaler of 128, and ADC sample time of 13 ADC clocks). Therefore, depending on the timing of OneWire transactions and ADC samples, there will probably be occasions where a single sample is missed.

Other areas of difference: Emontx3-continuous calculates both real, reactive and total apparent power, power factor; and the mains voltage, frequency and quality factors. Emontx3-continuous report samples less frequency (30 seconds versus 10 seconds for EmonLibCM), but it will report immediately if power usage changes suddenly.

So if you are interested in a more detailed look at your power usage (finer sampling, more measures of usage and quality), you can still give Emontx3-continuous a try.

1 Like

Apologies for leaving your work out of the kudos I was flinging around in my recent defence of continuous sampling post. It wasn’t deliberate, I’d just forgotten about it. I’ve updated my post to provide pointers to both solutions.

2 Likes

I just installed your firmware on my emontx and have a couple of questions:
image
In your readme on github you say _enac and _enre are in kWh. Are you sure it’s not Wh? :slight_smile:

I’m getting negative readings on one of my current sensors, but usually just during low load, do you know why that might be? _enre also negative sometimes…

I have yet to calibrate, need to grab my fluke from work, but I’m liking this already, thanks for your work on this.

Cheers!

Cool, thanks for your critique!

You are right, it’s Wh. I will fix that for the future.

Good question. There are obvious reasons that the “reactive” components can be negative, since it is designed to be. But you are seeing the “active” components be negative.

The level you are seeing is about -91 Watts, which corresponds to a current variation of about 1.25 ADU (rms), exactly out of phase with the mains voltage. This amount is too high to be explained as DC bias error or quantization error. I take it that you are not seeing the same effect with the “standard” firmware?

At very low loads the standard CTs provided with the kit are known to be less accurate, but that would not explain a “negative” current measurement.

Good luck!