The algorithm we use came I think originally from the Atmel Application Note AVR465. Have you got the results of your ‘paper’ comparison - or better still comparative measurements using an emonTx and the ‘shop’ current and voltage transformers?
I am familiar with Ankur Tomar’s 2011 paper (AVR465).
He used transformerless voltage coupling and linear interpolation for phase correction.
I am using the YHDC SCT-013-000 CT and tested with the Mascot 9VAC type 9580 VT.
It seems that the Mascot VT exhibits quite a bit of distortion at 240Vrms. For this reason one cannot get the PF close to 1 by applying a phase correction and the difference between the two methods is less profound:
Using Sinusoidal interpolation: PF= 0.9979 (3.67°)
Using Linear interpolation: PF= 0.9971 (4.31°)
Using a bulkier 9VAC VT (Amigo) which distorts less, I can trim the PF down to 0.9998 (0.87°) using sinusoidal interpolation. Since this VT’s phase lead is more than the CT’s, the linear interpolation cannot be used.
Using a simulation for ideal transformers, the difference between the two methods is:
Using Sinusoidal interpolation: PF= 1.0 (0.0°)
Using Linear interpolation: PF= 0.999230641 (2.24°)
This simulation was done for φ = 2.25°, Δ = 4.5° and f = 50Hz
Thank you for those numbers. As you may have read elsewhere, I haven’t been ‘on form’ lately, but I’ll carefully check out all that you’ve written and if appropriate, I’ll make or propose the necessary action.
I think one major problem, not appreciated initially, is that the a.c. adapter was the bigger of the two errors, and that the reading order in emonLib failed to take advantage of the time between readings as a useful aid towards correcting the phase difference between the two transformers. And that of course led to a PHASECAL value of 1.7 being the appropriate default value, which was far from optimal in terms of signal fidelity. Making any change there would have led to many misunderstandings as the changed library would not have been backwards compatible with the user’s pre-existing calibration. The version of emonLibCM presently being tested is radically different and I don’t see backwards compatibility as a concern.
Interesting to note that you found a ‘better’ v.t. I have an ex-equipment one that I believe was originally designed exactly for use as a phase reference, and that shows a phase lag of 0.28° at 185 V falling to 0.04 ° at 253 V. The difference between input and output, as recorded by my sound card at 44100 samples/s and plotted on a spreadsheet, is just discernible on a graph. Unfortunately, this is of little practical value to OEM as I don’t know (and have no means of tracing) the manufacturer, who may well be no longer in business anyway.
I think I recently wrote in another thread that it’s been pointed out that a small, low current 1:1 c.t fed via a multiplier resistor gives much better performance than our standard (Ideal) v.t. This is attractive for the home constructor unburdened by regulatory requirements, but to assemble and offer that for sale requires extensive testing and certification. It would also require the use of a separate 5 V d.c. unit to power the emonTx (though that’s already required for the emonPi).
I believe this is exactly the same as saying that the filter in emonlib attenuates the signal. Multiplying the output by the correct scale factor (Gain) results in exactly the coefficients ursi is referring to. Given that Δ and φ are fixed the “new filter” is still linear, with X = PHASECAL*Gain and Y = (1-PHASECAL)*Gain and Gain is buried in the calculation of filteredV (by calibration)
Actually what happens is that the linear interpolation “distorts” the sinusoidal voltage signal which cannot be corrected with a simple gain adjustment (like approximating a circle with a hexagon).
What I have found is that if one assumes that the signal between two samples is sinusoidal iso a straight line when doing an interpolation to shift the phase, the signal is less "distorted” and the PF calculation becomes much more accurate.
If you want to, I can bore you with the mathematics behind my interpolation formula.
It is remarkable that both of us pointed out the same thing in the span of 3 days.
My point is mathematically you are doing linear interpolation (X, Y are constants) and it is equivalent to adding gain to the filter, or adjusting the calibrated gain of the voltage.
Gain = X + Y
PHASECAL = X/(X+Y) = X/Gain
phaseShiftedV = (lastFilteredV + PHASECAL * (filteredV - lastFilteredV)) * Gain
= lastFilteredV*Gain + PHASECAL*Gain * (filteredV - lastFilteredV)
= X*lastFilteredV + Y*lastFilteredV + X*filteredV - X*lastFilteredV
= X * filteredV + Y * lastFilteredV
I believe you are confusing filter with the relationship between φ (the phase difference) and PHASECAL which as you point out is not linear, but as it is calibrated, rather than derived, it doesn’t matter.
Of greater concern is
the zero point filter was not nearly slow enough (low pass enough) and it introduces a distortion around the middle zero crossing
the general insensitivity of PF on resistive loads to PHASECAL (i.e. near unity PF and small phase deltas)
I and another am already looking at this, along with something possibly much more important but inspired directly by Andries’s formula (and something about it which, much to my chagrin, I’d failed to notice) and if it gains support, I think there’s a good chance it will get taken up. The low-pass filter problem had already been put into the upcoming version of emonLibCM.
And it is a lot worse when a factor of 1.7 becomes necessary and it turns into extrapolation, which is particularly damaging with the ‘flat-topped’ 50 Hz voltage wave common in the UK (at least). If either of you want a 44.1 k samples/s recording to experiment with, ask. (It will be a csv file to import into a spreadsheet.)
It cannot. But for most people, who do not run large loads with very poor power factors, it is ‘good enough’. And there is a large body of users who don’t have a programmer and cannot calibrate the emonTx at all, and rely on scaling in emonHub to achieve some sort of correspondence with the readings of the energy supplier’s meter.
And there’s probably an even larger body comprising those who buy commercial energy monitoring products that rely on a nominal voltage and an assumed power factor.
I have considered getting a large, mains-rated capacitor so that I can get a reasonable current that’s close to being purely reactive, but then such a beast is not available to (almost) every other user, so what point? Surely I’m right to think that the shift can’t be simulated in software without measuring the mains frequency, as a possible (but unlikely) 2% frequency swing rather complicates the whole idea.
No more than you already do. I realize there can be a large drift but I have to imagine the frequency does change with any frequency (sorry couldn’t resist).
Regardless, I believe, and again I have no large phase error components to test, assuming your stuck on filtering V (!), the idea would be (the paper seems to get unnecessarily complicated at this point - you may find there is a reason! - but I hope not):
Delay the voltage (Vd) and the current (Id) by a fixed (arbitrary) number of samples, calculate the “PF” of VVd and then tune PHASECAL (operating on V - Vf) and filter Gain (so RMS V = RMS Vf) until the PF of VfId matches at which point the phase delay of V-Vd is the same as Vf-Id.
Remove the sample and the phase difference of Vf-I should be zero.
I suspect a simple binary search with some averaging would hunt for a solution automatically adequately, or you could try a slightly fancier fancier trapezoidal search.
The paper suggest using a delay so that the simulated PF is not too small. I think it is because they are calibrating at the same time with a reference meter. Without trying it, I can not see any reason why you shouldn’t aim for a 90° shift and zero real power in which case you don’t have to worry about the filter gain at all, but as the boat launches next weekend, unless it is rainy and cold I may not get time to experiment for a while.
Yes the Ti paper is confusing, I first did not understand what he was doing, then didn’t think it should work (I sort of thought you would lose as much as you could gain). In the end my procedure (different from the paper) should work, but makes me wonder why the author didn’t simplify it (It probably wouldn’t have been “academic” if it didn’t have lots of formulas!) and it certainly works in excel - but of course there it is many digits of precision perfect sines.
I didn’t really think (frequent problem), but I might be able to something with your recording. I assume it is of the sensor output?
I have a computer/monitor on the boat - though usually I use it to play games (and watch movies) and usually much beer has been consumed after working (cleaning, fixing) on the boat. All of which is a pre-excuse in case I don’t/can’t do anything useful!
As an aside are you saying that power in the UK is not sinusoidal? (flat-topped??)
The mains input was recorded using a resistive divider, at 240 V (the adapter’s nominal input voltage), using the complete test circuit in the report. The presence of the Variac makes no discernible difference.
Column 1 is the sample time, column 2 is the mains input, column 3 is the adapter output (unloaded).
It is the No.2 adapter in the report.
The standard emonLib records approximately 53 sample pairs (voltage & current) per 50 Hz cycle, not phase locked. The CM library will record one sample every 104 μs, a sample set is a voltage sample followed by 1 - 4 current samples; therefore between 96 and 38 sample sets per 50 Hz cycle, also not phase locked.
It is definitely not a problem with the recording, my hardware 'scope shows exactly the same shape. ‘Flat-topping’ is a well-known phenomenon (but obviously not common where you live), it’s most likely due to the reservoir capacitor in a myriad of plug-in power supplies all over the country taking current only near the peak voltage.
I assume you want the transformer outputs? What sort of current would you like? You can’t have the latest black-lead moulded-plug c.t. (because Glyn had it back), but you can have either a.c. adapter and one of the older black-lead c.t’s from the reports.
I’ve only just got round to looking at the TI paper again. As I see it, it’s suggesting adding a synthetic phase shift so as to (in effect) move off the flat top of the cosine curve onto the slope, where a small change is easily detected and measured. But it still requires knowledge of the actual power factor of the load, and of the synthesised power factor. The first is easy enough - an electric kettle. The latter doesn’t seem quite so simple. It implies a special calibration sketch with an accurately-known built-in delay of around 60°, so that the user (or a automatic procedure) adjusts for a known ‘synthesised’ power factor.
In other words, it’s doing exactly as I’ve always done to check the three-phase sketch on a single-phase supply, which is adjust for a p.f. of -0.5. on phases 2 & 3.
I haven’t missed your procedure, I briefly skimmed over it while was feeling particularly poorly and gave up. It’s going to get looked at in a few days, when I was hoping to write a sketch to test it.