where is rob in this discussion, a chance to contribute
Very interesting discussion,
If i understand, its quite difficult to get a perfect measurement of power factor, and here in AUS (Victoria) we have “smart meters”, where the previous meters were basically mechanical devices (a sort of spinning wheel), replaced with 'digital" devices driven by software - Its a “commercial grade” meter - Landis & Gyr E350 - Type 1200.
Given all the issues you describe, how do these commercial dudes
1 - Run the software to do all you describe
2 - Do the calibration - and if its against some ‘reference source’, how does that map to reality
Or is the qualification just against the reference source anyway
Coz in reality (or at least my reality
all I want is the IotaWatt to be as close as possible to that being measured by my electricity provider - until I go pure “off grid” of course
“perfect” would be difficult ;-), but anything else is not that hard. The three basic ingredients you need are Vrms, Irms and RealPower, then PF is simply RealPower / (Vrms * Irms). That approach works regardless of the shape of the two signals (V and I).
It’s only if you start talking about angles that things get tricky. The textbook power triangle, and cos φ definition of PF only works when the two signals are pure sinewaves (no harmonics) and the only time that’s likely to happen is in the lab.
The commercial dudes calibrate their meters against a power calibrator. Something like this: 5077 Power Calibrator - Time Electronics. Before they do that they’ll have to ensure the calibrator’s calibration certificates are current. If not they send the calibrator off to the local calibration lab, who’ll calibrate it against their equipment (even more accurate). That equipment will also need to have current calibration certificates. The whole chain is traceable back to god, via calibration certificates, or at least to your national standards lab who are in charge of deciding how big a volt is, and from there no doubt to some international standards body so that our volt doesn’t drift too far from a UK volt etc.
They also have the advantage of not using VTs. Some use CTs and some use shunts. So their starting point is from a relatively good place.
The generally accepted test around this forum is that so long as your energy monitor stays in broad agreement with your revenue meter over a suitably long period, that’s close enough. I’ve never been a huge fan of that test (unless maybe the period is extremely short). Passing that test simply means your errors are averaging out to zero, which is better than them not averaging out to zero but doesn’t really give you much confidence in answering questions like: how much is my sub-woofer using in standby… is it worth unplugging it? To me, that’s where energy monitors (especially per-breaker monitors) are far more useful than just checking your quarterly bill from Origin looks about right.
It’s a bit like Treasurers who promise to balance the budget over the economic cycle. It’s all dandy, but sometimes you want to know what the deficit is right now.
Thanks to all who have helped make my point. If you’ve read this whole thread, and I commend you on your patience if you have, I’d ask you to go back and read my opening post.
My two main points:
Traditional methods of measuring phase-shift are not as useful in developing methods to correct for the effects of VTs and CTs.
This isn’t rocket science, but there are those who want you believe that it is.
Actually, I use a very similar technique to what you’re using. I use arctan (Reactive/Real) while you use arccos (Real/Apparent) but that’s neither here nor there. I have easy access to Reactive, you have easy access to Apparent.
I don’t think it’s rocket science. I’m just trying to help you improve on:
I’ve pointed out several possible causes for that error in my postings above. If I had to pick which I thought would be the most significant contributor I’d say it’s calculating Vrms (and Irms) based on a single cycles worth of data +/- some “tail” of the cycle. At least I think that’s what you said you do. Do you at least start near a zero-crossing so that the “tail” is relatively low amplitude?
The variance changes with the quality of the wave. The more distorted, the more the variance. I think that’s because these are not pure sine waves. I’ve had it work to within .1 degree. When I did it while making that post, it was off more with that crappy VT.
The IoTaWatt is within 0% to .5% variation from several revenue grade meters in my home, and within .5% of the power reported by several solar edge inverters. I don’t think there’s a problem.
At 640 samples/cycle, one sample represents .5625 degrees. That would average out to about .28 degrees off over time.
I start collecting samples with the first one after zero crossing and I end with the sample after two more crossings. The “tail” is more like a stub. Why do you guys keep trying to fix something that aint broken?
Yes, that’s what I tried to explain above.
Great! But not entirely consistent with:
I don’t think anything is broken, we’re all striving for constant improvement and sharing techniques we’ve found that help in that regard. Well, ok, when you wrote:
I admit I thought it was somewhat broken, but your latest reply has put my mind at ease.
The power coming out of your Solaredge inverter is some of the easiest power around to measure. You could have significant phase errors in your monitor ( could - I’m not saying you do) and it would still match nicely. Now the power going into your Solaredge inverter as it puts itself to sleep at night is somewhat more interesting; mine goes very reactive for a short while on the way down. Some brands of micro-inverters go very reactive and stay that way all night. That’s when uncorrected phase errors in your monitor start to matter.
Here’s how my SolarEdge inverter looks when it’s putting itself to sleep. If you see a similar spike in current with no corresponding blip in Power then I’d say your phase correction is working well.
That small dip at about 18:27 from -4.4W to -2.5w happens every night. It seems about 20 minutes after putting itself to sleep (as indicated by the vertical cliff on Reactive Power) it goes into an even deeper sleep state, and saves another 1.9W,
The cycle count is whatever I end up with after two crossings, although I occasionally have to discard a set of samples when the count is low - indicating that the ESP8266 serviced an interrupt while I was collecting samples.
So let me start at the top again:
The shifting in the VTs and CTs is not simply a constant time shift of the entire cycle. It is a convoluted thing that appears to be greater when the slope is steep, and less when the slope is flatter. This can cause introduce error into the measurements. I think we can all agree on that.
The mechanism available for mitigating (correcting not a good term here) the error is to shift one set of data points relative to the other. When we do that, we shift all of the data points, so the net effect will be a change in the power factor by the angle represented by the shift. For instance, if there were 720 data points collected evenly throughout the cycle, shifting one set of points by 7 would represent an angle of 3.5deg.
So we can do this to mitigate the error, but the question is “what is the angle”?
Back to my initial post. By using the power factor calculations (or a phasor diagram - same thing), on the output of the transformer vs the primary (unity load for CT), you get an angle that, if used to shift an entire cycle evenly, will have the effect of mitigating the error.
This number is not the same as the phase shift that you will get from the traditional phase shift measurement methods. It is always smaller.
The phase correction number in IoTaWatt is not the phase difference in the traditional EE sense. It is the angle that the data must be shifted by, to mitigate the error introduced by the waveform distortion of the transformer.
To carry this a step further. There are two transformers involved. The VT and CT. Both exhibit leading shift. The mitigation angles for both components are provided in the configuration tables, and the net shift is computed for each CT/VT pair. That net shift is applied to the data after each sampling when the data is crunched for real power.
It’s not a question of whether phase correction is used, it’s where you get the data to determine how to do it.
You’re right, the Solar Edge inverter has a high power factor. The closer to unity, the less the effect of the phase distortion. Nevertheless, it is important to mitigate the distortion correctly.
My heat pump runs at pf .89 - 27 degrees. I have a revenue grade meter on it (EKM) and also monitor it with IoTaWatt. Over the last5 weeks the EKM meter says I used 514.3 kWh. IoTaWatt says I used 514.0 kWh. This is without any calibration or adjustment. The correction for the VT and CT are straight out of the tables.
If the shift needed to mitigate the VT error were off by 2 degrees, the difference between the traditional phase measurement and the measurements I’m advocating, the pf would be 25 degrees or .91. That would be a 2% difference, yet the two agree.
This isn’t about the importance of mitigating the phase distortion, that, that’s a given, it’s about how you mitigate it, and my point is that you have to do power-factor type calculations to determine how much simple phase shifting is needed. Time will tell with IoTaWatt. There are quite a few out there now. I’m confident the accuracy will surpass the legacy equipment without the need to reprogram and calibrate.
If you have any more questions about how it works, the firmware is on GitHub.
As far a I know, that’s how everyone does it.
Those few words there demonstrate your misunderstanding of this. You can’t swap between a PF of 0.89 and a measurement of degrees (except in a spreadsheet or a textbook where pure sinewaves rule). Unless your heat pump is extremely old, I bet it has an inverter in it. It’s current signature is going to look a lot more like that of switch mode power supply, probably like the DC brushless motor in my washing machine. Here’s how it looks on it’s spin cycle:
My monitor clocked that signal at:
Apparent Power: 559VA, Real Power: 395W, Reactive Power: +115 VAR, Distortion Power: 378 VAR.
arccos(395/559) is 45 degrees, but that’s a completely meaningless quantity in this situation.
Did you read and understand: Measuring reactive power to build NILM system - #7 by dBC ?
Right back at you DB.
Sure I can.
Here’s my heat pump running at a pretty steady rate with a CT that has a phase correction of 0.45 and a VT that has a phase correction of 1.9. (so net 1.45 degrees used to mitigate the effects of phase distortion)
The pf is .7997 which I think we both agree is properly computed by dividing real power by the product of Irms and Vrms. The real power is 746 watts, so you can compute the apparent power as 933 VA.
Now I add 2 degrees to the phase correction factor of the VT making it 3.9 instead of 1.9. The net correction is now be 3.9-.45 = 3.45 degrees. Here’s what happens:
The measured PF went up by 2.3 degrees (cos .8234 vs cos .7997)
The Real Power went up 3% to 769 watts, yet the Apparent Power stayed the same at 933 VA.
You sure can express power factor as an angle and if you add or subtract to the phase you can expect it to be reflected in the power factor.
This is the same heat pump that IoTaWatt has tracked perfectly for 5 weeks. The extra 2 degrees of shift that would be reflected in using the oscilloscope method would have it off by 3%. So the correction method and particularly the methods I use to develop the correction factors for phase distortion seem to work well hand-in-hand.
If you don’t understand how this works, don’t be changing the subject to your washing machine. I really don’t appreciate your condescending explanation about how that works. At nearly five times the sample rate, my instrument takes that in stride and samples it better in 20ms vs the 20 seconds that you were talking about to get viable samples.
Not only do we agree that’s how you compute it, that’s the definition of PF.
Actually the PF went up by 0.0237. PF is always a number in the range 0 to 1, it’s never measured in degrees. What went up by 2.3 degrees is the arccos of the PF. Picky perhaps, but you can’t just go changing the definition of PF. We’ve already agreed on its definition above. There is simply no way that definition can produce something measured in degrees.
But since you calculated the arcos’s of the before and after PFs, are you not even the slightest bit curious as to the discrepancy between 2 degrees and 2.3 degrees? It’s explained in the 3D vector diagram you’ll find at the reference above. Think carefully about which of those vectors you’re changing when you deliberately introduced your 2 degree phase error. Think carefully about which angle you’re calculating when you do an arccos (Real / Apparent).
Do you know of anyone who uses the “oscilloscope method” to calibrate their energy monitor? The reports on VT and CT phase errors use oscilloscopes to demonstrate the fundamental issues. The techniques to calibrate your meter are described in the Learn section and as far as I can tell, use the exact same PF techniques you use. I explained above why the numbers you see on the oscilloscope can’t be plugged straight into your calibration (and I’m pretty sure it’s also explained in the Learn section).
I think you completely misunderstood my point there. I sample all 18 channels continuously at 8kHz. There are 8000 x 18 x 24-bit (V, I) pairs arriving every second forever. That’s a lot of data. I don’t attempt to get anywhere near it (apart from when I do wave captures). The IC accumulates energy (Real and Reactive) and I can read the energy registers at my leisure. From memory, I think I’ve set it up so the LSB of the energy registers represents 1/10th of a WattSecond.
Also from memory, I calibrate at 230V, 2A 60 degree phase shift. To calibrate phase shift I do an arctan(Reactive/Real) and tune the IC’s filter until it reads 60 degrees. (Sidebar: the only reason I can do that is because I’m feeding it two perfectly formed sinewaves). At calibration time, when I do the Reactive/Real division, I want those numbers to be relatively big to avoid quantisation errors. For that reason, I let the energy values accumulate for a full 20 seconds. If I let it run for a minute, the division would be even more accurate. I’m talking solely about calibration measurements here, not normal ops.
Very little of that is relevant to your setup except for the basic principle that when calculating calibration coefficients that you plan to lock in forever, averaging is your friend. Keep in mind that at the time I was banging on about that, I only had the following descriptions of your methods:
That sounded pretty reckless to me, but you later revised it to:
People may be wondering why I’m having this argument with you. My policy of late is to ignore people like you when it doesn’t infringe on my story. But as you know, this all started a week ago when an IoTaWatt user inquired about voltage swings and whether this would adversely impact accuracy in his device.
I responded that the voltage swings he described were in the normal range and that any change in phase caused by those voltage swing should have a second or third order effect on his accuracy at worst.
At this point, you chimed in to point to a report on the VT and to say that it makes a difference when measuring individual circuits. I don’t believe that’s true and it’s not the message I want to leave unanswered on this forum.
I explained that although the report didn’t specify the method used to arrive at those shift numbers, it was implied that it was done measuring the shift at zero crossing and that’s not what the IoTaWatt uses for phase correction. That prompted a quick and biting rebuttal from the author of the report, complete with the usual ad-hominem attacks.
Fast forward a week, and that author publishes a description of the method used, which is indeed the classic oscilloscope method of measuring phase difference. I didn’t say that was wrong, I said that’s an inappropriate way to determine the phase correction factor for IoTaWatt.
So that brings us to this thread where I take a fresh start to explain how it all works and why the phase correction in IoTaWatt is different from the classic measurement of phase shift using an oscilloscope.
I don’t know if you have figured this out yet, but most people don’t read the learn section. It’s a nice reference feature for people who are interested in AC power theory, but most people are just interested in how many kWh they are using for what. My idea is that a lot more people would be interested in managing their energy use, and OEM products, if they were not being directed to find something in the learn section in order to monitor how many kWh they are using. It’s a busy world. Most people want the quick-start guide.
That’s the philosophy of IoTaWatt. You can connect it and be monitoring your usage very accurately in 15 minutes, without knowing anything about power factors or phase shifting. You can use any of the widely available CTs and VTs that have been tested and included in the configuration tables without a running dialogue with someone that involves FTDI adapters, downloading sketches, a commentary on Arduino IDE vs platformIO, and the way CTs and burden resistors work. Those things are a barrier to wider acceptance of the notion of being aware of and monitoring personal energy use.
While I freely admit that I can’t keep up with you and a few others in the EE specialty, I think I have demonstrated an acquired proficiency in hardware design and production, firmware design and publication, user-interface development, and data communications. The end result is a next generation monitor that was not possible a few years ago because of advances in ADCs and the availability of low-cost, and high speed processors with integrated WiFi. It was only a matter of time before someone did this. Things change, I call it progress.
You can pick it apart and talk all day about harmonics and distortion power, but at the end of the day, most folks just hear blah-blah-blah. The bottom line is that it works - well. Between your washing machine and some-kind of 18 channel specialty device I’ve lost track of why you keep beating me with the ignorant stick.
If you’re not part of the solution, you’re part of the problem.
I have re-opened this thread, I’m not going to get involved with specifics. However, open discussion is encouraged.
Please try and keep discussion on-topic, productive and impersonal.
great to see a moderator step in - thanks
Given that Emon Pi uses CTs and reference voltages, have we compared the 2
Opps, sorry I started the flame war ;-),
Me too. I’m not here to canvass the relative merits of different OEM monitoring products, I’m here to discuss phase error calibration. Personally, having gone the energy IC route, I find all these products that do the maths in an general purpose CPU rather quaint. But whichever route we’ve all taken, they all ultimately do it pretty much the same way, and they all share a lot of the same calibration issues. By sharing our approaches to solving those issues we all get to benefit.
I don’t doubt it, but that shouldn’t exclude discussions that may help it work even better. Did we find any easily achievable improvements that don’t involve investment in $10K worth of calibration kit? Possibly not but that doesn’t mean the exploration was pointless, especially since your initial description strongly suggested there was some low hanging fruit. “I take 640 samples that aren’t linecycle aligned” was a reasonable interpretation of your initial description and early response for further clarification.
I invite you to re-read my initial response in this thread. It was genuinely about trying to be part of the solution, and I was surprised by the reaction it got. My interest was particularly piqued by your claim that 0.5 degrees was as good as it got. That seemed pessimistic to me, and I thought it worth exploring whether it could be improved.
My later responses were then all about correcting mis-information. And with regards the washing machine which seem to have particularly tweaked a nerve for some reason…
My heat pump’s PF is 0.89 is a perfectly sensible statement to make.
My heat pump’s PF is 27 degrees is completely meaningless.
The scope trace I posted above is my best guestimate of what your heatpump’s current signature looks like. It demonstrates that the 27 degrees means nothing. If you try really hard, you can identify which angle in that 3D vector graph is represented by the 27 degrees but even then you’ll come to the conclusion “so what… that angle means nothing”. Unfortunately, I don’t have a heatpump, so I had to make do. I took a punt that the electronics and motor in my washing machine is a lot like those in your heatpump. If you think your heatpump’s current signature is vastly different from what I posted then you merely need to put your scope across the relevant burden resistor and post the picture here. I’m happy to then discuss why its PF is 0.89 and whether there’s any significance to the 27 degrees.
Should you discover that the heatpump’s current signature is a pure sinewave and that the voltage going into it is also (or a very close approximation) then the 27 degrees has some meaning. It should be the displacement between the V and I sinewaves.
So anyhow, getting this thread back on topic…
I think the two techniques under discussion are very different. The description of yours suggests you’re using Lissajous (which should only be used on sinewaves). Robert’s approach does an FFT on the signal to determine the phase of the fundamental component.
How can you know that without quantifying how phase shift changes with voltage?
Once you move to per-breaker monitoring it’s quite common for the signal on some circuits to be entirely dominated by non-linear modes such as those you see with switch mode power supplies. As an example, here’s how my lighting circuit looks every night. You’ll see it’s pretty much all over by the time the voltage reaches its peak.