DIY Current Monitor on Raspberry Pi. My power calculation isn't accurate. Help please?

david0 · 22 March 2020 01:17

I would like to keep it code compliant, so thank you for sharing. Plus, the noise I am picking up from CT leads that are disconnected from a conductor is in fact 60Hz. Do think this is a clear indicator of EMI?

Robert - is the inductor L1 in my schematic intended to filter out 60Hz noise on the 3.3V input? I am asking so I can determine the inductance and capacitance values needed. If it is in fact 60Hz, I’ve determined that a 70H inductor and 100nF capacitor should block 60Hz. I want to submit my order on LCSC for the missing components I’ll use on the custom PCB.

Robert.Wall · 22 March 2020 02:35

On a p.c.b? You’re kidding. That component would be HUGE.
No, they’re intended to take out r.f.i. If you’ve got mains ripple, on the 3.3 V, it’s more likely to be 120 Hz (from a rectifier), so if it’s genuine 60 Hz, it’s straightforward pick-up from adjacent cables or radiated from a transformer (not one in a switched-mode p.s.u, they don’t work at 60 Hz).

The leads are disconnected, yet you’re picking up noise from them? That doesn’t make sense, unless you mean they’re disconnected at the far end. If they’re disconnected at your p.c.b, and it’s really 60 Hz, it’s either on the ADC reference, the op.amp output, the 3.3 V or a combination of those, or maybe your grounds aren’t all the same voltage - meaning you have a large 60 Hz current flowing that’s giving you a voltage (V = IR) between two points that you think are both GND at the same voltage.

david0 · 22 March 2020 04:00

Haha! I had no idea.

I just pulled some sample data using the board voltage on an ADC input. Out of 4000 samples, all but one sample came back as 1023. One sample came back as 1021. It doesn’t look like there is any noise on the 3.3V input at all. Would you still keep the inductor in the schematic and just wire a bridge where the inductor would go, or remove it from the schematic?

Yes, disconnected at the far end is what I meant. The CT sensors themselves are not clamped onto a conductor, but the leads are plugged into the board. When I had my cat5e cable/CT sensor bundle running next to the high voltage line in the same conduit, the noise pattern coming in on the CT inputs was indeed 60Hz. I am going to put in a new conduit for my CT sensor cable to keep it away from the high voltage.

I’m hoping the custom PCB will bring some improvements in the noise handling area. I know I still need to look into the phase correction calculations… and I have your PDF saved… it’s just a matter of figuring it out and doing it!

david0 · 22 March 2020 04:19

I’m beginning to review my PCB layout and I have a question.

Since I’m using a ground plane, can I/should I get rid of the traces that were previously the ground connections? For example, the red trace underneath the ADC socket in the image below is the ground. The solid blue region is the ground plane, which if you look closely, is connected to the same pins that have the red traces going it. Can I remove the red traces and rely on the ground plane for grounds? If yes, I would assume that the same logic applies throughout the board. Is there anywhere I should leave both the connection to the ground plane and the ground trace?

I know the ground plane is hard to see in the picture above, so here it is with a direct view:

Robert.Wall · 22 March 2020 13:51

I’d leave a wire link there, as it’s a 1-off. That way, the option is still open should you find it necessary.

Yes, you can do that. A ground plane will have much lower resistance than a trace. Look at it this way: if you’ve got a current that gives you 1 mV of noise between the “ground” for the op.amp and the “ground” for the ADC, you’ve got ½ - 1 mV of noise on the input. If the resistance of the ground plane between those points is ⅛th of that of the trace, you’ve got more like 100 µV of noise on the input.

david0 · 22 March 2020 21:31

After I removed the unnecessary ground traces, I have been focusing on making the ground plane as large as possible and trying to avoid having both “islands” and choke-points in the ground plane. This has lead me to another question for you - is using a lot of vias considered a bad thing?

For example, I added these two vias (on the ends of the diagonal red trace) because it was previously a long trace that was kind of isolating the ground plane.

I’ve employed the same design technique in several other areas of the board too - where the only purpose of the via is to add some continuity to the ground plane.

Robert.Wall · 22 March 2020 21:42

If the ground plane is in separate pieces that aren’t connected, it’s not a ground plane - or not an effective one. An ungrounded patch of copper on one side of the board will be one plate of a capacitor, with the board substrate as the dielectric, with traces on the other side as the other plate, capacitively linking then together. From what I can see, unless there’s another route to connect the top left copper to the bottom right copper, then what you have is necessary.

david0 · 22 March 2020 22:02

I did have one “island” piece of the ground plane which I fixed by rearranging some components and adding vias.

Here’s the full board - the link I shared a couple posts ago should take you to the live version if you are available to take a look.

Robert.Wall · 22 March 2020 23:03

There are a few immediately obvious places where I think you can remove links and vias by rerouting traces, it’ll take a bit of studying to identify them all.
Here are a few to give you a flavour of what I’m noticing.

You don’t need the top link between the screening can legs of RJ1, check the dimensions though of those legs.
If the screening can of RJ1 doesn’t touch the board, the mid-rail common to R0 - R4 could go round under RJ1 and meet up with pins 6 & 7 there.
The trace from U1 to R6 could go under R7 and round the top of R6, removing another 2 vias and a link.
The current in R5 is a 1-turn coil around the c.t. input traces from RJ1 - rotate R5 90°

Also, check with your board manufacturer about 90° corners - they are usually a bad idea.

david0 · 23 March 2020 02:18

Thanks Robert, that is extremely helpful.

I’m not sure why the link between the RJ1 metal screen was there - I’ve removed it. I think it came about when I was trying to attach the RJ1 screen to the ground. This is the female jack that will be placed in RJ1’s position. I am planning on grounding the two pins coming off the shell of the jack to the ground plane - which, if you recall, I was questioning their purpose in the rough draft of my schematic. I checked the hole size and they were a bit small - by about 0.5mm. Good eye!

Why would it matter if the screening can touches the board or not? From the picture of the jack at the link above, it looks like the bottom side of the jack only has the metal screen around the perimeter. But aren’t traces in the top layer still insulated under the PCB coating?

I’ve moved the trace from U1 to R6 and will continue to look for other traces that can be improved in the same manner.

Rotating R5 was a good improvement. I was too focused on the aesthetics that I did not see that easy optimization.

The fact that you can say they’re usually a bad idea means you know way more about this than I do! I’ve eliminated all 90° trace corners, but out of curiosity I will still check to see if I can find anything about it with the manufacturer. 90 degree corners that meet at a via should be fine though?

Here is the board with the latest changes:

Robert.Wall · 23 March 2020 13:07

I wouldn’t trust it.

yes, or at a component pin. I was taught (about 50 years ago) to avoid sharp corners, I can’t even remember the exact reason now. As it’s so long ago and production methods have changed, that’s why I wrote “check with the manufacturer”. It might not be the problem it was then.

david0 · 23 March 2020 20:29

Thanks!

It turns out I need one more CT sensor in my project, for a total of 5. I did not realize that my subpanel was also fed with two 120V legs like my main panel is. To summarize, my CT needs are 2x for the main panel, 2x for the subpanel, and 1x for the solar input. My solar input is 240V also but I am simply doubling the single CT measurement based on the assumption that the PV inverter distributes the power equally between both legs.

Since I have two unused ADC inputs, I figured I might as well include two additional CT inputs. I am thinking of using a 3.5mm plug for the additional 2 inputs, but I can’t seem to find a simple 3-pin female jack that is horizontally mounted. Most of them have 5+ pins, like this one. It took a bit of research to uncover what each of the pins does, but I believe I have it figured out. I’ve added two of these to the schematic - they are below the ADC and to the left of the op amp.

Pin 1 is the sleeve and pin 2 is the tip. Pin 3 is the ring, which is not needed for the CT sensor input. Pins 10 & 11 are mechanical switches that are normally closed if the socket is empty. When the jack is inserted, the switches open.

Would you recommend that I wire the unused pins (3, 10, 11) to the analog ground, or leave them open/un-grounded? Here is my schematic with the two additional CT circuits - I’ve grounded the unused pins to the analog ground to start out.

Robert.Wall · 23 March 2020 22:17

That’s most probably reasonable. Even if there is a neutral connection, it’s likely it’s only for the control circuitry, and a few watts.

They are called “break” contacts to tip and ring, which are closed when there’s no plug inserted and the contacts break when a plug is inserted. Think a loudspeaker that’s cut off when headphones are plugged in. Leave them all disconnected. If you ground it, you’ll short the mid-rail to ground, disable all the other inputs and damage the bias op.amp when there’s no plug inserted. You won’t get much pick-up on that input with the burden resistor in place.

david0 · 24 March 2020 01:17

Thank you! I had to completely redesign the PCB. I realized I had pointed the ethernet jack and the DC barrel jack right at the Raspberry Pi’s onboard USB ports, which would have rendered my ports inaccessible. Here is the new design with the two additional 3.5mm jacks for the 5th and 6th CT inputs.

The pins that are highlighted white are all the ground plane connections.

I have a bit more optimization to do (and some 90° bends to remove), but this is pretty much what I’m ready to submit for manufacture. I am super excited! Do you see any obvious pitfalls?

Picture updated 3/24, 10:24am PST

david0 · 26 March 2020 00:05

Well, I submitted the board for production! Hopefully there won’t any issues, but I won’t be disappointed if there are since it’s my first PCB. They are cheap enough that I can update the design and re-order them.

Bill.Thomson · 26 March 2020 01:20

david0 · 29 March 2020 01:50

So, a couple of things - I realized I did not test the op amp bias voltage supply in my prototype board before submitting the order for my PCB. Fingers crossed that this doesn’t give me an issue! I even purchased a SMT pinout board so I could plug the tiny op amp into my prototype board. I just didn’t get around to it… oh well.

Next, I’ve been reviewing the phase calibration procedure and I want to make sure I’m on the right track.

Since I added two additional CT sensor inputs to my custom PCB (for a total of 6), I increased my ADC clock speed (by more than 2x). I was already getting some squaring off in the waveform at the previous clock speed with only 4 CT readings + 1 voltage reading. I understand the increase in read speed will reduce the phase error between samples, but I am still interested in correcting for it. I want to be as accurate as possible. I’ve calculated the approximate phase error per channel, using the voltage sample as the point of reference for comparison. Here is what I did… does this sound correct?

My board takes 2k samples across 6 channels in about 0.383s. This means the time between samples is 32µs. I read 1 sample from each channel sequentially, meaning, read channel 1, then 2, … then 6, repeat.
1° in a 60Hz waveform takes 46µs.
From one channel sample to the next, the AC wave moves 0.695°.
The time between samples for a single channel is 6*32µs, or 192µs.
The phase error will vary per channel depending on how many cycles away it is from the “reference” wave form.

Here is a table where I’ve calculated the phase error for each channel.

Sampling Order	Time Shift Needed	Equivalent Phase Error	PHASECAL
ct0	3 * 32µs = 96µs	96µs / 46µs = 2.08696°	1.5
ct4	2 * 32µs = 64µs	64µs / 46µs = 1.3913°	1.3333
ct1	1 * 32µs = 32µs	32µs / 46µs = 0.69565°	1.1667
AC Voltage	0	0	0
ct2	1 * 32µs = 32µs	32µs / 46µs = 0.69565°	1.1667
ct3	2 * 32µs = 64µs	64µs / 46µs = 1.3913°	1.3333

I calculated phasecal by taking the time shift needed for each channel, and dividing that by the time between a single channel’s samples (196µs), and adding 1. So far, so good, I think.

What I’m not grasping in all of this is the how. I don’t understand how including the initial sample from the previous set of data corrects phase error. My gut feeling is that I’ve misunderstood the phase calibration process. The process would make sense to me if it were applied to a single set of data, consisting of samples taken at a regular interval (which, in my case, is 32µs). However, there is a short break between one sample set and the next because I have to perform all the calculations for a single set and send it to my DB. In other words, every single sample is not 32µs apart. I take 2000 samples, one every 32µs, perform some calculations, and then take another set of 2000 samples. There is not a regular interval between the first sample in set #1 and the first sample in set #2. From my understanding of the phase correction algorithm, the first sample would be captured, then the 2001^st sample is captured, to be used as lastFilteredV and filteredV respectively. I have more questions about the algorithm, but I wanted to start off with this one.

To take a step back, I revisited the posts in this thread about the CT phase error (post 66 has the most detail). You had mentioned:

I believe my table above accounts only for the time between readings. Since I’m using the voltage sensor as my reference wave, I don’t think there needs to be any phase correction for it (only the linear accuracy calibration is needed).

I did some research on current transformers and the source of phase error… what a rabbit hole that was! I got into the CT model and the phasor diagram of a CT. I would be lying if I said I could tell you how the phase error in a CT sensor comes to be, but at least now I understand that CT sensors have one by nature of their existence. I didn’t know that before. My research led me here, which says:

But in an actual current transformer, there is a phase difference between the primary and the secondary current because the primary current has also supplied the component of exciting current.

This is probably one of those things that I can keep asking why ad infinitum. I think I am satisfied knowing that there is a phase error due to the time between measurements, and a phase error relative to the primary current. This now leads back to my initial lack of understanding for the how. How is the phase error (against I_p) corrected when I don’t have a trustworthy measurement of I_p in the first place? Is it simply by means of tuning to match my handheld clamp meter?

Robert.Wall · 29 March 2020 12:47

Not quite.

Strictly, that’s a timing error that can be viewed as part of the phase error.

Correct.

It doesn’t. It’s going to insert a small error in the first of many power values, which will disappear in the averaging process.

A-ha, I see where your thinking has gone wrong. Just clarify, what are you calling a sample set? I thought you meant one voltage and 5 currents, but later it looks as if you mean 2000 sets of a one voltage and 5 currents.

What emonLib is doing (and I phrase that carefully because you might need to to it slightly differently) is making a new voltage wave that is an interpolation between one voltage sample and the next. Go and read Learn→Electricity Monitoring→Current & Voltage→Advanced: Explanation of the phase correction algorithm→Explanation of the phase correction algorithm slowly and carefully, calculate a few values by hand on a sheet of graph paper, then if necessary write a spreadsheet to do the rest.

The v.t. has a phase error, and like the c.t, it’s variable as is the c.t’s. It’s due to exactly the same mechanism - it’s another hole belonging to that wascally wabbit. You do need to take it into account because it’s your reference. However, it’s only the difference between the phase error of the v.t. and c.t. that you’re interested in. The overall correction is the difference in the two phase errors plus the time between the respective samples being read.

You don’t need to know the phase of I_P. What you need to know is the difference between the c.t. and v.t. outputs when V_P for the v.t. and I_P for the c.t. are exactly in phase - and you get that with a pure resistance load (= an electric kettle or a fan-less electric heater).

That’s why the calibration instructions say adjust for zero error (i.e. real power = apparent power) with a purely resistive load.

I haven’t checked your maths in the table - instead here’s a diagram that I did when working on emonLibCM. The strategy is very similar to yours - read the voltage and currents as a set, then interpolate the voltage to ‘time-shift’ it. It’s done in Inkscape (free), the boxes represent the expected range of the phase errors. It’s for the Atmel '328P and 50 Hz, so if you want to use it, you can’t - directly, you need to recreate it using your sample rate, mains frequency etc. The 4th c.t. is different because in the emonTx, it has a different burden.
Phasecal 4CT.zip (6.0 KB)

david0 · 29 March 2020 17:56

Yep, this is where my misunderstanding was stemming from. I am calling a set of data the collection of 2000 samples from each channel (5CTs + 1 voltage)… or 12k samples overall. The terminology of a set in the Learn section means a single reading of one or more CT values paired with a single voltage reading, right?

It all makes sense now. With a purely resistive load, I can assume that the current wave should follow the voltage wave exactly, and the discrepancy between the two when I measure it is the combination of the time shift and transformer phase errors. I can calculate the time shift error, but I don’t think I can calculate the phase error between the voltage and current transformers, so I will have to rely on measurements alone for this error.

Assuming I’m on the right track now, when it comes to using measurements for correction any remaining phase error, should I be looking where the “zeros” of each wave are? For instance, if the zero of ct1 happens in sample number 123, but the zero of the voltage reading happens in sample number 125, there are 2 samples worth of time that I need to correct for, which in my case is 64µs.

So, when generating a new waveform to correct for phase - can I apply the correction technique to the individual current waves, or do I have to generate 5 slightly different voltage waves for each individual current sensor? I’m thinking that perhaps the irregular current waveforms for various electronics won’t take nicely to the phase correction algorithm, whereas the nicely shaped AC sine wave will.

I intend to collect 2k samples from each channel, then go back and perform the phase correction maths for each channel using either: 5 variations of the voltage measurements applied to the respective current measurements, or adjust each of the 5 current wave forms. (To me, it doesn’t matter which method I choose as the number of CPU operations will be the same).

For a bit of miscellaneous stuff… Yesterday I invested the time in adding a debug mode to my software. Previously, in order to visualize the raw data (like for all the screenshots of graphs I shared thus far)…
I would have to set the filename for the CSV data dump in the code, run the program, terminate it, copy the CSV out to my computer, upload it to Plotly, build the chart, name the traces, etc. Now, with my new debug mode, I simply run my program, pass the debug keyword in, followed by a filename, and it dumps an interactive chart with a fresh set of 2000 samples to a local webserver on the Pi that I can view within seconds. Example:

$ python3 power-monitor.py debug "example for OpenEM"

Web server root now has “example for OpenEM.html”:

Clicking on the above filename gives me the chart itself, which I can pan and zoom any which way, or turn off individual data series.

Well worth the time!!

Robert.Wall · 29 March 2020 21:02

Right. Can I call the 2000 of my sets a “batch” of samples?

That’s correct - unless you have the ability to measure each transformer individually, or each pair.

No, that’s unreliable, because the wave shape does get distorted. In your research, did you come across the B-H curve? What it means is the c.t. isn’t linear, so a sine wave in comes out distorted - but worse, the distortion has memory, so the distortion as the current comes down from a peak is different to that on the way up. The only realistic way is to to compare the real power you calculate against the product of voltage and current.

That’s more than likely, and I saw it coming earlier when I wrote

What I’d do, given that you have a store of 2000 samples of voltage and each current, and you’re processing them post-collection, is take a few more samples, then depending on the amount of correction, pick the offset for the pairs of voltage samples that you’ll interpolate between so that you are interpolating (i.e. PHASECAL in the equation is between 0 and 1). That gives the least errors. How many extra samples you need will depend on the offset.

[We can’t do that with the '328P because there isn’t enough memory, so the samples have to be processed on the fly.]

That’s exactly why the shift is always applied to the voltage wave.

I haven’t thought this through, but I don’t think that’s necessary. If you do the maths the way emonLibCM does them, you need what I term the “partial products” for a spread of voltage samples. It’s really hard to explain, so take a look at the maths on the first page of this:
emonLib Maths.pdf (152.8 KB)
and the “partial products” are A & B. Depending on the amount of shift you want, when you apply average power = ... you pick A & B or B & C, or C & D so that PHASECAL is always between 0 and 1.

That indeed looks well worth the investment in time and effort.