I’ve got one of the Adafruit boards on my “radio hub” pi. They’re a good quality board with convenient labels and a layout that makes it very easy to transfer a circuit from a breadboard.
Thanks for your input! I went ahead and ordered a few.
Your ADC is a MCP3008?
If you haven’t done so already, read this: http://ww1.microchip.com/downloads/en/Appnotes/00688b.pdf
OK, you’re not making a p.c.b, but there is a lot of good info in there. Microchip do know what they’re talking about.
Note they recommend 1 µF, not 0.1 µF. I’d go along with that. It won’t hurt to leave the 0.1 µF there as well, in parallel.
According to me, the DGND (digital ground) is pin 9 and VDD is pin 16, so those are the pins that have the 1 µF across them. I think from your photo that VREF is also directly connected to the 3.3 V. That’s likely to be full of digital noise. You need to use a small inductor in series to block the noise coming in that way, again with a 1 µF to decouple the pin, this time to the analog ground. You need to keep the analog ground and digital ground separate, as far as possible. All the analog circuitry needs to come together and be connected to the analog ground, with only one connection to the digital ground, ideally at that 220 µF capacitor, which needs to be where the 3.3. V first comes onto your board from wherever the power comes from. As you have it, the current supplying the ADC ( = full of digital noise) passes along those bus bars to which all the analogue grounds are connected. You need to take the power for the ADC directly from the breakout board and only have the analog ground connected to the bus bar.
It wouldn’t do any harm to twist those wires to the c.t. socket together.
If you have them available, I’d put a 1 µF at each end of each of those power supply busses, in addition to the one as close to the ADC pins 9 & 16 as you can get it.
It’s quite normal to read a fairly high noise value when you’re looking at the rms value, because the random noise is rectified by the squaring process in the rms maths, so the positive and negative values of noise can’t cancel each other out. When you move to real power, the noise should be much lower.
Yes, thank you for the link! I don’t have any disc capacitors that are 1µF - only electrolytic. I was doing some additional reading about MCP3008 noise on the RPi3 and the discussion I found suggested not to use electrolytic capacitors because they have inductive properties due to the way they’re made. I can get some ceramic caps if it’s recommended, though.
Yes, that was the case in the initial picture on a separate, unenergized breadboard. I did further reading and corrected it, then replaced the picture above just a few minutes before you replied, and I think the timing worked out so you didn’t see the new picture.
I’ll have to redo my grounds. Right now everything goes through the bus bar. I will try taking the Dgnd directly to one of the Pi’s ground pin and leave the bus bar for the analog ground (which will go to a separate ground pin on the Pi).
Done!
The real power reading seems to be spot on - where the noise levels are between -1 and +1 W. Since the real power is accurate, I could do something like the following to account for noise on an empty line, but I’m hesitant to do any sort of “cheating”:
if real_power > -1 and real_power < 1:
real_power = 0
if rms_current <= 0.3:
rms_current = 0
… so that the RMS current will read 0 only if the real power is essentially 0. In fact, I’ve tested the above, and it’s not that great when the real power is low, like around 25W or so. The RMS current is still artificially inflated by ~0.29A with such a small load. The tuning I’ve done fixed up the accuracy for larger loads, around 700 - 1300W.
While the above cheat-method might work for the house mains where the current will most likely never be < 1A, it won’t really work for my solar main due to the gradual rise from 0W to peak, and back down again. Real power is fairly accurate, but I’m intending on storing all metrics including RMS voltage and RMS current, so I’m striving for accuracy. As described in the paragraph above, the RMS current on the solar main won’t be accurate for small loads (probably from 0-100W+) until I can improve the noisy CT input. I’m still waiting a few more days for the additional CT sensors to arrive, so I will wait until the weekend to revisit the noise problem once I have the other CT sensors connected to the board.
I’ve also began thinking about my data storage mechanism. I’ve read the Time Series section of the guide and wow, what a journey you guys have had! My implementation will have some small averaging of the raw data before putting it in the database (otherwise I’d be looking at 31.1M rows per year), and I’m currently investigating disk space requirements for different averaging intervals.
Indeed - that’s why most people use the two in parallel - the smaller disc ceramic deals with the high frequencies where the capacitor’s inductance reduces its effectiveness.
1 Like
Maybe a good reason to look at InfluxDB and Grafana.
You can set retention policies to keep your database from growing past a specified point, as well
as downsample the long term data. (Paul Reed does that, see this thread: Grafana Dashboard Project)
Grafana has the ability to display data smoothed via a moving average.
In fact, you can actually do a fair amount of math right in Grafana.
Food for thought…
2 Likes
Thanks for the recommendation, Bill. I was planning on using SQLite and Grafana for my backend/frontend. I’m not that familiar with InfluxDB but at first glance it seems to be a solid candidate for the job.
I’ve never used Grafana before - historically speaking, I’ve used matplotlib and Plotly for my charting and reporting needs in other projects. But my research into Grafana has shown me it’s so much more than that! Thanks for the link to Paul’s thread.
Odd coincidence that you mention SQLite, as that’s the database Grafana uses to store its internal
configuration.
I run both, (Influx and Grafana) as does Paul, and there may well be other users here who do too.
I can think of at least one user who runs Influx but uses a different graphing app.
I’m willing to help with both apps when and where I can.
Both apps have active forums with helpful users, so support shouldn’t be too much of an issue.
We’re here to help.
1 Like
Thanks Bill! SQLite was my initial go-to for this project since I am going to try to run everything on the Pi 3B+, and SQLite is fairly lightweight as I’m sure you know. It looks like InfluxDB requires a server process like other DB engines, but I will give it a try because I’m interested in learning first-hand about the benefits it offers as a time-series database.
If I run into performance issues on the Pi I may have to move to SQLite, upgrade to the Pi 4, or do the data collection and monitoring on another system (like my QNAP NAS) and just probe the Pi for raw data. I had originally wanted to use the Pi Zero W, but I’ve already ran into performance issues just with the raw SPI data collection on that system. Looks like I have plenty of options to get started!
Although that’s true, Influx/Grafana performance is reasonable on a Pi 2, and as you would expect,
even better on a Pi 3. I haven’t bit the bullet and bought a Pi 4 yet, but I can’t imagine the performance
not being good on a “4.”
Not sure what you mean, but both Influx and Grafana are their own servers, and both are single file executables.
You should be OK. I ran both on a Pi 2 for about two years and never had an issue.
That’s understandable, as the Pi Zero is essentially a Pi 1 running at 1 GHz vice the 700 MHz
the original Pi 1 ran at. On top of that, the CPU is a single core device. RPi 2, 3, and 4 all have
multi-core CPUs, as well as faster clock speeds.
1 Like
That’s good news for my 3B+! Just curious, why would you not expect performance to be good on a 4?
I was comparing SQLite to Influx on the basis that SQLite does not require a server process to interact with the database, whereas Influx and other DBMSes do. I was just trying to suggest that Influx will have that much more overhead… however much that is, I don’t know, and it probably won’t matter too much.
I was admittedly over-optimistic about the Zero’s capabilities, while at the same time not understanding all the complexities I was facing as I opted to venture out on this project. I’m not opposed to buying a Pi4 for it, but I’ll try to stretch the 3B+ a lot further than I did with the Zero.
On another note, I believe I’ve resolved the source of the “noise”. Turns out it was a programming error! 
I walked through my code step by step to see why the RMS current seemed to be calculated at a static value of 0.29A. I dumped the raw data from the ADC and took a look at it - my circuit’s “zero” (or “reference”) level is 511, and I was using 512 in my calculations. So, for a single poll cycle of 2000 data points, my sum_squared_current variable was about 2000, leaving an RMS current value of sqrt(2000), times my current scaling factor and tuning correction, for a final, RMS current approximation of 0.29A. Now that I’ve corrected that, it is looking much better:
Real Power: 0.018 W | RMS Voltage: 121.878V | RMS Current: 0.011A | Apparent Power: 1.369 W | Power Factor: 0.013
Real Power: -0.022 W | RMS Voltage: 121.874V | RMS Current: 0.014A | Apparent Power: 1.767 W | Power Factor: -0.013
Real Power: -0.047 W | RMS Voltage: 121.914V | RMS Current: 0.013A | Apparent Power: 1.581 W | Power Factor: -0.03
Real Power: -0.016 W | RMS Voltage: 121.867V | RMS Current: 0.017A | Apparent Power: 2.091 W | Power Factor: -0.008
Real Power: 0.095 W | RMS Voltage: 121.854V | RMS Current: 0.018A | Apparent Power: 2.235 W | Power Factor: 0.043
The output above is from a CT sensor that is surrounding a non-energized conductor. I’m not sure why some real power values are negative, but I would guess it’s because I have a small uncorrected phase error between the voltage reading and the CT sensor in use.
I really appreciate everyone’s input, time, and dedication to helping me through this project so far. I’m almost there!
Next steps will be to wire up the other 3 CT sensors on the breadboard and connect them to the mains so I can start working on the software. Once I’m confident that my circuitry is solid and have some momentum with the backend programming I’ll transition the project to the prototype boards.
I think you misunderstood me…
There’s a double negative there. 
You should be good to go with a 3B+
I noticed an improvement in performance when I went from a Pi 2 to a Pi 3.
Keep 
ing on it! 
2 Likes
Do you mean you’re using a fixed value to remove the offset imposed by the need to apply an offset voltage to the ADC input?
If yes, that’s a seriously bad idea because drift in the component values can make it shift over time, so your problem may creep back and you’ll not notice it. Provided that the offset is not changing too quickly, i.e. not discernibly within the averaging period of your measurements, it’s much better to calculate the average and then subtract the power/current/voltage that represents. (Or if you’re worried about the size of the numbers, subtract the fixed ‘512’ just to get close and then calculate the average to trim the values accurately.)
True power = calculated average power - (offset voltage × offset current)
True rms current = √([calculated average of current²] - [average current]²)
1 Like
Doh!
Yes.
This makes total sense now! When you had mentioned it last time, I wasn’t too sure about it. After my testing last night and inspecting the raw data, I was using the average from 10 sample sets (20k samples) to find the reference value in both the voltage and current sensing circuits. I’ll make the change this evening. Thanks Robert!
Also, I do intend on releasing my code when I have something working. I want to contribute to the project and maintain the ethos, and sharing my project is the best way I can think of.
1 Like
Robert, I’ve been thinking more about optimization since I removed the static 512 “reference” variable from my calculations, and I have come across a couple of points I wanted to ask you about.
First off, it would be nice to remove all static values from my calculations, but I can’t seem to come up with a way to get rid of the static value I’m using in my scaling factor calculation. The static value is 3.31 (as previously measured with a multimeter) for the voltage of my board. I currently take 10 readings from the board (via a channel on my ADC), and I multiply the average reading by a constant 3.31 to get the approximate board voltage, ie:
samples = []
while len(samples) <= 10:
data = readadc(board_voltage_channel)
samples.append(data)
avg_reading = sum(samples) / len(samples)
board_voltage = (avg_reading / 1024) * 3.31
Later on in the execution of my program, I use the board_voltage in the scaling factor calculation:
current_scaling_factor = (board_voltage / 1024) * 100 * (1 + CT_accuracy_factor)
voltage_scaling_factor = (board_voltage / 1024) * 126.5 * (1 + AC_voltage_accuracy_factor)
total_scaling_factor = current_scaling_factor * voltage_scaling_factor
My efforts in finding a way to remove this constant have led me to believe there has to be an assumption somewhere in the code, otherwise we can’t turn the ADC output into meaningful data.
My next question is about the phase correct algorithm. Dealing with such small timescales, the order at which voltage and current are sampled will affect the outcome. I was just experimenting with this idea by changing the order in which voltage, CT1, and CT2 are being read.
My 2 CT sensors are on the same conductor about 10cm apart from one another. Out of curiosity, I tried to “sync” the CT sensor readings by removing a single sample from the head of one CT data set (and removing the tail on the other to keep the data sets the same length). This was too large of a correction and increased the phase difference between the two sets.
The question is - can the phase correction algorithm be applied to different sensors to correct for the timing difference at which they are read? Or, is calibrating at this level likely not necessary? Here is a data sample of my voltage, CT1 and CT2 readings, where the order of collection is voltage, CT1, CT2:
Also, it looks like I’m only getting about 35 samples per 60Hz period, per channel, with 3 channels currently being read. That’s about 2100 SPS per channel - or about 6300 SPS for the ADC. When my other 2 CT sensors are in use, the per-channel rate will go down. Earlier in this thread (post #11), you had mentioned that EmonLib gets about 44 sample pairs per cycle. Is a pair, in this context, 1 CT reading + 1 voltage reading? I just want to ensure I don’t fall too far below what the project has already established as fast enough sampling.
I suspect I’m hitting the limitations of the Pi’s clock speed because the MCP3008 has a max sampling speed of up to 75 KSPS at 2.7V. I’m running the chip at 3.3V so in theory, the ADC should be able to hit sample speeds near this threshold, assuming a device can request data from it at that speed? This reminds me of a saying my flight instructor would say: “In theory, practice and theory are the same. In practice, they aren’t.”
Somewhere, you need a way to tell the code the relationship between ADC counts and the reference voltage. Usually, the maximum count represents the reference voltage. Either you can hard code what you think it should be - meaning that any error in it gets added to the general calibration error, or you can measure it and type it in as a calibration variable, or if you have an internal reference of sufficient accuracy that it can measure against its own reference, then you can derive the actual calibration from that. You must have one of those, there’s no escaping it.
What is board_voltage_channel actually measuring?
Yes, that’s correct.
It can. The total error you’re trying to remove has 3 components - the time between readings, the phase error of the v.t. and the phase error of the c.t. The two transformers are likely to be different. If they’re different by exactly the time difference (converted to degrees), get them in the right order and no correction is needed. Otherwise, and most likely, you should choose the order so that the minimum correction and preferably always interpolation (i.e. the correction constant is between 0 and 1) is needed.
EmonLibCM does things differently and more like the way you’re doing it. That does one sample every 104 µs, so for you with 3 samples per set now, and later 5, that would do 53 or 32 sample sets per cycle. The library will actually take 1 voltage and 5 current inputs (all the analogue inputs of the '328P) and that’s really on the bottom limit of acceptable, especially on a 60 Hz system.
Or the SPI - which is (apparently) tweakable.
Hmm. I’m a lapsed private pilot.
1 Like
board_voltage_channel is just an integer representing the ADC output channel. readadc() is the function that returns a single reading from the channel passed to it.
Since my last reply, I’ve been busy incorporating the remaining 2 CT sensors on the board. I also increased the MCP clock speed to 1 MHz where I can now read about 95 samples per 60Hz cycle, or about 4588 SPS, per channel, for a total rate of 22.9K SPS (5 channels total). The Pi is handling the processing of the data flawlessly.
Here is how the data looks from all 4 CTs and the voltage sensor with a ADC clock speed of 1MHz.
Anything over a 1 MHz clock speed on the ADC yields inaccurate data which appears to not be of concern since I’m able to achieve a fairly high sample-per-cycle rate. Now I have first hand experience of why the original ADS 1115/1105 chip that I started with is not fast enough!
I was totally incorrect on my suspicion. I had mistaken the Python spidev’s library max_speed_hz attribute for the max sample speed possible. The MCP3008 ADC reportedly has a max sample rate of 200k SPS when run @ 5V (slower, if run at 3.3), and I made the assumption that setting the max_speed_hz attribute was the same as setting the sample rate. When I incorrectly suspected I was hitting the limitations of the Pi’s clock speed, I was under the assumption that it wouldn’t make a difference if I set the max_speed_hz variable higher than 200k. Boy was I wrong!
Very cool! I am a private pilot as well (though I am fully current) 
I am preparing to move my project to the Adafruit Perma-Proto boards I purchased - unfortunately, all but one of the 5 individual circuits fit on a single Perma-Proto board. I’m going to have to either stack two boards and deal with wiring the ADC input from the top board to the bottom board, or come up with a different prototype board solution. I feel that my circuits are already as small as I can get them, with a single CT circuit taking up 6 rows, and the voltage sensor taking up 9 rows (with a female DC jack).
1 Like
Is it worth your time to grab KiCad/Eagle/Fritzing and design your own Pi “Hat” PCB?
You can get PCB’s made in small quantities quite cheaply these days, most place offer very cheap rates for “first time buyers”.
Look at the DIYBMS threads for more details of custom PCB manufacturers.