EmonLibCM - Version 2 (Support)

Robert.Wall · 2019-10-26T22:54:29.617Z

I’m not the right person to answer his, however my perception is the mechanism is as follows:

Incoming data is dropped into a “slot” depending on its arrival time. While the drift between the sending clock and the receiving clock remains negligible (and of course the ideal starting situation is when the incoming data arrives exactly in the middle of its slot), all is well. If the drift is consistent in one direction - say the sending clock runs slow, the data will arrive late and eventually fall into the next slot, and there will be a null value recorded in the intended slot. I believe this is what feed quality is measuring. Alternatively, if the sending clock runs fast, data will arrive early and eventually catch up with a slot that already contains data. In that case, the previous data is overwritten and lost. I believe this was the strategy adopted for the default sketch that used emonLib, i.e. the time between samples was made shorter than the nominal 10 s (or 5 s) rate of the database clock. It was deemed less bad to overwrite a sample than to leave a null value.

dBC:

Does anyone have any tips on how I can check if my grid induced jitter is messing up my feeds please?

Short of changing to the database that records the arrival time (maybe temporarily, because of the cost in data storage) and checking the time stamps individually, I don’t know of a way to do that. If it’s genuine jitter and not drift, you could be in a good situation and not lose any samples if the mean time of arrival falls on the midpoint of the slot, or a bad situation if the mean time falls on the slot boundary, in which case many samples will miss their slot and either leave null values or overwrite previous values.

I guess a possible solution would be to have a receiving clock phase locked to the arrival time of the data, so that the data always arrived near to the centre of the slot. It gets rather unwieldy with more than one data source, and the immediate question is how do you know what is the correct time to label the data with?

Another possible solution (as intimated only today in another thread) would be to poll the sensor node rather than have it free-running. That is a complete change of strategy that would be difficult to implement and maintain a degree of compatibility with the battery-powered emonTH, for example. And more so for emoncms.org.

Of course, if both ends use mains time and are on the same supply, the problem is jitter alone and phase locking, provided the jitter is less than half the reporting rate, is a complete solution.

dBC · 2019-10-27T08:36:51.344Z

I guess in the spirt of “leave no cycle unmeasured” that all argues for the front end accumulating energy locally and sending the current total to emoncms regularly. Then if things fall in the wrong timeslot it just means the consumption was recorded as occurring at the wrong time (slightly), but no consumption gets lost - which could start to add up.

Can someone tell me where I find the “quality” measurement for a feed please?

rcojoe · 2019-11-18T21:44:07.015Z

@Robert.Wall Thank you for providing this code. I’ve been using the non CM version of the library for many years and it has taught me a great deal about my power usage. So I am about to change out some appliances in my home, but before I do that, I wanted to upgrade my library to the CM version so I could get comparative results using the old appliances with the CM library.

I am using a Mega2560 because of the additional analog ports (A8-A16) and but I was only using A8-A10. When I take a reading from those analog ports using the CM, the values are off by about 8x, so I took a look at your library code and it looks like the second ADC is not being initialized at all (probably due to not even considering using it). If I extend the EmonLibCM_Start and EmonLibCM_Stop functions to include the ADCSR(b) ADC settings that you had initialized in the ADCSRA, I think it might work. It just seems that the settings are 8X larger that they should be. ADCSRA analog reading are dead on.

Why allow for 5 channels max? Is it to make sure we get enough samples for an accurate average?

Robert.Wall · 2019-11-18T23:43:55.134Z

rcojoe:

Why allow for 5 channels max? Is it to make sure we get enough samples for an accurate average?

Basically, yes and because it was written for the Atmel ATMega328P (it does actually say that in the documentation) and that has rather fewer analogue inputs than the Mega2560.

What you need to look at (I don’t know the '2560 and I’ve only thought about this for a few seconds) is the time for each ADC conversion - you may be able to improve that by restricting the width - and running both ADCs in parallel (of course, the '328P has only one) but for that you must create a second ISR to handle the results and make them available to the main loop.

I wish you well with your endeavours.

dBC · 2019-11-19T01:34:14.713Z

Actually, the 2560 doesn’t have a second ADC. I sometimes hear it referred to as a “more powerful” Adruino, but for as far as compute power and ADC performance the two are identical. They run the same AVR core at the same clock speeds, and use the same lone ADC. What the 2560 does offer is more pins, a wider analog mux in front of the lone ADC and more flash and ram.

So you can write bigger programs, and declare bigger C arrays, and sample more analog pins… in that sense I guess you could say it’s more powerful, but in terms of ADC performance and V*I maths performance, the two processors are the same. Total sampling rate is identical, so introducing more channels means the sampling rate per channel will be proportionally reduced.

Robert.Wall · 2019-11-19T13:27:18.577Z

5 posts were split to a new topic: Data Quality in emonCMS

Robert.Wall · 2019-11-19T13:26:15.298Z

@dBC Thanks for clarifying that, I’d taken Joe’s statement at face value.

@rcojoe
On that basis, I don’t think its feasible to extend the number of channels being monitored beyond 6 (5 current, 1 voltage, giving roughly 32 samples per cycle per channel - 15th harmonic for us but even less for you in the USA) because the number of samples per cycle will reduce to the point where the bandwidth suffers and aliasing - unless effective anti-aliasing filters are incorporated into the hardware - will become a problem.

I think you should stop and do some basic maths to decide how many samples per cycle you need, if you can control the ADC resolution then what bit depth you need there (because there’s a speed trade-off to be had), and that will give you the maximum number of channels you can ‘simultaneously’ monitor.

rcojoe · 2019-11-19T15:49:29.542Z

Thank you @dBC and @Robert.Wall. I looked at the ADC for the 2650 last night and I couldn’t find any differences in the ADC so it must be the same ADC. I should only be sampling the pins that need to be sampled by the way I initialized the emonlibcm, keeping my pin count to the 5. 1,2,8,9,10. My readings on 8,9,10 are not as expected like they are on pins 1,2. So I may have to do some pin swapping to see if the odd readings are related to the higher pinset or related to my hardware. I am using the exact same configuration # and phase offset as I did in the original library, and my first two inputs on 1 and 2 are reading correctly. I’ll give it another try tonight.

Joe

dBC · 2019-11-19T21:37:14.539Z

rcojoe:

My readings on 8,9,10 are not as expected

OK, when I said the ADCs were identical but for a wider mux I was over-simplifying things a bit. The core of the ADC (and hence its performance) is identical, but sitting in front of it is not only a wider mux but also some op-amps allowing differential inputs and 10x and 200x amplification.

If I’m looking at the right code, it does this…

static byte ADC_Sequence[max_no_of_channels+1] = {0,1,2,3,4,5};        // <-- Sequence in which the analogue ports are scanned, first is Voltage, remainder are currents
...
ADMUX = 0x40 + ADC_Sequence[next];       // set up the next-but-one conversion

The datasheet reveals why you can’t just replace those numbers with 8,9,10. A setting of ‘8’ there will get you ADC0 with a 10x gain. A setting of ‘9’ will get you (ADC1 - ADC0)x10 and a setting of ‘10’ will get you ADC0x200. You’ll need to do some simple maths on the channel number first. For channels 8 and above you need to set MUX5 and it lives in a different register (ADCSRB). Check out how analogRead() in the standard Arduino libraries converts channel numbers into register writes and mimic that.

Screenshot%20from%202019-11-20%2007-31-16

Screenshot%20from%202019-11-20%2007-31-29577×792 45.3 KB

Screenshot%20from%202019-11-20%2007-31-43584×645 42.2 KB

rcojoe · 2019-11-19T22:29:50.717Z

@dBC Yes! That’s the answer to the question I didn’t know to ask!! I did see the same tables in the datasheet and noticed the the MSB was 1 when querying the 8+ pins. It was late and I was trying to figure out where the library was querying the ADC for each pin, which I had not fully comprehended yet. So MUX5 32,33,34 = physical pins 8,9,10. lets see how that goes. Thank you so much for taking the time to laying it out. You saved me a ton of research time (even though I do like that part too cheers).

Joe

dBC · 2019-11-19T22:44:21.665Z

You’re welcome.

But not exactly. In those tables above they show MUX5:0 as a nice contiguous 6 bits, but the reality is they’re not. MUX4:0 are the bottom 5 bits of ADMUX while MUX5 lives in bit 3 of ADCSRB. Here’s how the Arduino analogRead() handles it…

#if defined(ADCSRB) && defined(MUX5)
	// the MUX5 bit of ADCSRB selects whether we're reading from channels
	// 0 to 7 (MUX5 low) or 8 to 15 (MUX5 high).
	ADCSRB = (ADCSRB & ~(1 << MUX5)) | (((pin >> 3) & 0x01) << MUX5);
#endif
  
	// set the analog reference (high two bits of ADMUX) and select the
	// channel (low 4 bits).  this also sets ADLAR (left-adjust result)
	// to 0 (the default).
#if defined(ADMUX)
#if defined(__AVR_ATtiny25__) || defined(__AVR_ATtiny45__) || defined(__AVR_ATtiny85__)
	ADMUX = (analog_reference << 4) | (pin & 0x07);
#else
	ADMUX = (analog_reference << 6) | (pin & 0x07);
#endif
#endif

Robert.Wall · 2019-11-19T22:45:12.862Z

I’ve got to repeat, the library was written for the '328P. You must expect to have to make changes when you port it to a different processor, even when it’s part of the same family.

rcojoe · 2019-11-19T23:05:49.953Z

@Robert.Wall Thank you for your continued work on this library. You have contributed greatly to all of us interested in power consumption and made it extremely easy to load and measure quickly. This is a great solution, and my topic is not a bug, fault, or a bump in your roadmap. I’m the one at fault here by hijacking the support thread for your well commented library. Thank you again for all of this.

@dBC Thank you for spoon feeding me. I see exactly why you wanted me to look at the analogread code. Your knowledge and attachments will get me over the goal line.

Robert.Wall · 2019-11-19T23:47:01.369Z

I wasn’t criticising you for porting the library, but pointing out that there might well be other areas where you need to look very carefully at the differences between the two processors.
E.g., the recently added API call to use the 1.1 V reference might well need changing, and I see the '2560 also has a 2.56 V reference, which the '328P doesn’t have - so there’s no provision to switch to it - but which in some circumstances you might want to use.

I don’t have access to a '2560, so even if I did put forward changes, I couldn’t verify them.

rcojoe · 2019-11-22T03:53:57.268Z

@dBC Thank you for your help. I added these lines and everything is working correctly now.

@Line 920
//****Begin New Block for ADC A8 and above**********
#if defined(ADCSRB) && defined(MUX5)
	// the MUX5 bit of ADCSRB selects whether we're reading from channels
	// 0 to 7 (MUX5 low) or 8 to 15 (MUX5 high).
	ADCSRB = (ADCSRB & ~(1 << MUX5)) | (((ADC_Sequence[next] >> 3) & 0x01) << MUX5);
#endif
//****End New Block for reading ADC A8 and above**********

//*********Modified existing line below to add "& 0x07" to handle A8 and above
  ADMUX = ADCRef + (ADC_Sequence[next] *& 0x07*) ;     // set up the next-but-one conversion

Derek_Jennings · 2020-04-22T12:19:54.951Z

I recently used the EmonLibCM library in a project and during debugging I noticed what looks to me like an error in the maths.

In EmonLibCM_Init() we have


double phase_shift = (phaseCal_CT[i] / 360.0 + ADC_Sequence[i+1] * 
                               (double)ADCDuration * cycles_per_second/MICROSPERSEC) * two_pi;                // Total phase shift in radians

If phaseCal_CT[i] is in degrees and the intention is to convert to radians then given there are 2pi radians in 360 degrees, then shouldn’t this calculation be :


 double phase_shift = (phaseCal_CT[i] * two_pi / 360.0 + ADC_Sequence[i+1] * 
                               (double)ADCDuration * cycles_per_second/MICROSPERSEC) * two_pi;                // Total phase shift in radians

The same calculation occurs in EmonLibCM_ReCalibrate_IChannel()

Apologies if I have this completely wrong. It is a long time since I did my GCE maths.

Derek_Jennings · 2020-04-22T15:03:42.909Z

Ignore my last post. I just noticed the two_pi outside the brackets.
As expected it is my mind at fault

djh · 2020-04-22T15:32:16.859Z

It would be more usual to put multiplication by a constant like that at the front of the calculation.

Robert.Wall · 2020-04-22T15:42:47.009Z

As if the compiler cares.

djh · 2020-04-22T15:47:51.686Z

Exactly. With a modern compiler that can optimise effectively it’s more important to position terms for human readability than perceived machine optimisation. And multiplication by a constant factor like 2π usually comes at the front of an expression in mathematics.