The big penalty for using the higher voltage is it restricts your choice of current transformer. There is a huge range of c.t’s designed for a 1 V reference (or thereabouts), i.e. 0.333 V rms output. As you increase the reference voltage, you either reduce the resolution by using only part of the input range, or you need higher power rated current transformers to deliver the higher voltage.
It’s easier in the voltage department. For a start, the a.c. line voltage is more or less constant (compared to the current range, anyway), but even so, the only purpose-made voltage transformer I know of has a 0.333 V output.
The disadvantage of the more sensitive input is it ought to pick up more interference and noise. However, in our experience, the usual problem with noise is not noise picked up on the input cables, but noise getting into the ADC via the power supply or the voltage reference, and that’s down to proper circuit layout, filtering and grounding.
A little while ago, my colleague @Bill.Thomson posted a link to a piece about the ADC and clock speed - that is worth tracking down and reading. (My search skills have failed me - I can’t provide a link.) Also, don’t ignore the Atmel data sheets and application notes - they really should be authoritative sources.
See also About ATMEGA328P with 8Mhz internal oscillator crystal. Standalone - #3 by dBC