I don’t know the details of the Arduino, so I cannot be certain - especially by remote control! But you do need to check the voltages between the various earths, it’s a common problem when you are connecting multiple items together.
Have you looked at the Arduino 5 V with everything that it’s possible to disconnect, including your op.amp etc, totally disconnected? If it’s rubbish then, you’re going to struggle to use the analogue inputs to measure anything much less than half a volt.
Also, try looking at the raw power without the Arduino (but if you can rustle up a resistor load that takes roughly the same current as the Arduino, with that connected. You should soon see whether it is the power supply or the Arduino that’s the source of the noise.