First, note that channels that are not connected to anything will pick up noise and read wildly varying random values, which are probably not 0!
Second, the channels can and do cross talk a little through the chip so grounding any analog inputs you don't use is wise.
An unstable power supply will cause varying readings as well.
Note that it is normal for an A/D to have a few bits of noise. Also, most things you measure are not stable down to 1 bit resolution anyway (about 5mV by default). It is not uncommon to see small variations.
If you need maximum accuracy, consider using adconf to set up a separate voltage reference using a stable voltage reference IC (not a regulator). A crystal oscillator instead of a ceramic resonator can also make minor improvements in repeatability.
Averaging A/D values can also reduce noise. The repeat command is very handy for this since you can issue a repeat to quickly read a number of samples (say, 4 or 8) and then add them together and divide in software (remember dividing by 4 or 8 is just a shift right if you are trying to get best performance).