You're absolutely right. The focussing is different in comparison #25. That's very slack of Dpreview. I'd never be so sloppy if I were conducting such tests .
The focusing inconsistency is not the only fault in DPReview's testing scheme. Using JPEG at defaults leads to many problems since the default tone curve and sharpening parameters may be quite different for different cameras and this will affect the results. Besides, most critical users do not use JPEG with its limited color gamut (restricted to aRGB on Nikon cameras), 8 bit depth, and relatively unsophisticated sharpening. Bruce Fraser would roll over in his grave.
Use of raw files is more appropriate, but most cameras do not have pure raw output with no processing. The choice of ACR for a raw converter is reasonable, but the ACR algorithms may work better for some cameras than others, and ACR does apply noise reduction in the demosaicing process even if NR is set to zero. They use the same sharpening parameters for their ACR tests, but this gives an advantage to the camera with a weaker blur filter. The D3x apparently has a relatively weak blur filter and I don't know how it compares to that of the 1DsMIII. A strong blur filter requires more sharpening with a small radius as Bruce explained in his sharpening book. It is increasingly common for some lens aberrations (e.g. chromatic aberration) to be corrected in software. For example the Nikon D3 series has an algorithm for automatically correcting lateral chromatic aberration; this correction is applied by the in camera JPEG engine and by Nikon Capture NX using metatdata written to the raw file. In ACR the chromatic aberration correction must be applied manually.
Their resolution tests are relatively useless as they measure resolution at extinction, at which point resolution vanishes. It makes more sense to measure resolution where there is resolution. Without even looking at their test results, you can predict that the resolution will be about 75% of Nyquist. MTF at 50% contrast would be more useful, but this value is markedly affected by sharpening.
In their dynamic range tests, they talk about highlight and shadow "headroom", but this is dependent on the tone curve in use. Raw files are linear and there is no inflection point to indicate mid range values. For the limiting shadow noise used in determining DR they use an unspecified standard deviation, but a standard deviation does not adequately quantify noise, since perceived noise depends on the noise spectrum. As explained on the DXO site, high resolution sensors may have a higher standard deviation for noise, but the perceived noise may be less because it has a higher frequency. Also, noise reduction can have a dramatic effect on DR--you could have a high DR but no resolution.
In summary, the DPReview tests reports are useful for giving a detailed list of camera features and specifications, but they are relatively useless for determining how a camera will perform in the field with a given work flow used by sophisticated photographers.
Resolution tests require the utmost accuracy of focussing. (But seriously, we all make mistakes sometimes.)
Yes, but when you set up yourself as an expert authority, mistakes are more glaring and less forgivable.
