DxO tests

[ripgriffith]

I'm confused!  The DxO sensor tests of the Sony A65, A77 and NEX 7 show significant differences between the cameras, particularly in the low-light test (717, 801, 1016 respectively), even though I am under the impression that they share the same sensor and processor.  Where am I mistaken here?

[NikoJorj]

The semi-transparent mirror?
http://www.dxomark.com/index.php/News/DxOMark-news/Sony-NEX-7-comparisons-and-review

[ripgriffith]

My understanding is that the tests of the sensor are just that- tests of the sensor alone- and not of any intervening structure such as mirrors or lenses.  Also, don't the 65 and the 77 share the same translucent mirror technology?

[deejjjaaaa]

My understanding is that the tests of the sensor are just that- tests of the sensor alone- and not of any intervening structure such as mirrors or lenses.  

your understanding is wrong... DxO tests the data in raw files received from a camera and written by camera's firmware (hence w/ whatever manipulations manufacturers do) and if you will actually bother to read their explanations about their tests you can see that they indeed use some (may be not OEM - but custom designed) lens (= they do not illuminate the sensor directly) - there are photos of their lab and test in process, for example = http://www.dxomark.com/index.php/About/In-depth-measurements/DxOMark-testing-protocols/Noise-dynamic-range ( w/ lens) and http://www.dxomark.com/index.php/About/In-depth-measurements/DxOMark-testing-protocols/ISO-sensitivity (w/o lens for ILC and w/ lens for  P&S) and http://www.dxomark.com/index.php/About/In-depth-measurements/DxOMark-testing-protocols/Color-sensitivity (w/ lens)

[ripgriffith]

Thank you for adding to my rather meager store of information about their testing.  However, this does not speak to the question of different results from the same sensor/firmware pair, such as one finds with the Sony A65 and A77 which, if I correctly understand the various reviews, are essentially identical as to the sensor, firmware, translucent mirror, etc. 

I do not want to get into a deep discussion about various testing technologies and practices, I am simply looking for a basic answer to this question: Do the A65 and A77 have identical sensors dealt with in an identical manner by the processor and, therefore, deliver identical information to the raw file and, if so, why the difference between the tests in DxO?  Truth be told, I spend my time shooting, not reading about testing procedures.

[deejjjaaaa]

However, this does not speak to the question of different results from the same sensor/firmware pair, such as one finds with the Sony A65 and A77 which, if I correctly understand the various reviews, are essentially identical as to the sensor, firmware, translucent mirror, etc. 

DxO measures (look @ graphs - not @ scores) A65 and A77 almost identical - the only difference is that senior model has ISO50 enabled - the rest is within margins of testing errors...

[ripgriffith]

Thanks, 'nuff said!

[bradleygibson]

...you can see that they indeed use some (may be not OEM - but custom designed) lens (= they do not illuminate the sensor directly)...

I can't speak to all of their test procedures, but in at least one of them that you point to (ISO testing), this appears to be mistaken for cameras with removable lenses.

From DxO (emphasis mine):
"To measure the camera sensor’s ISO sensitivity, we set up the camera body alone (without a lens) on a stand to receive light from a controlled source. The source is positioned far enough away from the camera sensor to ensure good light uniformity on the sensor plane. We then precisely measure the illuminance received by the sensor with a certified lux-meter."

http://www.dxomark.com/index.php/About/In-depth-measurements/DxOMark-testing-protocols/ISO-sensitivity

[deejjjaaaa]

I can't speak to all of their test procedures, but in at least one of them that you point to (ISO testing), this appears to be mistaken for cameras with removable lenses.

yes, just one (not least of them) and I noted that in my posting

[ripgriffith]

This is what I was referring to, and in their low-light tests, the difference between the A65 and the A77 is more than 10%.  If this is, indeed, within their "margin of error", as one poster suggests, then I would suggest that this test is relatively worthless.

[Robert55]

I'd suggest forgetting about the 'scores' on the first tab. No-one has ever explained what they mean by that low-light score [it certainly is not ISO according to the definitions]
The measurement tab can be very informative. If you look there you'll see A65 and A77 very close, which can well be explained by sample variation between the cameras tested and/or margin of error in the measurements

[Guillermo Luijk]

This is what I was referring to, and in their low-light tests, the difference between the A65 and the A77 is more than 10%.  If this is, indeed, within their "margin of error", as one poster suggests, then I would suggest that this test is relatively worthless.

Forget about the single-number ratings. If you go to the Measurement plots, you will clearly see which is the winner in which aspect.

For me the only really meaningful plot is Dynamic Range. It provides at a glance sensor's performance in terms of noise where noise is actually a problem, i.e. in areas with low RAW exposure (the SNR 18% plot is fairly useless IMO since it measures noise when noise is not a problem for IQ, i.e. in areas with good RAW exposure).

The ISO sensitivity plot is also interesting to know about RAW highlight headroom in your camera if you expose according to the camera metering, but that information is implicit in the other plots.

Finally I have no idea how to interpret Tonal range or Colour sensitivity and translate them to visible results.

Regards

[ejmartin]

As far as determining sensor properties, 18% SNR at low ISO can be used to determine the conversion factor between raw levels and electrons, and therefore saturation capacity in electrons; then the ISO measurement together with this result determines quantum efficiency; finally the DR measurement determines the read noise using the preceding results.  The RGB color conversion numbers give a rough idea of the spectral properties of the color filters.

[bjanes]

As far as determining sensor properties, 18% SNR at low ISO can be used to determine the conversion factor between raw levels and electrons, and therefore saturation capacity in electrons; then the ISO measurement together with this result determines quantum efficiency; finally the DR measurement determines the read noise using the preceding results.  The RGB color conversion numbers give a rough idea of the spectral properties of the color filters.

Emil,

I presume the methodology using 18% SNR at ISO 200 for the Nikon D3s DXO data (as an example) would involve converting the SNR of 40.7 dB to a linear ratio, which gives 108.4. The number of electrons collected would be 108.4^2 or 11749. Since this is for 18% saturation, 100% saturation would be 11749*100/18 or 65272 electrons. The DR at ISO 200 is 11.66 EV or 3236. Since DR = Full Well/RN, the read noise would be 65272/3236 or 20.2

The figures given by Sensogren are 81876 for full well, and 25 for the read noise. I see that the Sensogren method (in which you were involved in the discussion on DPReview) uses curve fitting for multiple points to determine read noise, shot noise, and PRNU. The above calculations ignore PRNU, which Roger Clark neutralizes by subtracting two frames.

Your comments and possible corrections would be appreciated.

Regards,

Bill