Is DPreview right about 40D DR at ISO1600? 8.9EV

[Guillermo Luijk]

In the Canon 40D's Dynamic Range test in Dpreview, they claim this camera has got the following DR values at different ISO

(below middle gray, over middle gray, total):
ISO 100   -5.7 EV  3.4 EV  9.1 EV
ISO 200      -5.7 EV  3.4 EV  9.1 EV
ISO 400      -5.5 EV  3.4 EV  8.9 EV
ISO 800      -5.5 EV  3.4 EV  8.9 EV
ISO 1600 -5.5 EV  3.4 EV  8.9 EV
ISO 3200  -4.0 EV  3.4 EV  7.4 EV

Before doing any test I suspected that a reduction of just 0.2EV when going from ISO100 to ISO1600 could bee too optimistic.

I did a DR test at ISO100 here: 40D DR as ISO100 obtaining 9 f-stops which is consistent with Dpreview.

Now I have repeated this test over a 40D RAW file shot at ISO1600 (find it here: canon40d_drtest1600.cr2) with the usual subjective criteria and I got 7 f-stops of DR, nearly 2 less than Dpreview's test.


Original scene (ISO1600, 1/12.3 sec, f/8.0):




Bright corrected just to be able to distinguish the scene (not used in the analysis):




SNR samples:



Looking at the result, I wouldn't say the 8th f-stop (-7EV) is usable, so considering the wood texture in the 7th f-stop (-6EV) quite recognisable I would make a subjective estimation of 7 f-stops.

Reading Dpreview's I don't find clear when they are talking about JPEG or RAW mode. They talk about tone curves in HTP (tone curves in the RAW?).
They also claim "We can make some assumption that this improved shadow range is thanks to the new 14-bit processing pipeline", when it has been quite extensively agreed here that more bits by themselves don't improve DR, it's a matter of better SNR in the deep shadows.

I would appreciate your comments.

[Jonathan Wienke]

If you have a 40D, I would be interested in seeing what DR results you get using my DR test chart. I think DPReview may be confusing the engineering definition of DR with the DR that is useful in photography.

[Guillermo Luijk]

If you have a 40D, I would be interested in seeing what DR results you get using my DR test chart

I wish
But I will do that on my 350D, patience.

[Jonathan Wienke]

Please post the results; I'm interested in seeing how consistent your results are with other test methodologies.

[John Sheehy]

In the Canon 40D's Dynamic Range test in Dpreview, they claim this camera has got the following DR values at different ISO

(below middle gray, over middle gray, total):
ISO 100    -5.7 EV  3.4 EV  9.1 EV
ISO 200    -5.7 EV  3.4 EV  9.1 EV
ISO 400    -5.5 EV  3.4 EV  8.9 EV
ISO 800    -5.5 EV  3.4 EV  8.9 EV
ISO 1600 -5.5 EV  3.4 EV  8.9 EV
ISO 3200  -4.0 EV  3.4 EV  7.4 EV
[a href=\"index.php?act=findpost&pid=161270\"][{POST_SNAPBACK}][/a]

These figures can't possibly be about "the camera".  Canon DSLRs generally have 2.5x the (over a stop more) read noise at ISO 1600 than at ISO 100, and all cameras have 4x the shot noise (2 stops more), yet they show only 0.2 stops difference in shadow footroom?

Clearly DPReview's test is not about the camera, but about a specific style of conversion, and perhaps a weak criterion.

[Guillermo Luijk]

I agree with that idea. And I have another question about DPreview plots, they always plot response curves having in the X axis the f-stop with respect to camera's middle gray, and a Y axis scale ranged 0..255:

Canon 40D curves from DPreview.com



I thought the Y-axis scale was gamma corrected scale, i.e. those level values that we would achieve on a developed image for every linear input data with no further processing (curves...). But I have calculated them in a simple Excel sheet, and the curve is totally different. For each INPUT value in the linear range 0..256 I calculated:
- A: f-stop it falls into according to camera's relative position of the middle gray with respect to saturation:

  EV=LOG(INPUT/256)/LOG(2) + MIDDLE_GRAY_FSTOPS

- B: the gamma corrected output as

  OUTPUT = 256 * (INPUT/256)^(1/GAMMA)


Plotting EV = f(B ) results in:




Am I missing something or DPreview plots data with a strong compression in the highlights so that they get this 'S'-shaped plots? more similar to film response curves.
What do you think? Are they again plotting curves of processed styles where toning curves have been applied?

My experience (through objective tests over a gray card) tells me my digital sensor (350D) is a very linear device up to the higher ~20% of the last f-stop before saturation, so these plot could never be this way DPreview is displaying if the Y-scale is a real gamma value range.

PLEASE HELP

[Guillermo Luijk]

I have a hypothesis for DPreview plots: they are shooting against a B&W step wedge, but captured channels are tri-coloured: R, G and B, so they don't blow at the same time, so we have some kind of "compresion" in luminance when reaching the highlights for this reason.

Moreover, the distribution of light in each channel will not be 100% uniform, in fact it will produce an histogram with some width. Since steps are 1/3 EV apart, one channel can get partially blown in one of the gray rectangles.

All this may have the effect of a progressive highlight compression which produces the response curve to smoothly reach value 255, differing from the math definition of the gamma curve.

Could be?

[Panopeeper]

Guillermo,

the DPreview page you are referring to explains the reason clearly. The graphs are JPEG based. The explanation under "picture style options" says

the various Picture Styles use either one of two tone curves, the first more contrasty curve for Standard, Portrait, Landscape and Monochrome Picture Styles and a slightly flatter curve for Neutral and Faithful Picture Styles. Neither curve delivers more dynamic range and they both clip highlights at the same point

None of the picture styles are flat; what you see is the effect of the applied contrast.  See the raw based curve with "ACR best" below, that is more like your graph. Compare that (contrast -50) with the "default", contrast +25, added the "medium" curve, which is a slight S.

My opinion is, that the DPReview DR evaluation is mostly BSing. For me there is only one way to measure the DR with any comparative value: with the step wedge (like DPReview does), the result analyzed on raw level.

Addendum:

I have a hypothesis for DPreview plots: they are shooting against a B&W step wedge, but captured channels are tri-coloured: R, G and B, so they don't blow at the same time, so we have some kind of "compresion" in luminance when reaching the highlights for this reason

That's the reason I insist on pure raw data.

Moreover, the distribution of light in each channel will not be 100% uniform, in fact it will produce an histogram with some width.

I don't understand this.

[Guillermo Luijk]

Hi Panopeeper, look at this plot:



Some of the graphs are almost a straight line. Do you think they are still picture styles here? or they could be a mix?

[Guillermo Luijk]

"Moreover, the distribution of light in each channel will not be 100% uniform, in fact it will produce an histogram with some width."

I don't understand this.

Yes, I mean: if you are shooting a uniformly lighted surface, your histogram should be very narrow (ideally a delta, in practice noise will add some gaussian deviation, but negligible near saturation). I claim that if you are shooting a surface that is not so uniformly lighted, the histogram will get wider. If so, when you overexpose you will start to blow some pixels when others are still not blown (I am talking of a single channel here). If you average level in all pixels of the image to plot this graph, you will get a compressed value of highlight.

What I point is that to achieve the theoretical perfect curve, we should:
1. Plot the curve for individual channels, not in luminance.
2. Making sure lighting is uniform to get all pixels blown at the same time.

If these conditions don't apply, we may get some highlight compression, i.e. our curve will have some S shape near saturation.

[Panopeeper]

Some of the graphs are almost a straight line. Do you think they are still picture styles here? or they mixed?

Picture styles are a group of setting parameters, nothing more (contast, saturation, sharpness). These parameters control the in camera raw->JPEG conversion. On the 40D you have to turn down fully the sharpness and contrast and set the saturation and color tone to 0.

Btw, the default setting of "Neutral" is medium contrast.

Just in the very end (maybe the last 1/3 f-stop before saturation) sensor seems to slightly loose linearity in these links.

The D200's reds and blues clip at a single point, there is no non-linearity. The greens clip between 3970 and 4010 (what I saw; different copies can behave differently to some degree). That range is non-linear, but it is tiny.

However, the DPReview analysis starts at the "middle point gray". From that point upwards the clipping points (or the end of the numerical range) is not on a stop boundary, and that results in the change at the right end of the curve. It's a byproduct of the evaluation, not of the sensor.

I find this "middle gray" blathering useless confusion of the subject. The camera's dynamic range has nothing to do with middle gray. It has a top point (the clipping point or start of non-linearity), and the low end is determined by what you regard acceptable. That's it.

[Ray]

Interesting! I notice that dpreview are claiming almost 11 stops DR with 40D RAW files, but no guarantee of color accuracy in the highlights.

They talk eleswhere about improved shadow noise but as far as I understand this is a user selectable NR which apparently does not reduce resolution.

According to Bob Atkins review of the 40D, without this NR feature on, the 40D has very marginally more shadow noise than the 20D. With NR switched on, the 40D exhibits marginally less noise than the 20D.

Perhaps dpreview are getting their incredible DR result at ISO 1600 by swithing on both NR and 'Highlight Tone Priority". The increased noise in the shadows, resulting from HTP is cancelled by the NR, leaving just the benefits of HTP in the highlights.

[Panopeeper]

if you are shooting a uniformly lighted surface, your histogram should be very narrow (ideally a delta, in practice noise will add some gaussian deviation, but negligible near saturation

The shot of the Stouffer wedge contains black and blown whites. The strips are uniform (well, more or less), but the entire image is not.

Here are the histograms of a Stouffer wedge shot (the much dark comes from the surrounding of the wedge, which is blended somehow in order to prevent light leaks).

[Guillermo Luijk]

The D200's reds and blues clip at a single point, there is no non-linearity. The greens clip between 3970 and 4010 (what I saw; different copies can behave differently to some degree). That range is non-linear, but it is tiny.

However, the DPReview analysis starts at the "middle point gray". From that point upwards the clipping points (or the end of the numerical range) is not on a stop boundary, and that results in the change at the right end of the curve. It's a byproduct of the evaluation, not of the sensor.

I find this "middle gray" blathering useless confusion of the subject. The camera's dynamic range has nothing to do with middle gray. It has a top point (the clipping point or start of non-linearity), and the low end is determined by what you regard acceptable. That's it.

Hi Panopeeper, I am really glad to read you, this is EXACTLY what I am trying to explain in other forum, and I find the (intellectual) opposition of some experienced film photographers.

Today I wanted to test my 350D linearity as precisely as possible. I shot my wall using a 300mm focusing to oo, and analysed 39 shots 1/3EV apart. I took -5EV and +5EV from camera's middle gray (just to take the same reference as in DPreview plots).

I analysed each channel independently not to have any compression issue in the highlights due to partial channel clipping. I also analysed only the central piece of all my shots to have a thin histogram unlikely to clip partially (picked a 250000 pixels centred square patch), and managed to plot this response curve for the G channel (X-axis is EV ref. to middle gray, Y-axis is RGB gamma=2.2):

Canon 350D response curve - G channel



It fits the theoretical expected curve perfectly. It's real RAW data from the camera, but looks like an Excel calculation.

[Panopeeper]

I shot my wall using a 300mm focusing to oo, and analysed 39 shots 1/3EV apart. I took -5EV and +5EV from camera's middle gray (just to take the same reference as in DPreview plots)

Great. With this test you verified the shutter operation of your camera and the aperture operation of the lens (except if wide open).

Be assured, that if you would find anything non-linear, that would be the result of the shutter or aperture mechanism. The sensor can't be non-linear, except the around the clipping point (there is no point to talk about the linearity of the noise).

Download the layered TIF from http://www.panopeeper.com/Demo/NikonD200_clipping.tif and look at the sequence of layers, you will see how the non-linearity looks.

this is EXACTLY what I am trying to explain in other forum, and I find the (intelectual) opposition of many experienced film photographers

You mean like this mixing up the metering with the DR?

-----------

Guillermo, in which time zone are you living? Are you suffering under insomnia?

[bjanes]

Hi Panopeeper, I am really glad to read you, this is EXACTLY what I am trying to explain in other forum, and I find the (intellectual) opposition of some experienced film photographers.

Today I wanted to test my 350D linearity as precisely as possible. I shot my wall using a 300mm focusing to oo, and analysed 39 shots 1/3EV apart. I took -5EV and +5EV from camera's middle gray (just to take the same reference as in DPreview plots).

I analysed each channel independently not to have any compression issue in the highlights due to partial channel clipping. I also analysed only the central piece of all my shots to have a thin histogram unlikely to clip partially (picked a 250000 pixels centred square patch), and managed to plot this response curve for the G channel (X-axis is EV ref. to middle gray, Y-axis is RGB gamma=2.2):
[a href=\"index.php?act=findpost&pid=161582\"][{POST_SNAPBACK}][/a]

I agree with Panopeeper that the DPreviw habit of plotting their test results about "mid gray" is meaningless with digital, where the tonal response is linear from shadows to highlights. The midpoint is an artificial distinction dating back to film.

It helps to use a consistent format for the axes. F/stops are log notation and the pixel values should also be log. This is the standard way to plot characteristic curves. Here is an actual Excel plot showing sRGB (which has a linear component in the shadows) and "simplified RGB", which is a pure gamma conversion. Log-linear and log-log plots are shown.

[attachment=4314:attachment]
[attachment=4315:attachment]

And here is an actual step wedge photograph taken with the Nikon D200 showing in camera JPEG and ACR rendering with default settings. Both apply an S-curve and clip the shadows, the default ACR more than the in camera JPG.

[attachment=4316:attachment]
[attachment=4317:attachment]

[Guillermo Luijk]

It seems is always the same digital vs film pre-concepts discussion in the forums Panopeeper. Totally clarifying plots Bill.
It's always a pleasure to post in this forum with people like you two, thank you.

Best regards.

PS: BTW Bill I borrowed your plots to show in another forum, hope you don't mind: OD forum DR discussion.

[Guillermo Luijk]

Guillermo, in which time zone are you living? Are you suffering under insomnia?

GMT +1. Yes, but it's linear insomnia no worry  

[Guillermo Luijk]

To write an article (FOTOGRAFíA ANALÓGICA vs FOTOGRAFÍA DIGITAL, Spanish), I have plotted the complete RGB sensor curve response of my 350D. The plot is made of 37 shots, 6 f-stops both sides of the camera metering point.

The linearity is fine except for one point in the highlights of the B channel, which seems to provide a slightly higher than expected value once the R and G channels totally burnt.

Characteristic Log-Gamma curve (pure gamma=2.2):



Characteristic Log-Log curve:




Any explanations? the only things I can think of are:

1. In the way sensor electronics are designed, B was affected by the severely blown R and G cells surrounding it.
2. Some shutter speed error (strange just in that shot).
3. This is the way my sensor behaves in the B photocells highlights and we can't change that.

[ejmartin]

Measurements of 40D sensor properties may be found at

http://theory.uchicago.edu/~ejm/pix/20d/po...D300_40D_tests/

It is easy enough to use measurements of sensor read noise and gain to construct a SNR plot from which you can determine DR according to your favorite criterion.  Specifically,

SNR = S / sqrt[ R^2 + S/g]

where S is the signal level in raw levels (ADU), R is the read noise in ADU for a given ISO, and g is the gain (electrons/ADU) for that ISO.  For the 40D this comes out as follows:



The various curves from top to bottom are ISO 100,200,400,800, and 1600.  The horizontal axis is exposure in raw levels, from 1=2^0 to 16384=2^14, in log base two units, ie stops.  So eg 5 along the horizontal axis is raw level 2^5=32 above the raw blackpoint which Canon sets at 1024 ADU in the 40D.  Note also that due to this blackpoint, the maximum raw level ~16200 is ~15100 above the blackpoint, or 13.9 in powers of two (except ISO100, which saturates at about  12800 above the blackpoint, or 13.6 stops).  The vertical axis is SNR, again in powers of two or stops; the zero level is set to SNR=1.

Qualitatively, the knee in each curve represents the point above which noise is limited by photon statistics, and below which noise becomes more and more dominated by sensor read noise.

So, if your criterion is the range of EV over which SNR>1, that's the range of EV along the horizontal axis for which the curve is positive.  For instance, 11.1 stops for ISO100 (from 2.5 to 13.6), on down to 9.2 stops for ISO 1600 -- roughly comparable to the engineering definition of DR.  If you want a more stringent criterion, say SNR > 4=2^2, then that's the range above 2 on the vertical axis; this gives a DR of about 9 stops at ISO 100, etc. Direct sensor measurements are not "useless" as the first response in this thread seemed to imply; they contain all the data needed to reconstruct the camera's noise profile and signal-to-noise ratio at any exposure level.