Nikon D7000 Dynamic Range

[BernardLanguillier]

"Optimal" tonemapping seems to be closely supported by theory, just like you suggest, yet many photographers are strongly against it. Using whatever nonlinear, non-controllable mapping inherit in analog film seems to be favored amongs many photographers. Why is it so? Only history/habits, or some technical reason that I have missed?

It is so because we all do love the security provided by negative films.

In the digital world, we have gotten used to having vivid colors, sharp details,... all the attributes we used to like with slides films together with the impression that we have the DR of negatives... except we don't since a mistake with exposure can be impossible to recover.

Cheers,
Bernard

[PierreVandevenne]

Fair enough! I stand corrected (provisionally). The ND filter is a good analogy, but I don't understand why such increase in exposure to the light before it enters the lens should not reduce the proportion of shot noise in the detected signal to some degree.

Get rid of all the photographic analogies the term "shot noise" evokes. The term "shot" here initially came from... a gunshot analogy.
See it as a dry description of the behaviour of the measuring/sampling process.

You can do many intersting things with "shot noise" - that one is, I believe, one of the best teaching experiment on the topic (although it could probably be done with other tools nowadays)

http://www.math.temple.edu/~cmartoff/teaching/ph4796_10/shot_noise/AJP000554.pdf

and even more exotic stuff such as this one

http://europa.agu.org/?view=article&uri=/journals/gl/GL010i001p00005.xml

On a personal basis, I confess that one of the big shocks of my life was to discover that knowing the characteritics of "noise" could yield useful results . Something that I saw as mere measurement pain and trash suddenly became very beautiful.

[Ray]

Get rid of all the photographic analogies the term "shot noise" evokes. The term "shot" here initially came from... a gunshot analogy.
See it as a dry description of the behaviour of the measuring/sampling process.

You can do many intersting things with "shot noise" - that one is, I believe, one of the best teaching experiment on the topic (although it could probably be done with other tools nowadays)

http://www.math.temple.edu/~cmartoff/teaching/ph4796_10/shot_noise/AJP000554.pdf

and even more exotic stuff such as this one

http://europa.agu.org/?view=article&uri=/journals/gl/GL010i001p00005.xml

On a personal basis, I confess that one of the big shocks of my life was to discover that knowing the characteritics of "noise" could yield useful results . Something that I saw as mere measurement pain and trash suddenly became very beautiful.

Pierre,
Thanks for the links.I'll study them when I have time.

On relfection, I've decided that my speculations on the amount of shot noise existing in the light before it reaches the sensor are philosophically unsound.

We can never be certain about the existence of anything prior to its detection. We can speculate upon its existence and draw inferences about its character, but reality is in the detection.

Therefore I have to concede the point that shot noise cannot exist prior to the detection of the photons. It is nonsense to try to imagine that it might. (Hope I haven't misled anyone   ).

To get back to the purpose of the thread, which was an attempt to find out if the purported increased DR of the D7000 is real and significant, I have now received my D7000 with 24-105/F4 lens, and have made some preliminary comparisons with my 15mp Canon 50D.

The message from some of the more technically-minded members of the forum was that such claimed increases in DR for the D7000, by DXOMark, would be irrelevant because shot noise is already the dominant noise at the extreme end of the DR range in current DSLRs.

The results of my preliminary tests indicate that such a view is incorrect, false, untrue, and plain wrong, to put it mildly.

Of course, I can't speak for the characteristics of every DSLR camera. I can test only the cameras I own. My latest Canon camera is the 50D, and that's the camera the D7000 will replace (at least much of the time).

The Canon D60 might be very marginally better than the 50D in some respects, but so marginally better it's of no significance. For example, the maximum DR of the 50D is 11.4 EV, and the maximum DR of the 60D is 11.5 EV (according to DXOMark). An irrelevant difference.

Comparing the 50D with the D7000, DXO imply in their graphs that the D7000 should have at least similar image quality in the 13th stop as the D50 in the 11th stop. To be precise, the D7000 maximum DR is 13.87 EV and that for the 50D 11.4 EV.

If I ever take the trouble to address this issue more precisely and repeat the test, I'll get some results for the D7000 at 13.67 stops, and for the 50D at 11.33 stops, but for now 13 stops compared with 11 stops should be sufficient for a fair idea.

You should be able to see that the D7000, at 2 stops less exposure than the 50D, actually provides slightly greater detail than the 50D shot at 2 stops more exposure.

This is a remarkable acheivement for Nikon, and a remarkable acheivement for the accuracy and practical relevance of DXO testing procedures.

I'm not going to show you comparisons between the D7000 10th stop and the 50D 10th stop here, but believe me, one is of usable quality and the other ain't.

[Ray]

I've got no idea why this image I've just posted above does not appear in my Win7 Inernet Explorer browser when I try to view it.

What's going on?

[hjulenissen]

Google chrome reports it as 0KB when I download it.

-h

[hjulenissen]

It is so because we all do love the security provided by negative films.

In the digital world, we have gotten used to having vivid colors, sharp details,... all the attributes we used to like with slides films together with the impression that we have the DR of negatives... except we don't since a mistake with exposure can be impossible to recover.

Cheers,
Bernard

Not trying to turn this into a "film vs digital", but:
If a given imaging technology can produce discernible details from a large window of dynamic range (lets say 14 stops), but has to store it within a format that has limited SNR (lets say 100:1, 8 bits or whatever), there are limits to what you can recover (apply a new non-linear transform) before noise will be apparent.

I think this applies to film as well as JPEG files.

If some camera did in-camera processing of raw files, that would be similar to what film-cameras does to the film (or how the film inherently works). So lets say that controlled by camera parameters such as ISO, a spatial lowpass filter (or median, bilateral, whatever) was used for the dark range of the pixels, smoothing out noise and making (hopefully) real detail more apparent against the noise, although smeared. Would that be a higher DR camera? Higher DR as measured by DXO, higher DR as visually inspected by its users?

-k

[Ray]

Google chrome reports it as 0KB when I download it.

-h

I believe it's now fixed. Must have been caused by a quantum fluctuation.

[Ray]

Of course, those who are skeptical might well wonder if the first stop was an equal ETTR for both cameras. Perhaps the D7000 received an effectively greater exposure than the 50D, which might partly explain the 2-stop difference in DR.

To allay any such concerns that critical readers may have, I show here the ACR window for each RAW file at the maximum exposure of 4 seconds, showing the results of the auto feature which applies a -0.8 EC adjustment to the 50D file and a -0.7 EC adjustment to the D7000 file.

The target was printed on Epson Enhanced Matte, and the RGB values of the white border of the paper, at approx. the same position in each image, are shown to the left of the histograms.

I think we can reasonably state that the 50D does not have an exposure disadvantage.

The 13th stop for the D7000 is 1/1,000th of a sec. The 11th stop for the 50D is 1/250th.

[bjanes]

If some camera did in-camera processing of raw files, that would be similar to what film-cameras does to the film (or how the film inherently works). So lets say that controlled by camera parameters such as ISO, a spatial lowpass filter (or median, bilateral, whatever) was used for the dark range of the pixels, smoothing out noise and making (hopefully) real detail more apparent against the noise, although smeared. Would that be a higher DR camera? Higher DR as measured by DXO, higher DR as visually inspected by its users?

One way to check for filtering of the raw data is to perform a Fourier analysis as Guillermo Liujk did in an analysis for the Pentax camera that uses the same Sony sensor as does the Nikon D7000. He found no evidence of any filtration.

Regards,

Billo

[Ray]

Not trying to turn this into a "film vs digital", but:
If a given imaging technology can produce discernible details from a large window of dynamic range (lets say 14 stops), but has to store it within a format that has limited SNR (lets say 100:1, 8 bits or whatever), there are limits to what you can recover (apply a new non-linear transform) before noise will be apparent.

I think this applies to film as well as JPEG files.

If some camera did in-camera processing of raw files, that would be similar to what film-cameras does to the film (or how the film inherently works). So lets say that controlled by camera parameters such as ISO, a spatial lowpass filter (or median, bilateral, whatever) was used for the dark range of the pixels, smoothing out noise and making (hopefully) real detail more apparent against the noise, although smeared. Would that be a higher DR camera? Higher DR as measured by DXO, higher DR as visually inspected by its users?

-k

As far as I understand, the limits to the DR that can be recorded on a linear device such as a digital camera's sensor, are governed by factors such as the level of noise in the camera, including shot noise within the detection system as well as shot noise inherent in the received signal, the size of the pixels (full-well capacity), and the number of 'bits' in the camera's internal computer processor such as its analog-to-digital converter and other amplifiers.

I believe the 14-bit system now used in current DSLRs is theoretically capable of recording 2 stops more DR in the shadows than the older 12-bit systems, provided camera noise levels are sufficiently low.

Having recorded that wide dynamic range (say 14 stops), whether using a digital camera or some special high-DR film, the latent image in the RAW file, or on the exposed film, has to be developed before the image can be seen.

Those who are familiar with wet-darkroom practices would choose their developer carefully, in accordance with film type, in order to extract maximum detail from the shadows. When making a print from the developed film, they would also choose the paper-type carefully if they wanted to bring out detail in the shadows. For example, a high contrast paper would tend to clip the shadows to black and reduce the apparent DR. A low contrast paper would be preferred, plus the application of some degree of dodging and burning during the exposure of the photographic paper under the enlarger.

The RAW file from a digital camera is analagous to an exposed, but undeveloped, piece of film. Like the exposed film it has to be 'developed' before the image can be seen, but developed in the digital domain, in Lightroom or ACR etc.

In order to 'see' any detail or information in the shadows, the shadows need to be lifted. The dark areas need to be lightened and sometimes the light areas need to be darkened. Tone curves need to be applied.

Once the image has been processed in such a manner so that the image on monitor closely simulates what the eye saw at the time the shot was taken, I don't see any problem in converting the 16 bit file to 8 bits and compressing to jpeg. Any loss in quality will be marginal, provided the best quality of compression is used. The higher bit levels are useful for processing purposes, but 8 bits per color is considered sufficient for viewing purposes.

However, in the above description, I have assumed that the goal is to make an accurate representation of what the eye saw when the shot was taken. This is not always the goal of photography. One can attempt to be creative and try to improve upon reality.

[Ray]

Okay! What about the 9th stop in the DR range? 9 stops of DR was considered a useful range for color negative film. It gave a reasonable leeway for overexposure.

Even BJL would surely not turn up his nose at detail in the 9th stop. Is such detail usable on a modern 35mm cropped-format DSLR?

Well, on the D7000 is certainly appears to be. But I'm not sure about the Canon 50D.  (The 60D would have very marginally better DR, tonal range & color sensitivity than the 50D at base ISO, but not above base ISO.)

These files are crops of the centre of Jonathan Wienke's DR Test Target. Their purpose is to show the 'real' value of an extended dynamic range. Forget noise! That's just one parameter. Can you see the detail despite the noise?  That's the true test.

I'm showing here  a centre crop of the test target, but the LL site is telling me that the file is too big, although it is definitely under 4MB after de-compression, so I'll show you the D7000 crop in this thread, and the 50D crop in the next thread.

These 9th stops are at an exposure of 1/60th sec, as opposed to the full ETTR exposure of 4 seconds, of the same target under the same lighting conditions.

[Ray]

This the Canon 50D centre crop of the same target at the same exposure. Here's crossing my fingers that the noise will not be too great for the system to handle  .

[Ray]

Weird! The colors in the D50 crop seem much more muted than they do on my monitor prior to download. Ill try a reduced size comparison of the two together.

That's closer. What's going on? Some more quantum fluctuations perhaps.

I should mention that no noise reduction or sharpening has been applied to these images. They've been converted in ACR 6.3 beta using EC, brightness and contrast level, to equalize the appearance of the images as closely as possible.

[Ray]

Have failed to download the 50D crop after several attempts. I get an error message which suggests that my download speed might be too slow. I'm on ADSL2.

[hjulenissen]

Ray:
We fully agree that the specs needed for capture are different from those needed for presentation. When "developing" the captured data into a human-observable image, all kinds of non-linear/linear transforms can be carried out to make the end-result more visually similar to the original scene, or just to make it subjectively pleasing.

My point was that it may (and I am just speculating here) make more sense to talk about DR and SNR at different input levels in one go. Having 20 stops of sensor DR may not help that much if most of that range is very low SNR? Having 14 stops of discernible information (film) may not make that much sense if the non-linear process needed to even out the inherent non-linearity makes it noisy to look at.

So what I was hoping for was to overlaid curves, with input level on the x-axis, and output level and noise level plottet. The "window" between those two should define the signal-dependent DR at a range of input levels, right?

I am struggling with shot-noise being signal dependent, and not additive. Perhaps my post is just jibberish.

-h

[JR]

Hi, Ray! I like your D7000 Dynamic Range post.  I share your conclusions and wish Canon will come up with better DR sensors. Especially the shadow DR should be improved. Highlights are good I think. With HTP on you get about 4.5 stops of DR in the highlights which is as good as it gets.

It seems like many do not care about DR and some even think you are crazy when you empathize this. I don´t. I have invested time and money in my Canon gear and know how to get the best out of it but I still think they need to work on their sensors. I don´t know if they have "fallen behind" or if they just don´t put any effort in to improving DR? I believe it is a marketing issue. They have been focused on video for a long time now. The average mass consumer don´t care about better shadow DR. They care about video and features. Canon give them what they want. I think that is the main reason Canon is lagging behind with respect to DR in the shadows. Whether Canon think they need to improve DR or not I don´t know but my guess would be that they don´t care about it at the moment. Only if they see a huge demand for better DR they will do something about it, thus I doubt we will see 14 stops of DR in any Canon camera for at least 2-3 years if at all. It is only a guess of course but it is based upon what we have seen from Canon the past 5-6 years.

I have tried Nikon and there is no doubt that they have better shadow DR. I can easily lift the shadows in post and do not have to worry about banding or posterization. And I am not taking about extreme editing. Only moderate editing. The D7000 is doing exceptionally well in this regard. It deserves a gold medal 

I am not going to trade in my Canon gear. It is too expensive to change brand and I am not sure if it is worth it. Not at the moment.

[PierreVandevenne]

It seems like many do not care about DR and some even think you are crazy when you empathize this. I don´t. I have invested time and money in my Canon gear and know how to get the best out of it but I still think they need to work on their sensors.
I am not going to trade in my Canon gear. It is too expensive to change brand and I am not sure if it is worth it. Not at the moment.

It really seems Sony sensors have the edge at this point. That's an undeniable fact. Reading their technical litterature, what they do at the individual pixel is quite amazing. I was particularly impressed by their A/D converter (fig 3 in this page http://www.sony.net/SonyInfo/technology/technology/theme/cmos_02.html). Just as Nikon was way behind Canon before they started using Sony sensors, Canon is definitely behind Sony and Pentax now (note: I am mostly a Canon owner myself). I am sure Canon would like to catch up, no so sure they can match Sony on sensors though. The K5 images are _really_ impressive imho.

But as you point out, focusing on other features is probably a good strategy as well: I've heard that a computer manufacturer has been quite successful in that respect, despite significantly underperforming others in terms of processing power ;-)

[Ray]

Hi, Ray! I like your D7000 Dynamic Range post.  I share your conclusions and wish Canon will come up with better DR sensors. Especially the shadow DR should be improved. Highlights are good I think. With HTP on you get about 4.5 stops of DR in the highlights which is as good as it gets.

Hi JR! Thanks! I'm glad at least someone appreciates all the hard work I'm doing. 

Extending DR means lowering noise and improving detail in the shadows. Noise is never a problem in the highlights, although shot noise can sometimes be a problem in the mid-tones.

With HTP on you get about 4.5 stops of DR in the highlights which is as good as it gets.

You probably know already that HTP surreptitiously drops the ISO setting down a stop, so with HTP activated, you are effectively underexposing your image in relation to your chosen ISO setting by one stop. The consequences are more noise in the shadows. Of course, it's usually preferrable have a bit of additional noise in the shadows than blow the highlights, which is why some folks find HTP useful. I personally prefer to auto-bracket exposure.

The D7000 would have no need for HTP when shooting RAW. Whatever ISO you're at, if you are at all worried about losing highlight detail, just drop down to base ISO. I imagine, if one is at ISO 6400 shooting a night scene with artificial light, then dropping down to ISO 100 would probably allow one see the shape, hue and color of the semi-molten filament through the transparent glass of a clear tungsten light bulb. But I haven't tried that yet. 

I have tried Nikon and there is no doubt that they have better shadow DR. I can easily lift the shadows in post and do not have to worry about banding or posterization. And I am not taking about extreme editing. Only moderate editing. The D7000 is doing exceptionally well in this regard. It deserves a gold medal

I am not going to trade in my Canon gear. It is too expensive to change brand and I am not sure if it is worth it. Not at the moment.

I agree, it's probematical carrying two systems. However, I haven't made a complete switch from Canon to Nikon because I can't find an improvement on the Canon 100-400 IS zoom. The Nikkor 80-400 VR will no doubt be staunchly defended by Nikon fanboys, but from reports that I've come across, the Canon has better performance at 400mm.

Lens quality is important to me, within reason. It has to be balanced with flexibility of use, and weight and cost considerations. The Canon 100-400 has that balance right, but I wish Canon would produce an upgrade with a razor sharp performance at F5.6, at 400mm.

I take most of my photos when travelling. If I were to set off tomorrow for Nepal, I'd be carrying 3 cameras instead of the usual 2, but still 3 lenses. I'd be carrying the Nikon D700 & D7000, the Canon 50D, the Nikkor 14-24/2.8, the Nikkor 24-120/F4 and the Canon 100-400. The total weight would be slightly greater, not just because of the additional camera, the 50D, which is about an extra 800gms, but also the additional 0.5Kg of the D700 with 14-24/2.8 lens, which replaces my 5D with Sigma 15-30. Total additional weight would be of the order of 1.3Kg.

[Ray]

Ray:
We fully agree that the specs needed for capture are different from those needed for presentation. When "developing" the captured data into a human-observable image, all kinds of non-linear/linear transforms can be carried out to make the end-result more visually similar to the original scene, or just to make it subjectively pleasing. Having 20 stops of sensor DR may not help that much if most of that range is very low SNR? Having 14 stops of discernible information (film) may not make that much sense if the non-linear process needed to even out the inherent non-linearity makes it noisy to look at.
 

As far as I understand, it's those lower stops in the DR range that are the problem regarding noise obscuring detail. In reality, one does not expect to see full detail in a shadow, but one does expect to see some detail if detail is present. One doesn't expect, when viewing a real-world scene, to suddenly see an area of total blackness in a shadow, whether such a shadow is in the corner of one's living room or at the foot of a waterfall in the rain forest, unless of course the corner is in an area, for example, which has been painted in matte black and therefore should be black.

One certainly doesn't expect to see speckled color and streaks of banding when viewing the shadows in a real-world scene with one's naked eyes.

So what I was hoping for was to overlaid curves, with input level on the x-axis, and output level and noise level plottet. The "window" between those two should define the signal-dependent DR at a range of input levels, right?

There are plenty of graphs at DXOMark and explanations of noise characteristics. It's a fairly safe bet that the lowest 2 or 3 stops at the extreme end of the DR range of any camera that DXOMark have tested, have very limited use in normal photography. Too noisy by far.

However, signal-to-noise in the shadows improves more dramatically with increases in exposure, than it does in the midtones. A doubling of exposure will increase the SNR in the shadows by about 6dB, but the same increase of exposure will increase SNR in the midtones by only 3dB.

When you underexpose your shot, ie, fail to get an ETTR, it's the shadows that suffer the most.

[JR]

Ray, just a comment on HTP as it concerns DR

I am familiar with HTP. I know a lot of people don´t like the feature, they say they get to much noise in the shadows. I don´t agree with that. I like to push all my equipment to their limits, and my personal findings are completely in line with the findings of Bob Atkins.

I find that HTP gives m a little more than one stop of DR in the highlights with no extra noise in the shadows. When I say no extra noise I am looking at it from a practical point of view. I know it can be measured. I know I can see it if I enlarge the image 400% and study the shadows closely but I can´t see it in an A3+ print. I have tried and it is not possible. From a practical point of view it is a non issue for me. I therefore leave HTP on all the time. It gives med extra highlight detail at no cost.

Here´s an image from Bob´s site. As you can see you have to push pretty hard and look pretty hard to see the extra shadow noise produced by HTP. It is as Bob said, "there is an increase in the shadow noise if you look closely enough."


Credit to Bob Atkins: http://www.bobatkins.com/photography/digital/canon_eos_40D_review_6.html#highlight2

I therefore think HTP is a "nice" feature that gives you that little extra DR in the highlights. Not by much, I know, but you take what you get from Canon      I know I can "push and pull" in several ways to extend the DR of the camera, but HTP saves me time in post. I also think some people who complain about the usefulness of HTP are more busy looking at scientific measurements than actual use.

Despite my "praises" of Canon´s HTP they still have to much shadow noise. Just to be clear on that.   I think it is good that Nikon, Pentax and Sony ( particularly Nikon ) are coming up with these great sensors and are putting pressure on Canon.