Putting DR into perspective..

[ErikKaffehr]

Hi,

Lot of (DR) Dynamic Range oriented discussions recently. This posting is intended to put DR in perspective.

My experience is that my cameras mostly had decent DR for my needs. One of my observations was that I very seldom needed to resort to HDR to get good images. I also feel that the need of DR is often overrated.

Let's start looking at this image:


Now, lets look at an area of deep shadow, brightened up in Lightroom:


The disc of the sun can be recovered in Lightroom pretty well:


Below is the raw histogram of the full image, something like 10EV of dynamic range in this image:


This is one of the very dark areas. Check the histogram, the pixels have a nice Gaussian distribution. Full well capacity on this sensor is around 60000 e/pixels and readout noise perhaps 2-3 electron charges. The raw data is 14 bit wide so each digital number corresponds to about 4 photons. The red channel is centered about 20 counts corresponding to about 80 photons. So noise should be SQRT(80) 8.9 photons, say  photons corresponding to +/-2 counts. So the histogram should be something like 2Sigma * 2 wide. Well it looks like a bit wider than that, but we still see very little evidence of readout noise.

I guess that most modern cameras would be able to handle this scene pretty well.

Best regards
Erik

[ErikKaffehr]

Hi,

This is another image, with wider dynamic range, first let's look at an HDR exposure (P45+ exposures from 1s to 30s), fused in Lumariver HDR.



The whole luminance range is impressive, perhaps 14 stops


Now, lets look at small detail of the piano:



And also check a small area on the piano cover in RawDigger. The peaks are nice gaussians.


The image below is from a 2.5 s exposure on the Sony Alpha 99:


The histograms on the tiny part of the piano cover still look good, albeit each second channel is empty (due to Sony "lossless" compression?)


Here is the same part of the piano on the P45+, note 1s exposure compared to 2.5s on the SLT99:

The P45+ had about 1 stop less exposure, and here the piano cover got noisy, I guess we can see the effects of readout noise:


Raw images are here:

http://echophoto.dnsalias.net/ekr/Articles/DRArticle/NativeRaws/CF045286.IIQ
http://echophoto.dnsalias.net/ekr/Articles/DRArticle/NativeRaws/20140617_lumariver.dng
http://echophoto.dnsalias.net/ekr/Articles/DRArticle/NativeRaws/_DSC4758.ARW

Just a comment, these images were from a "real world" shooting. The P45+ was exposed at 1s while the SLT99 had a 2.5s exposure. Both exposures were based on camera histogram. The idea is not to demonstrate the difference between the two camera/sensor combination. Lab conditions are fare more appropriate for that kind of comparison.

Also, I have been told that my P45+ is a decent sample.
Best regards
Erik

[BernardLanguillier]

Nice building!

Cheers,
Bernard

[NancyP]

That's pretty interesting. RAW Digger seems to be a learning tool, I will have to look into it. I suppose a lot of the DR talk is centered around the sorts of images where one can't get a bracket set without some movement, backlit runners or birds, waving grain and deep shadow, etc.

Do you ever get posterization with the Sony lossless compression data?

[NancyP]

Consulting Dr. Google, Dr. Google in the house? 

The answer to my posterization question is yes, says the RAW Digger blog:
http://www.rawdigger.com/howtouse/sony-craw-arw2-posterization-detection

[ErikKaffehr]

Hi,

Posterisation, I would say not.

Now, Sony employs two kinds of compression. The first is in essence similar to a gamma curve. At high data numbers they have larger steps. That is basically sound, but it may be that they would have overdone it, I don't think so, and I have not ever seen posterisation in my images.

In addition they have a kind of "delta coding", that can induce artefacts. I don't say I have observed it in my images clearly. I have seen some artefacts, in the very same image posted on this thread, but it is in the wrong direction, I think.

This image digged up by Diglloyd is the best illustration of the artefacts I have seen:


And here is a long article from the "rawdigger site": http://www.rawdigger.com/howtouse/sony-craw-arw2-posterization-detection

My take on the issue is that they apply the tone curve so they can put more than 12 bit of data trough the "Bionz" processor which may be just 12-bit wide. I would thing it is absolutely OK.

The delta compression can clearly yield artefacts. On the Sony Alpha 99 I am using mostly there is RAW and short RAW. I always use the larger file format. It may be I don't have the "Delta" compression. I don't know.

Jim Kasson, a real scientist, has done a lot of research on this issue, perhaps a year ago. It was presented on his blog: http://blog.kasson.com/?p=4838

Best regards
Erik

Ps. I have seen that you googled on the issue while I was posting my response... :-)

Do you ever get posterization with the Sony lossless compression data?

[MarkL]

Maybe 10stops is enough for most scenes but that means absolutely nailing it exposure-wise with zero latitude either way and the shadows will be right near the noise floor. Often though the raw files has some detail, torturing it out in software often leads to the tonal and colour information falling apart (though this has slowly improved over the years.)

You'd have to drag me back to my D700 DR from my D800E kicking and screaming.

[dwswager]

Hi,

Lot of (DR) Dynamic Range oriented discussions recently. This posting is intended to put DR in perspective.

My experience is that my cameras mostly had decent DR for my needs. One of my observations was that I very seldom needed to resort to HDR to get good images. I also feel that the need of DR is often overrated.

Best regards
Erik

While I wholeheartedly agree that most shooting circumstances do not require extreme DR to execute the shot proplerly, if I have to choose between 2 $3000 cameras I'll take the one with more DR and less shadow noise.  While a most situations don't call for extended DR, some do and all situations may benefit from the DR if proper exposure is not selected.

DR is just another performance characteristic to be evaluated in the selection process along with other measures of performance and functionality.

[Jack Hogan]

Good examples, Erik.

[spidermike]

While I wholeheartedly agree that most shooting circumstances do not require extreme DR to execute the shot proplerly, if I have to choose between 2 $3000 cameras I'll take the one with more DR and less shadow noise.  While a most situations don't call for extended DR, some do and all situations may benefit from the DR if proper exposure is not selected.

DR is just another performance characteristic to be evaluated in the selection process along with other measures of performance and functionality.

I think Erik's comments point to the fact that nowadays the need to spend $3,000 in the first place is greatly reduced.

[Paulo Bizarro]

Good examples thanks for posting them.

It is always better to have "larger DR", to cover for any "if" situations that might arise. But for the majority of photographers, "normal DR" is more than enough.

[Bart_van_der_Wolf]

It is always better to have "larger DR", to cover for any "if" situations that might arise. But for the majority of photographers, "normal DR" is more than enough.

Hi,

Yes, that sums it up nicely.

The use of proper technique can help in that process, and sometimes it can be quite simple to handle a problematic situation. Then there can be tools to assist us, and some do a better job than others.

Cheers,
Bart

[Jimbo57]

This discussion is really just one (of many) examples of how the Law of Diminishing Returns applies to camera features.

I normally try to demonstrate this to my advanced students (who are often toying with the idea of moving up the camera ladder towards more "professional" models) by showing how each doubling of camera price leads to geometrically reduced increases in features and capability.

My usual example goes along the lines of:

£250 will buy an entry-level DSLR and kit lens.

That will do 95% of what any enthusiast-level photographer is likely to want to do.

Spend £1000 and you will get a camera that will do 97%

Spend £2000 and you will get a camera that will do 98%

Spend £5000 and you will get a camera that will do 98.5%

.....and so on.

But the caveat that I have to place upon that is that each increment of price/capability takes the photographer farther into the extremes of performance.

And so it is with dynamic range of sensors. The difference between a 12 EV DR sensor and a 14 EV DR sensor might only make a significant difference to the image quality in, say, 5% of the shots that the average enthusiast photographer will take. An increase from 14 to 15 might only add 1% to that.

Exactly the same consideration applies to other advances in camera technology such as AF speed, High-ISO performance, etc.

The good news is that, progressively with each new "generation", the performance of entry-level cameras improves along all of those dimensions, so that the whole equation shifts laterally.

Few photographers "need" to spend silly money keeping at the forefront of those advances in performance - but, of course, most of us do!

[shadowblade]

Hi,

Yes, that sums it up nicely.

The use of proper technique can help in that process, and sometimes it can be quite simple to handle a problematic situation. Then there can be tools to assist us, and some do a better job than others.

Cheers,
Bart

Technique only helps you if the scene fits within the technical limits of the camera (be it DR, ISO or resolution). If a scene has 10 stops of DR and you can't capture it with a 12-stop camera, then it's just poor technique. If the scene has 13 stops, it's not poor technique - just that the scene is beyond the technical capabilities of the camera.

Once the scene you are trying to capture falls outside those limits, no amount of technique will help you.

[Bart_van_der_Wolf]

Technique only helps you if the scene fits within the technical limits of the camera (be it DR, ISO or resolution). If a scene has 10 stops of DR and you can't capture it with a 12-stop camera, then it's just poor technique. If the scene has 13 stops, it's not poor technique - just that the scene is beyond the technical capabilities of the camera.

Hi,

Just to mention one item that most people rarely consider testing, how well is your lens hood dimensioned?

Many people shoot with zoom-lenses (image quality can be excellent), which may already be sensitive to veiling glare due to the many lens groups/elements. The lenshood that comes with it is a compromise which also has to accommodate the wider angle short focal length settings. Not all barrel designs change the depth of the front lens element with focal length. Without proper shading of non-imageforming light, you're lucky if you get 9 stops of dynamic range out of the lens into the camera ...

Once the scene you are trying to capture falls outside those limits, no amount of technique will help you.

My camera allows to automatically shoot 2-bracketed exposures (or 3, or 5 or 7). In the 2-bracket mode it's simple to shoot one ETTR shot exposed for the highlights, and a much (e.g. 8x) longer shot for the shadows (that adds almost 3 stops of DR). Two shots are simple to blend together, even hand-held it's often possible.

One can also shoot multiple frames with the same exposure and use (median) averaging to reduce the noise, but of course multiple-exposures are not the most used technique for moving subjects, but even that's not the end of our possibilities.

There are lots of techniques possible, both at shooting time as well as at post-production time. As DxO have shown, a lot is possible when it comes to high ISO noise reduction, and there are new techniques being developed all the time. We can also use dark frame subtraction to get rid of some of the pattern noise in the shadows. Some post processing techniques even trickle down to photoediting software for the masses.

Cheers,
Bart

[shadowblade]

Hi,

Just to mention one item that most people rarely consider testing, how well is your lens hood dimensioned?

Many people shoot with zoom-lenses (image quality can be excellent), which may already be sensitive to veiling glare due to the many lens groups/elements. The lenshood that comes with it is a compromise which also has to accommodate the wider angle short focal length settings. Not all barrel designs change the depth of the front lens element with focal length. Without proper shading of non-imageforming light, you're lucky if you get 9 stops of dynamic range out of the lens into the camera ...

I wish that were the case! If glare was actually applied evenly, essentially adding a fixed amount of light to every point in the image (which could equal four or five stops in the shadows, but a fraction of a stop in the highlights) it would essentially work as a giant fill flash, reducing the dynamic range of the scene and making it easier to capture. Unfortunately it is not, and doesn't really reduce the DR across the whole frame - merely where the lens flare is. Fortunately, it's usually easy to completely shield it with a well-placed hand forward of the lens but outside the field of view.

My camera allows to automatically shoot 2-bracketed exposures (or 3, or 5 or 7). In the 2-bracket mode it's simple to shoot one ETTR shot exposed for the highlights, and a much (e.g. 8x) longer shot for the shadows (that adds almost 3 stops of DR). Two shots are simple to blend together, even hand-held it's often possible.

Doesn't work when things are moving. In landscape photography, wind is the usual culprit.

One can also shoot multiple frames with the same exposure and use (median) averaging to reduce the noise, but of course multiple-exposures are not the most used technique for moving subjects, but even that's not the end of our possibilities.

I often do that. Functionally, it's the same as halving the ISO - you're collecting twice as many photons by exposing for twice as long, so each photon counts for half as much. It certainly minimises photon shot noise. I'm not sure that it actually increases DR, though, since the read noise is also counted twice.

[NancyP]

Most people who use multiples specifically to reduce shot noise also take multiple dark frames to be able to subtract the read noise. It is only worthwhile if you are dealing with very low numbers of photons in the first place - astrophotography. Shot noise reduction is proportional to the square root of the number of multiples taken. Major PITA. You need it to image faint objects, but I can't imagine any non-astro / non-scientific situation where you would go to that degree of trouble in shooting and processing.

[shadowblade]

Most people who use multiples specifically to reduce shot noise also take multiple dark frames to be able to subtract the read noise. It is only worthwhile if you are dealing with very low numbers of photons in the first place - astrophotography. Shot noise reduction is proportional to the square root of the number of multiples taken. Major PITA. You need it to image faint objects, but I can't imagine any non-astro / non-scientific situation where you would go to that degree of trouble in shooting and processing.

Dark frames only work for removing fixed read noise (including fixed pattern noise), not random read noise.

[Bart_van_der_Wolf]

I wish that were the case! If glare was actually applied evenly, essentially adding a fixed amount of light to every point in the image (which could equal four or five stops in the shadows, but a fraction of a stop in the highlights) it would essentially work as a giant fill flash, reducing the dynamic range of the scene and making it easier to capture. Unfortunately it is not, and doesn't really reduce the DR across the whole frame - merely where the lens flare is.

Veiling glare contributes/adds mostly to the shadows where signal levels are low. Since the glare is a product of intra-lens and inter-lens element/group reflections (aggravated by dust and atmospheric deposits), it is not confined to the regions where light is (besides the lens receives all scene light everywhere on the lens before it is finally focused on the sensor).

Fortunately, it's usually easy to completely shield it with a well-placed hand forward of the lens but outside the field of view.

Some of it, yes, but it would take unwieldly deep petal-shaped lens hoods to really do a good job. Hence the on average mediocre shielding peple use if it's even given proper attention to begin with. I use a different lens hood on my TS-E 24mm II when not using it shifted, or only a little. The EW-88C to which I added flocking material, does a better job, even though it was designed for a different lens. I use a separate (Lee bellows) if I want something deeper, and have a petal shaped design ready for 3D printing if that makes enough of an additional difference.

Doesn't work when things are moving. In landscape photography, wind is the usual culprit.

On the contrary, it works fine in most cases. It's often not the horizon line or other moving features that are contrasted with the brightest parts of the image. Most of the info is in a single shadow exposure shot, and only parts are in the ETTR highlight shot.

I often do that. Functionally, it's the same as halving the ISO - you're collecting twice as many photons by exposing for twice as long, so each photon counts for half as much. It certainly minimises photon shot noise. I'm not sure that it actually increases DR, though, since the read noise is also counted twice.

Yes, photon shot noise gets reduced, but averaging also averaged read noise. It does it so well, that pattern noise will be better visible. That's where improved sensors (and/or black frame subtraction) will shine, that is by absence of pattern noise. The patterns become more noticeable because we humans are good at pattern recognition, even where there are none we see details (like shapes in clouds, or faces in moon rocks).

Cheers,
Bart

[Bart_van_der_Wolf]

Most people who use multiples specifically to reduce shot noise also take multiple dark frames to be able to subtract the read noise. It is only worthwhile if you are dealing with very low numbers of photons in the first place - astrophotography. Shot noise reduction is proportional to the square root of the number of multiples taken. Major PITA. You need it to image faint objects, but I can't imagine any non-astro / non-scientific situation where you would go to that degree of trouble in shooting and processing.

Hi,

A Raw converter like RawTherapee makes it easy. Just point it to a sub-directory with a number of darkframes and it will select and average them if multiples are present, and subtract their average from the Raw lights before demosaicing. It's implementation is not as sophisticated as in dedicated Astro photograhy applications, but then the average photographer has relatively many more photons available, although exposure times are much shorter.

Cheers,
Bart