dynamic range and exposure

[jwarthman]

Now that sensor location gets translated or processed to represent a pixel which holds an RGB value - if that value is not based on 12bits for each color (=36bits), we lose some detail, 20bits if we only use 16bits!

Victor,
In general I think you're correct - but the way you put it could be misleading. You say that an RGB value must be based on 12 bits for each color = 36 bits. I would add "at least" 12 bits per color.

But then you say we lose some detail ... if we only use 16 bits. Again, true - but probably off-base. When we talk about 16-bit color, we're generally referring to 16 bits *per channel* - or 48 bits per pixel. Representing an RGB value in 16 bits would result in, perhaps, 5 bits per color, with one bit wasted.

In practice, photographers are generally using 24 bits (8 per channel) or 48 bits (16 per channel) to represent each RGB pixel. If each photosite captures 12 bits of data, you can see that 24-bit color loses some data, while 48-bit color does not.

Hope This Helps!

-- Jim

[BJL]

Do tell me what improvement you would prefer?

To start with, I want something that you, I and many others in this forum probably agree on: sensors that can handle a greater dynamic range and S/N ratio than currently: even a very high end product like the Kodak 16MP sensor in their MF backs has a S/N ratio of about 4000:1 or about 12 stops from blown highlights down to pure noise, and so distinctly less than 12 stops if you stay far enough above noise to get good looking shadow details. That is currently the one area I know of where 35mm film (print film at least) most clearly offers more than even high end digital, and dealing with high contrast subjects seems to be the one area where digital photographers seem to have to work hardest, with techniques like blending two images.

There is a lot of room for improvement (about 4 stops, or up to about 65,000:1 luminance range) before the capabilities of 16-bits per channel format are exhausted; that is one reason why talk of 512 bits does not interest me.


After that, my second priority is a "smaller number": bringing the price of that envisioned high end quality down to my price range!

[dlashier]

Yes, broad statements like "it's full of so much mis-information ...that I don't think it even warrants discussion" are useless to our ongoing conversation. If you disagree, explain why.

Dan, Michael has graciously retitled the article and removed or edited the references to which I objected. My problem was tying it to dynamic range and while there may be a tenuous connection it isn't the primary possible benefit of the technique. Note the sort of statements that appeared at dpr: "Michael Reichmann stated in his article that if you don't use the right hand (highest) 1/5 of the histogram, you lose 1/2 the dynamic range." which I'm sure will make even Michael smile a bit.  ::

Dynamic range and bit depth are two of the most mis-understood and confused concepts in digital photography and although I'm sure Michael understands them, his loose use of the terminology only sows seeds for ill-advice. Now if Michael had said 'minimize your noise levels', 'maximize your adjustability' or 'maximize your gradient' or such I wouldn't have had any argument, and now that things have been clarified I'm more interested in discussing other ramifications.

- DL

[dlashier]

> It’s my speculation that the histogram is displaying a gamma encoded interpretation of a capture. . .at least for Canons

Jeff, Chuck confirmed this over at RG, and also note that the TRC is involved so the ends are compressed a little.

> So, the question still remains. . .should one expose raw captures like a chrome or a neg?

I shot chrome for years and I think that all that's meant when folk say 'expose like chrome' is that you've got to pay more attention to exposure because processing is less forgiving, and this applies to digital also. But I agree that you can't expose digital exactly like chrome - you've got to pay more attention to highlights because of the hard landing compared to chrome. In my normal outdoor (high dynamic) situation I spot meter almost exclusively on the highlights letting everything else fall where it may, but sometimes checking midtones and shadows. These images rarely require much if any EC. In compressed situations I pay more attention to the midtones.

> Some early digital shooters tended toward underexposing to avoid hilight blooming and blowing out specular detail. . .but the habit of underexposing also leads to a lot of shadow noise.

No doubt, and Michael's article makes a very good case as to why you shouldn't underexpose, I'm not so sure about the case the other way.

> When confronted with such a histogram in PS in 'levels', to keep it simple, what sort of broad adjustments would be recommended? I mention this because Don seems convinced that any 'overexposure' will need at some stage to be corrected and the correction will 'undo' those advantages of having initially more levels to describe the image.

Ray, to me overexposed images need to have values lowered by definition. ;-) As to whether you spread compressed shots and how far are artistic decisions, but for me normally yes. But I spread them in the raw converter so we're still in 16 (ok, 12) bits, left values are well up off the noise floor, plus I'm pushing them down, etc.

- DL

[Ian Lyons]

>Perhaps comparing these two approaches in the field is preferable than endless attempts to anwer the question theoretically.

And WTF do you think some of us have been this past year and half or more!

[victoraberdeen]

Great curves Don, that is what we need to map the histogram and the number of digital sample points to.

My understanding of Gamma is a measure of contrast, and checking myself http://www.wordreference.com/english/definition.asp?en=gamma would confirm my thinking. So I understand how gamma effects the display of an image...

Now I have a question on how the histogram is showing gamma, surly it is just a measure intensity by tone. So if the contrast is there (high to low) it is not representative of the  contrast or gamma in the picture. Did I miss it?

[David Mantripp]

so, we can blow huge sums on a DSLR happily thinking we'll never have to worry about spotting scans again, and that we can take advantage of all the time we saved to do a part time degree in non-linear algebra and rocket science so that we can end up with a photo which looks almost as good as film!

(is there a smilie for tongue-in-cheek anywhere ?)

But seriously, does anybody actually enjoy all this stuff ?

[Ray]

Let me put this a different way, if the sensor has a 12 bit register that means it can only give you a 12 bit value! So the measurement is the intensity of the color it is filtered to - seems simple enough!

How about this as a non-technical explanation! Each photodetector is like a bucket that can hold a certain finite number of photons. At full exposure (highlights), the bucket is full. At close to zero exposure (deep shadows) the bucket is empty. The size of the bucket represents dynamic range. Let's say we have a 10 litre bucket and each litre represents an F stop of dynamic range.

Let's assume we have a completely noise-free system. We're shooting a high contrast scene that has a dynamic range of 10 stops, but we fail to get the histogram as far to the right as possible without overstepping the mark. Result? Something less than 10 stops and loss of detail in the shadows. We're throwing water away. Wasting it.

Now, in practice we don't have noise-free systems. The 10 litre bucket contains 3 litres of sludge. By exposing as far to the right as possible, we're getting as much of the image as possible away from the sludge.

[victoraberdeen]

Yes Dave, I agree that would be a good start.

It is however BJL, not about big numbers it is about having enough detail to show the subjects we photograph. The tones we see in shadows are a curve not a numbererd steps. The only limitation from your list to consider is the eye, many of us got the same story when CD's arrived but musicians knew the difference. Most current display methods fail to show what is on a good kodak/fujichrome image, which in turn fails to show what the photographer wanted. So we should set the objectives high, higher than film which will inturn challange the the display industries!

Victor...

One. Good is the Enemy of Great - Jim Collins http://shopping.yahoo.com/shop....7047419

[Dan Sroka]

Yes, broad statements like "it's full of so much mis-information ...that I don't think it even warrants discussion" are useless to our ongoing conversation. If you disagree, explain why.

[Ray]

That's not to say that I don't see merit in the technique in special situations, but I'd say those situations are the exception rather than the rule for most photographers. And the gains are in reduced shadow noise rather than increased dynamic range.

Seems to me (and I always stand to be corrected) signal-to-noise and dynamic range are inter-dependent. You can't have one without the other - a bit like resolution and contrast.

There always seems to be a certain vagueness in describing dynamic range. Is the true DR of the 1Ds, D60/10D 5, 6 or even 7 f stops? Well, it depends on how much noise you're prepared to accept in the shadows.

If we describe the DR of the 1Ds as being 6 f stops with a noticeable but not objectionable amount of noise in the lowest stop, then clearly, when shooting a scene that has higher than a 6 stop dynamic range (which would apply to most outdoor scenes on a sunny day) one needs to make sure the histogram reaches all the way to the right. If it doesn't, the result will be LESS dynamic range than the camera is capable of. Instead of 6 stops with an acceptable amount of noise in the shadows, we'll have say 5 or 5 1/2 stops with an acceptable amount of noise in the shadows. Or, to look at it another way, we'll still have 6 stops DR, but now with an UNACCEPTABLE degree of noise in the shadows.

Unfortunately, in such circumstance where the DR of the scene is higher than the DR of the camera, using the evaluative metering setting is quite likely to result in blown highlights (at least that's my experience with the D60). Yet the exposure is still in a sense 'correct'. The waterfall might look like a blank sheet of paper, but the person standing next to it looks good. Skin tones are correctly balanced, good detail in the shadows. The picture's perfect, except for the waterfall. The plain fact is, the camera has insufficient dynamic range for the scene.

If I want to retain detail in the waterfall, I have no choice (with a single shot) but to 'underexpose' other elements in the picture. That means I have to work on those underexposed elements in post processing to correct them. Lighten the mid-tones with the levels slider, or whatever.

I'd rather not have to do this because the person standing next to, or in the waterfall might be equally important, but I have to make a choice. In such circumstances, getting the histogram to touch the right side without clipping (or flashing) would seem to be the goal and the best compromise.

But what happens when the dynamic range of the scene is LESS than the camera's capability? The histogram might be the shape of a conical mountain slap in the middle of the range, not even nearly touching either the right side or the left side. Do we then deliberately 'overexpose' so the base of the mountain touches the right of the histogram screen and the gap between the left side of the mountain and the left side of the histogram screen actually widens?

When confronted with such a histogram in PS in 'levels', to keep it simple, what sort of broad adjustments would be recommended? Should one bring the left slider to the base of the left side of the mountain and then move the middle slider leftwards to lighten the mid-tones, or should one leave the left slider as it is and concentrate on the middle slider?

I mention this because Don seems convinced that any 'overexposure' will need at some stage to be corrected and the correction will 'undo' those advantages of having initially more levels to describe the image.

He has a point and it needs to be either corroborated or demolished.

[Schewe]

So, if the histogram is representative of a Canon SDK "default" conversion, I would be inclined to say the histogram is even more usless since it will not be a predictor of either C1 or CR's conversion. . .

If one takes the care to spot meter various tones in the scene, then clearly one can determine the scene luminance range and how best to set the exposure. . .but I suspect that many photographers don't do that. I often don't either because of lot of what I shoot is in the studio where I can control the scene's contrast.

But I still think that Michael's contention that the linear raw capture has the widest tone levels in the upper part of the scene and less in the lower part (as would be the case with a linear 12 bit capture) might explain te problems a lot of shooters who tent toward underexposing have with shadow noise.

and yes, it's Jeff. . .

[jwarthman]

Don,
I'm struggling a bit, myself, with this whole topic. I found the articles on your web site, notably the histogram comparisons of the various raw converters, to be quite interesting. I look forward to the examples you mention.

Enjoy!

-- Jim

[victoraberdeen]

Thank you Don and Jim, It means we keep taking photos...

Don, That was kind of my thinking also. I had only considered gamma as an adjustment for display (CRT's)...

I would be interested to see some real photos, with explanations of how they got them. In my experience getting the exposure right has always been better than pushing to the right! I have even got highlights burning to a complimentary color (magenta is common) see below.

First sample shows the sky going magenta in the top right corner!?



Next is an example of exposing to the left and see how the highlights have blown out! Both on an Fuji S2 pro (RAW).



Finally a film example, so far I have failed to get this much  from a digital camera, I'll keep playing!



Victor

[dlashier]

> sounds crazy but, I meter with my eye and then check with a spot meter.

Doesn't sound crazy at all Victor. The first ten years or more that I shot chrome I usually didn't carry a meter at all so developed a pretty good eye and mental calculator. When I got my first camera with a built-in meter (Canon FT) my exposures actually went downhill for a bit! I use nothing but spot on my 1D. Matrix is just too vague for me.

> I do think the software should do what film would and burn white here! What do y'all think?

Yes. C1 seems to do a better job in this regard but I don't think it'll handle your S2 raws.

- DL

[michael canyes]

Ian,
So this sounds too simple. All I need to do is use a WHITE card (in the same light as the subject) to adjust the histogram so that the spike is all the way to the right, but not touching. Am I missing something?
Thanks,
Michael

[victoraberdeen]

Doh!

Let me put this a different way, if the sensor has a 12 bit register that means it can only give you a 12 bit value! So the measurement is the intensity of the color it is filtered to - seems simple enough! Now that sensor location gets translated or processed to represent a pixel which holds an RGB value - if that value is not based on 12bits for each color (=36bits), we lose some detail, 20bits if we only use 16bits!

Also the value representing RGB surly has a relationship to the color temperature, and luminance?  So lets stop thinking that the Bayer mask is acceptable, that representing the light in 12 bits is acceptable and set a high expectation for the quality of image we require.

[BJL]

Victor, the limitations of the eye were my main point: over one hundred levels within a single f-stop in the deep shadows (and well over a thousand levels in the f-stop around middle gray) is, as fas as I know, vastly finer tonal disinctions than they eye can distinguish: in your audio analogy, it would be comparable to the 500khz frequency response boasted by a few specialist amplifiers, not the quibbling over whether 22kHz is enough. Dwelling on going vastly beyond what is visually relevant in one aspect of performance would, it seem to me, just take resources and market pressure away from making other far more relevant improvements.

[victoraberdeen]

Michael, can you please provide some references for your statements. I have found in the past that such comments are often not supported by the data! Your digital argument is lacking in detail. Now draw the characteristic curve for a sensor and show where the binary values fall, now draw the steps - I think you will smile! This is why we need more numbers on the curve.

Your article is IMHO only suggesting that you over-saturate, or expose for the shadows - yawn.

Victor  

[Schewe]

Well. . .inspite of all the back and forth (particularly aggressive from newcomers I might add), I think Michael’s basic premise is being ignored.

If you expose digital the way you exposed film, you are 1) making a mistake, and 2) failing to fully exploit the full potential of sensors.

You can argue the merits. . .in point of fact there may indeed be confusion since Michael has blended traditional analog nomenclature with digital terms-which to be honest is entirely legit since we’re all struggling to blend analog with digital as well-you can choose to try to grasp the fundamental differences, or not.

The only thing I might argue about is trying to tell anything substantial from looking at a camera’s histogram in the first place.

It’s my speculation that the histogram is displaying a gamma encoded interpretation of a capture. . .at least for Canons, a raw capture and a jpg shot at the same exposure looks like the same histogram to me.

So, if that is indeed the case, you can’t really ever examine the linear capture histogram. I’m assuming here that the histogram function is using Canon’s default jpg gamma conversion for the purpose of determining the histogram information. Too bad really because if you are shooting raw, it would be useful to examine REAL information in the histogram instead of a gamma encoded interpretation. . .particularly if you aren’t using the Canon SDK for conversion (as would be the case using Camera Raw).

So, the question still remains. . .should one expose raw captures like a chrome or a neg? My contention is neither. . .one should learn to expose digital for the chip, your raw linear to gamma encoded conversion and the aesthetics of the image you are trying to create. Which I think is what Michael was trying to get across. . .

I might also take issue with pegging the dynamic range of digital captures at 5 stops. . .I would argue that the usable dynamic range of raw captures with Canons at least (10D and 1Ds) might be more in the 8-9 stop range. . .The BetterLight backs are getting in the 11 stop range. . .yes, one must still be careful of individual channel clipping even if the other channels don’t clip.

Some early digital shooters tended toward underexposing to avoid hilight blooming and blowing out specular detail. . .but the habit of underexposing also leads to a lot of shadow noise.

Personally, I’m still learning to understand exactly how sensors and conversions are effecting the images I try to create. I think it’s useful to examine Michael’s article for the underlying concepts and the impact they have on capture & conversion.