Pages: [1] 2 3 ... 5   Go Down

Author Topic: Shooting to the right and raw conversion  (Read 45815 times)

billmcknight

  • Guest
Shooting to the right and raw conversion
« on: June 21, 2007, 12:53:35 pm »

Conventional wisdom is that for digital cameras it is best to expose so that the right side of the histogram is as far to the right as possible without blowing the highlights.  
However;  the raw converter I use allows exposure compensation of +/- 2 stops.
If I under expose does this mean I can bring out detail in the shadow areas or is this a false move.  Having preached the conventional wisdom I was asked this question during a talk and I could not answer.  Can anyone help.
Logged

digitaldog

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 20630
  • Andrew Rodney
    • http://www.digitaldog.net/
Shooting to the right and raw conversion
« Reply #1 on: June 21, 2007, 01:16:07 pm »

Quote
Conventional wisdom is that for digital cameras it is best to expose so that the right side of the histogram is as far to the right as possible without blowing the highlights.  
However;  the raw converter I use allows exposure compensation of +/- 2 stops.
If I under expose does this mean I can bring out detail in the shadow areas or is this a false move.  Having preached the conventional wisdom I was asked this question during a talk and I could not answer.  Can anyone help.
[{POST_SNAPBACK}][/a]

You want to place as much data to the right so that you end up with the most data in the last stop of the tone curve (shadows). If you have a 12 bit file that can produce 6 stops, the first half of the data is contained in the first stop of exposure data (2048 levels). The last stop has only 64. See:

[a href=\"http://www.ppmag.com/reviews/200612_rodneycm.pdf]http://www.ppmag.com/reviews/200612_rodneycm.pdf[/url]
Logged
http://www.digitaldog.net/
Author "Color Management for Photographers".

Roberto Chaves

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 50
    • http://www.tabi.se
Shooting to the right and raw conversion
« Reply #2 on: June 21, 2007, 01:18:21 pm »

Quote
Conventional wisdom is that for digital cameras it is best to expose so that the right side of the histogram is as far to the right as possible without blowing the highlights.  
However;  the raw converter I use allows exposure compensation of +/- 2 stops.
If I under expose does this mean I can bring out detail in the shadow areas or is this a false move.  Having preached the conventional wisdom I was asked this question during a talk and I could not answer.  Can anyone help.
[a href=\"index.php?act=findpost&pid=124188\"][{POST_SNAPBACK}][/a]

I'm not sure I understand your question. Do you wish to underexpose, with an idea of "exposing to the left" to bring out shadow detail?
If this is the question, then no.
You gain shadow detail by exposing to the right (and loose shutterspeed, not to be forgotten!) and then correcting the exposure in the RAW converter.
Logged
Best regards,
 Roberto Chaves  www.tabi.

bjanes

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3387
Shooting to the right and raw conversion
« Reply #3 on: June 21, 2007, 01:27:59 pm »

Quote
Conventional wisdom is that for digital cameras it is best to expose so that the right side of the histogram is as far to the right as possible without blowing the highlights.  
However;  the raw converter I use allows exposure compensation of +/- 2 stops.
If I under expose does this mean I can bring out detail in the shadow areas or is this a false move.  Having preached the conventional wisdom I was asked this question during a talk and I could not answer.  Can anyone help.
[{POST_SNAPBACK}][/a]

IMHO, conventional wisdom is correct. At base ISO you might get away with two stops underexposure, since it would be like shooting ISO 400 rather than the base 100. Signal to noise would be worse, especially in the shadows and tonality could suffer. At high ISO, noise would probably be objectionable, depending on the camera.

Actually noise performance is largely determined by the amount of exposure, not the camera ISO. Above unity gain of the camera (which varies from 800 -1600 with most DLSRs--see [a href=\"http://www.clarkvision.com/imagedetail/digital.sensor.performance.summary/]Roger Clark[/url]), if you are strapped for exposure by shutter speed or f/stop necessities, it does not help to raise the camera ISO any further than that of unity gain, and you can increase the exposure in the raw converter.

Two stops overexposure would most likely blow highlights beyond recovery. Raw converters such as ACR can recover 0.5 to 1 f/stops. It really pays to expose properly to the right, just short of highlight clipping.

Bill
Logged

SeanPuckett

  • Full Member
  • ***
  • Offline Offline
  • Posts: 244
    • http://photi.ca/
Shooting to the right and raw conversion
« Reply #4 on: June 21, 2007, 01:37:32 pm »

If you're "shooting to the right" in a natural lighting situation by using the camera histogram, you're probably (although not certainly) losing specular highlights -- the upper 0.05% of your image -- that help the scene come alive.  I prefer to underexpose significantly (a full stop) to capture these highlights (gaining shutter speed) and relying on post processing to correct exposure in a non-linear space.  The technique preserves  ultrahighlight detail while pulling deep shadows out of the noise floor.  I lose a little colour accuracy in zone 1, which I feel an acceptable compromise for the liveliness of the resulting images.

Situations with sky, clouds and/or sun further benefit from this tactic, which mimics the exposure latitude available to analogue media.  Again -- yes, you lose some precision in the darkest shadows, but highlights are to my eye a far more valuable resource in creating a natural image.

I expect this is one of those religious flame war topics, but it's one I feel pretty strongly about -- so strongly, in fact, that I developed software to manage noise-reduced shadow recovery when using this tactic.
Logged

bjanes

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3387
Shooting to the right and raw conversion
« Reply #5 on: June 21, 2007, 01:39:47 pm »

Quote
You want to place as much data to the right so that you end up with the most data in the last stop of the tone curve (shadows). If you have a 12 bit file that can produce 6 stops, the first half of the data is contained in the first stop of exposure data (2048 levels). The last stop has only 64. See:

[{POST_SNAPBACK}][/a]

That's a good reason to expose to the right, but an even better reason is to gain better signal to noise. Shot noise is the major contributor to noise with most cameras and the signal to noise ratio varies as the square root of exposure (the number of electrons captured by the sensor).

The human eye can distinguish only about 70 levels of the 2048 levels in the brightest f/stop of the exposure (Weber-Fechner law, see [a href=\"http://www.normankoren.com/digital_tonality.html]Norman Koren[/url]), so most of those levels are wasted. However, sometimes they come in handy when extensive tonal manipulation is needed.

Bill
Logged

bjanes

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3387
Shooting to the right and raw conversion
« Reply #6 on: June 21, 2007, 01:55:27 pm »

Quote
If you're "shooting to the right" in a natural lighting situation by using the camera histogram, you're probably (although not certainly) losing specular highlights -- the upper 0.05% of your image -- that help the scene come alive.  I prefer to underexpose significantly (a full stop) to capture these highlights (gaining shutter speed) and relying on post processing to correct exposure in a non-linear space.  The technique preserves  ultrahighlight detail while pulling deep shadows out of the noise floor.  I lose a little colour accuracy in zone 1, which I feel an acceptable compromise for the liveliness of the resulting images.

[a href=\"index.php?act=findpost&pid=124204\"][{POST_SNAPBACK}][/a]

If you are interested in the specular highlights, then you should test your camera's histogram and blinking highlights under these conditions rather than systematically underexposing. Many cameras indicate highlight clipping a full stop before it actually occurs and if you expose a stop below the indicated loss of highlights, you could actually be underexposing by 2 stops.

The Nikon D70 was particularly conservative about highlight placement with its metering and many users complained about "underexposure". With my D200, I have learned that I can pretty much trust these indicators and rarely have blown highlights  that were not indicated.

With motion picture film and transparencies, it is common to place the specular highlights at 200% so that they really sparkle, but prints lack sufficient dynamic range for this luxury.

Bill
« Last Edit: June 21, 2007, 02:26:45 pm by bjanes »
Logged

digitaldog

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 20630
  • Andrew Rodney
    • http://www.digitaldog.net/
Shooting to the right and raw conversion
« Reply #7 on: June 21, 2007, 02:46:49 pm »

Quote
If you are interested in the specular highlights, then you should test your camera's histogram and blinking highlights under these conditions rather than systematically underexposing.[a href=\"index.php?act=findpost&pid=124209\"][{POST_SNAPBACK}][/a]

Absolutely.

 When I get some time, I really need to play more with this Sekonic and the target they supply to see if it really is a solution for metering 'for the right'. Anyone else have it and tried the calibration with Expose Right in mind?
Logged
http://www.digitaldog.net/
Author "Color Management for Photographers".

billmcknight

  • Guest
Shooting to the right and raw conversion
« Reply #8 on: June 21, 2007, 02:48:47 pm »

I think my question is misunderstood.  I personnaly always shoot to the right using RAW and do not underexpose.  However if I do happen to underexpose, it appears that I can recover the situation using the exposure compensation facilities available in the RAW converter.
If this is an acceptable practice then why promolgate the view that you should shoot to the right?
I have read your responses with great interest and look forward to more.
Logged

digitaldog

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 20630
  • Andrew Rodney
    • http://www.digitaldog.net/
Shooting to the right and raw conversion
« Reply #9 on: June 21, 2007, 02:49:36 pm »

Quote
Actually noise performance is largely determined by the amount of exposure, not the camera ISO. [{POST_SNAPBACK}][/a]

I've got some brackets at high ISO of this Sekonic target (DNG's and rendered examples) on my iDisk if anyone wants to look em over and comment. Using ACR/LR to (as Michael calls it) Normalize the 'over exposed' image you can see the effects on noise compared to the 'normal' exposure. Look at the noise!

The folder is called Expose to the Right.

My public iDisk:

thedigitaldog

Name (lower case) public
Password (lower case) public

Public folder Password is "public" (note the first letter is NOT capitalized).

To go there via a web browser, use this URL:

[a href=\"http://idisk.mac.com/thedigitaldog-Public]http://idisk.mac.com/thedigitaldog-Public[/url]
Logged
http://www.digitaldog.net/
Author "Color Management for Photographers".

digitaldog

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 20630
  • Andrew Rodney
    • http://www.digitaldog.net/
Shooting to the right and raw conversion
« Reply #10 on: June 21, 2007, 02:52:51 pm »

Quote
I think my question is misunderstood.  I personnaly always shoot to the right using RAW and do not underexpose.  However if I do happen to underexpose, it appears that I can recover the situation using the exposure compensation facilities available in the RAW converter.
If this is an acceptable practice then why promolgate the view that you should shoot to the right?
I have read your responses with great interest and look forward to more.
[a href=\"index.php?act=findpost&pid=124223\"][{POST_SNAPBACK}][/a]

The point is, you're not getting the same data. You're making the image appear lighter at an expense of data in the shadows, or shadow detail if you will. There's no free lunch here. When you shot film and over exposed and under developed (chrome) it altered the rendering of the image and in some cases (depending on how far off you are) affected image quality. Its really the same here. We're talking about correct expsoure to contain as much usable data within the full linear encoded capture.

Under exposure isn't an acceptable practice IF your goal is to capture as much usable data as possible.
Logged
http://www.digitaldog.net/
Author "Color Management for Photographers".

SeanPuckett

  • Full Member
  • ***
  • Offline Offline
  • Posts: 244
    • http://photi.ca/
Shooting to the right and raw conversion
« Reply #11 on: June 21, 2007, 03:09:13 pm »

I don't expect to accurately represent specular highlights on a print.  I expect to have usable detail there that would otherwise wash to a paper white blotch.  Even if it's all compressed down to 240-255 there's still good stuff there that makes an image look better to me.  And you might be surprised what the right algorithm can do for shadows.

I shoot digital at least a stop down so I have the luxury of highlight data that would otherwise be forever gone.  It's my shoulder.  And maybe my toe has a bit more noise.  But I'm cool with that.  The right algorithm can take care of most of it anyway.  Highlight recovery is never more than a sop; shadow recovery can do much more.
Logged

digitaldog

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 20630
  • Andrew Rodney
    • http://www.digitaldog.net/
Shooting to the right and raw conversion
« Reply #12 on: June 21, 2007, 03:20:48 pm »

Quote
I don't expect to accurately represent specular highlights on a print.

Why not? You're not supposed to blow out highlight data with the correct exposure. The correct exposure is about capturing the highlight data below clipping but as close to that as possible.

Quote
I shoot digital at least a stop down so I have the luxury of highlight data that would otherwise be forever gone.

Its only gone if you expose improperly, no one is suggesting that!

Quote
It's my shoulder.  And maybe my toe has a bit more noise.  But I'm cool with that.  The right algorithm can take care of most of it anyway.  Highlight recovery is never more than a sop; shadow recovery can do much more.
[a href=\"index.php?act=findpost&pid=124228\"][{POST_SNAPBACK}][/a]

If you look at how a sensor, which is just a photon counter captures this linear data, you'll see you're not doing yourself any good and some substantial harm. But its your data so by all means.

What is being discussed here is pretty simple to back up both scientifically (mathematically) and visually using actual image examples.

When I did darkroom work at school, there were plenty of others in the darkroom that didn't expose the paper correctly so they just left the paper in the first develoepr a lot longer or tried to rub areas with their fingers or blew on the print. Heck, if it works for you, go for it. But this isnt' really best practices.

The best thing to be said for under exposure is it makes the preview on the LCD of your camera look better.
Logged
http://www.digitaldog.net/
Author "Color Management for Photographers".

SeanPuckett

  • Full Member
  • ***
  • Offline Offline
  • Posts: 244
    • http://photi.ca/
Shooting to the right and raw conversion
« Reply #13 on: June 21, 2007, 03:44:06 pm »

I would rather have an exposure that is predominantly dark but has a complete record of the scene than have one that is easy to process but loses highlight detail.  With such an exposure I can create a print I'm pleased with.

I think we differ in our definition of highlights, and that's cool.
Logged

Tim Lookingbill

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2436
Shooting to the right and raw conversion
« Reply #14 on: June 21, 2007, 04:06:41 pm »

I've been playing with this concept using my Pentax K100D's in-camera histogram to study how the histogram changes under different exposures of the same scene in the highlite region. Haven't tackled RAW yet.

As I expose toward the right in a scene with say for example a blue sky fading to white toward the horizon, the peak in the highlite region of the histogram will start to gain spikes the closer I make this region of detail brighter through exposure moving the peak closer to clipping. The more I underexpose from this point this highlite peak starts to lose its spikes as it moves farther back from the clipping point. The nice thing is the two shots show very little difference in overall luminance between each other viewed in PS and the shadow regions don't clip to black in either one as well.
Logged

bjanes

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3387
Shooting to the right and raw conversion
« Reply #15 on: June 21, 2007, 04:18:23 pm »

Quote
I shoot digital at least a stop down so I have the luxury of highlight data that would otherwise be forever gone.  It's my shoulder.  And maybe my toe has a bit more noise.  But I'm cool with that.  The right algorithm can take care of most of it anyway.  Highlight recovery is never more than a sop; shadow recovery can do much more.
[a href=\"index.php?act=findpost&pid=124228\"][{POST_SNAPBACK}][/a]

If you shoot one stop down, you are losing one stop of dynamic range. With a high dynamic range subject, you may be clipping the shadows and losing data. However, if the camera is able to capture the entire dynamic range of the scene (as shown by a histogram that does not occupy the entire scale), shooting one stop down would only cause slightly increased noise in the shadows, which could be handled, especially if you are shooting with a low noise camera such as the Canon 5D. With a P&S camera, I doubt that this luxury would be worth the cost in noise.

I wouldn't underestimate the utility of highlight recovery in ACR. Specular highlights are usually towards white, have little detail, and can be recovered well. Under such conditions with daylight white balance, the green channel may be blown but the red and blue channels may contain good data, permitting ACR to do a good reconstruction.

I don't think there is any major heresy here as I infer that you believe in shooting to the right as much as possible while still leaving some headroom.

Bill
Logged

SeanPuckett

  • Full Member
  • ***
  • Offline Offline
  • Posts: 244
    • http://photi.ca/
Shooting to the right and raw conversion
« Reply #16 on: June 22, 2007, 04:55:53 pm »

I note that the Canon 1dMkIII does what I describe above when "Highlight Tone Priority" is selected.  Exposes a stop down (ISO 100->200), then pulls the main body of the exposure up whilst compressing the recorded highlights.  It's nice that the camera does the work for you, though.
Logged

bjanes

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3387
Shooting to the right and raw conversion
« Reply #17 on: June 22, 2007, 05:14:31 pm »

Quote
I've got some brackets at high ISO of this Sekonic target (DNG's and rendered examples) on my iDisk if anyone wants to look em over and comment. Using ACR/LR to (as Michael calls it) Normalize the 'over exposed' image you can see the effects on noise compared to the 'normal' exposure. Look at the noise!
[{POST_SNAPBACK}][/a]

I downloaded the files and did an analysis, which demonstrates good and bad effects of the test series and makes for interesting discussion.

It's nice to have a fancy light meter like Andrew's which gives real time readings, but by looking at the raw files produced by the camera we can also get equivalent exposure data, since the sensor is linear over most of its range. The following analysis involves only the green channel, but could be extended to the others.

Here is a curve from the two exposures spliced together showing the relative exposures of the patches and the resulting values in the raw file which can be decoded with DCRaw, a freeware program much used by digital tinkerers. The raw values are actually 0..4095 but they have been converted to 16 bit format by DCRaw in order to output them by multiplying them by 16.

[attachment=2666:attachment]

The exposure values on the left are all bunched and hard to read, but they can be spread out with a log-log plot which is standard for plotting characteristic curves of film. The pixel value is normalized to one by dividing by 65635 for a 16 bit file (erroneously labeled 65625 on the image). This notation is more confusing at first, but best once you get used to it. Norman Koren uses this format in his Imatest charts. I used 2 base logs for the exposure, so the values correspond to f/stops.

[attachment=2667:attachment]

This curve shows that the brightest patch on the f/16 1/50 sec shot is blown and not on the linear portion of the graph.

Now we can look at the characteristic curves of the rendered images. The f/16 @ 1/50 exposure with default rendering has a blown highlight, but this is largely restored by the normalized rendering (shown in yellow).  This normalized rendering is similar to that of the normal exposure of f/15 @ 1/200 sec, but the brightest patch is at the maximum pixel value and still slightly blown as shown below. The midtones match fairly well but have slightly different density and slope.

[attachment=2668:attachment]

Here is the histogram from the blown brightest patch mentioned above. Note that the right side of the bell shaped curve is truncated. This histogram is from the free ware program ImageJ, which does 16 bit histograms, unlike Photoshop.

[attachment=2669:attachment]

Now finally for the noise, which is measured as the standard deviation of the pixel values in the patches. Actually some of this variation is in the target and possibly in nonuniform illumination, but most of it is likely random noise from the camera. Such noise is primarily photon sampling noise (shot noise), but read noise enters into the equation at low exposure values (see [a href=\"http://www.clarkvision.com/imagedetail/evaluation-1d2/index.html]Roger Clark[/url] for explanation and a better way to measure noise). This noise is proportional to the square root of the number of photons captured by the sensor and is shown in this plot from the raw data. The noise is actually higher in the file with more exposure and worse in the highlights, perhaps contrary to conventional wisdom which associates noise with the shadows:

[attachment=2670:attachment]

However, what we are interested in more is the signal to noise ratio. This varies with the square root of the number of photons captured and is greater in the highlights because it is related to the number of photons actually captured (signal) to the square root of the number of photons captured (noise), which reduces mathematically to the square root of the number of photons captured [N/sqrt(n) = sqrt (n)].

[attachment=2671:attachment]

So in the final analysis, we quadrupled the exposure, captured 4 times as many photons and the S:N improved by a factor of 2 (square root of 4), as predicted by theory. However, the highlights were blown and the recovery was less than perfect. All in all, I thought this was a good blend of theory and practice and worth the effort needed to write it up.

Bill
« Last Edit: June 22, 2007, 05:27:22 pm by bjanes »
Logged

Steven Draper

  • Full Member
  • ***
  • Offline Offline
  • Posts: 147
    • http://www.stevendraperphotography.com
Shooting to the right and raw conversion
« Reply #18 on: June 22, 2007, 09:23:29 pm »

Well, I have found the replies very interesting. This is one area that I have been playing about with a little over recent weeks.

With my D2x and nx I find I have just over half a stop of "sensible" highlight recovery. By that I mean, reasonably good detail within clouds etc.

I found by moving the histogram to the right by adjusting the exposure, to the point of allowing highlights to clip, and then overexposing by half a stop provided more details in the shady leaves of the test scene and less noise in some other shadow items.

However when using the histogram one must remember how any WB settings MAY effect it!!!

I often find in my images that I am "highlight limited" and personally prefere expose for the highlights and then to push the curves fairly hard at the expense of noise and some shadow detail in order to retain bright details when presented with that situation. (with the exception of skin tones which in colour do not seem to respond well to much pushing. But I certainly don't want to waste space and data at the "high" end and so knowing how much data is there (and slightly more if going to make a composite blending or single shot HDR) or overexposing scenes with limited DR seems to be proving useful.

As for the question, how do I get more than 2 stops of ev correction on post. If using nx, the design folk may well have decided that they feel that is a sensible limit, capture one proved, I think 2 1/5. Bibble seeoms to get a lot of good comments about highlight recovery.
Logged
image examples are at my website  [url=h

bjanes

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3387
Shooting to the right and raw conversion
« Reply #19 on: June 26, 2007, 09:05:25 am »

Quote
However, the highlights were blown and the recovery was less than perfect. All in all, I thought this was a good blend of theory and practice and worth the effort needed to write it up.

Bill
[a href=\"index.php?act=findpost&pid=124420\"][{POST_SNAPBACK}][/a]

No comments on the analysis?

Bill
Logged
Pages: [1] 2 3 ... 5   Go Up