Luminous Landscape Forum
Raw & Post Processing, Printing => Digital Image Processing => Topic started by: billmcknight on June 21, 2007, 12:53:35 pm
-
Conventional wisdom is that for digital cameras it is best to expose so that the right side of the histogram is as far to the right as possible without blowing the highlights.
However; the raw converter I use allows exposure compensation of +/- 2 stops.
If I under expose does this mean I can bring out detail in the shadow areas or is this a false move. Having preached the conventional wisdom I was asked this question during a talk and I could not answer. Can anyone help.
-
Conventional wisdom is that for digital cameras it is best to expose so that the right side of the histogram is as far to the right as possible without blowing the highlights.
However; the raw converter I use allows exposure compensation of +/- 2 stops.
If I under expose does this mean I can bring out detail in the shadow areas or is this a false move. Having preached the conventional wisdom I was asked this question during a talk and I could not answer. Can anyone help.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=124188\")
You want to place as much data to the right so that you end up with the most data in the last stop of the tone curve (shadows). If you have a 12 bit file that can produce 6 stops, the first half of the data is contained in the first stop of exposure data (2048 levels). The last stop has only 64. See:
[a href=\"http://www.ppmag.com/reviews/200612_rodneycm.pdf]http://www.ppmag.com/reviews/200612_rodneycm.pdf[/url]
-
Conventional wisdom is that for digital cameras it is best to expose so that the right side of the histogram is as far to the right as possible without blowing the highlights.
However; the raw converter I use allows exposure compensation of +/- 2 stops.
If I under expose does this mean I can bring out detail in the shadow areas or is this a false move. Having preached the conventional wisdom I was asked this question during a talk and I could not answer. Can anyone help.
[a href=\"index.php?act=findpost&pid=124188\"][{POST_SNAPBACK}][/a]
I'm not sure I understand your question. Do you wish to underexpose, with an idea of "exposing to the left" to bring out shadow detail?
If this is the question, then no.
You gain shadow detail by exposing to the right (and loose shutterspeed, not to be forgotten!) and then correcting the exposure in the RAW converter.
-
Conventional wisdom is that for digital cameras it is best to expose so that the right side of the histogram is as far to the right as possible without blowing the highlights.
However; the raw converter I use allows exposure compensation of +/- 2 stops.
If I under expose does this mean I can bring out detail in the shadow areas or is this a false move. Having preached the conventional wisdom I was asked this question during a talk and I could not answer. Can anyone help.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=124188\")
IMHO, conventional wisdom is correct. At base ISO you might get away with two stops underexposure, since it would be like shooting ISO 400 rather than the base 100. Signal to noise would be worse, especially in the shadows and tonality could suffer. At high ISO, noise would probably be objectionable, depending on the camera.
Actually noise performance is largely determined by the amount of exposure, not the camera ISO. Above unity gain of the camera (which varies from 800 -1600 with most DLSRs--see [a href=\"http://www.clarkvision.com/imagedetail/digital.sensor.performance.summary/]Roger Clark[/url]), if you are strapped for exposure by shutter speed or f/stop necessities, it does not help to raise the camera ISO any further than that of unity gain, and you can increase the exposure in the raw converter.
Two stops overexposure would most likely blow highlights beyond recovery. Raw converters such as ACR can recover 0.5 to 1 f/stops. It really pays to expose properly to the right, just short of highlight clipping.
Bill
-
If you're "shooting to the right" in a natural lighting situation by using the camera histogram, you're probably (although not certainly) losing specular highlights -- the upper 0.05% of your image -- that help the scene come alive. I prefer to underexpose significantly (a full stop) to capture these highlights (gaining shutter speed) and relying on post processing to correct exposure in a non-linear space. The technique preserves ultrahighlight detail while pulling deep shadows out of the noise floor. I lose a little colour accuracy in zone 1, which I feel an acceptable compromise for the liveliness of the resulting images.
Situations with sky, clouds and/or sun further benefit from this tactic, which mimics the exposure latitude available to analogue media. Again -- yes, you lose some precision in the darkest shadows, but highlights are to my eye a far more valuable resource in creating a natural image.
I expect this is one of those religious flame war topics, but it's one I feel pretty strongly about -- so strongly, in fact, that I developed software to manage noise-reduced shadow recovery when using this tactic.
-
You want to place as much data to the right so that you end up with the most data in the last stop of the tone curve (shadows). If you have a 12 bit file that can produce 6 stops, the first half of the data is contained in the first stop of exposure data (2048 levels). The last stop has only 64. See:
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=124199\")
That's a good reason to expose to the right, but an even better reason is to gain better signal to noise. Shot noise is the major contributor to noise with most cameras and the signal to noise ratio varies as the square root of exposure (the number of electrons captured by the sensor).
The human eye can distinguish only about 70 levels of the 2048 levels in the brightest f/stop of the exposure (Weber-Fechner law, see [a href=\"http://www.normankoren.com/digital_tonality.html]Norman Koren[/url]), so most of those levels are wasted. However, sometimes they come in handy when extensive tonal manipulation is needed.
Bill
-
If you're "shooting to the right" in a natural lighting situation by using the camera histogram, you're probably (although not certainly) losing specular highlights -- the upper 0.05% of your image -- that help the scene come alive. I prefer to underexpose significantly (a full stop) to capture these highlights (gaining shutter speed) and relying on post processing to correct exposure in a non-linear space. The technique preserves ultrahighlight detail while pulling deep shadows out of the noise floor. I lose a little colour accuracy in zone 1, which I feel an acceptable compromise for the liveliness of the resulting images.
[a href=\"index.php?act=findpost&pid=124204\"][{POST_SNAPBACK}][/a]
If you are interested in the specular highlights, then you should test your camera's histogram and blinking highlights under these conditions rather than systematically underexposing. Many cameras indicate highlight clipping a full stop before it actually occurs and if you expose a stop below the indicated loss of highlights, you could actually be underexposing by 2 stops.
The Nikon D70 was particularly conservative about highlight placement with its metering and many users complained about "underexposure". With my D200, I have learned that I can pretty much trust these indicators and rarely have blown highlights that were not indicated.
With motion picture film and transparencies, it is common to place the specular highlights at 200% so that they really sparkle, but prints lack sufficient dynamic range for this luxury.
Bill
-
If you are interested in the specular highlights, then you should test your camera's histogram and blinking highlights under these conditions rather than systematically underexposing.[a href=\"index.php?act=findpost&pid=124209\"][{POST_SNAPBACK}][/a]
Absolutely.
When I get some time, I really need to play more with this Sekonic and the target they supply to see if it really is a solution for metering 'for the right'. Anyone else have it and tried the calibration with Expose Right in mind?
-
I think my question is misunderstood. I personnaly always shoot to the right using RAW and do not underexpose. However if I do happen to underexpose, it appears that I can recover the situation using the exposure compensation facilities available in the RAW converter.
If this is an acceptable practice then why promolgate the view that you should shoot to the right?
I have read your responses with great interest and look forward to more.
-
Actually noise performance is largely determined by the amount of exposure, not the camera ISO. [{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=124202\")
I've got some brackets at high ISO of this Sekonic target (DNG's and rendered examples) on my iDisk if anyone wants to look em over and comment. Using ACR/LR to (as Michael calls it) Normalize the 'over exposed' image you can see the effects on noise compared to the 'normal' exposure. Look at the noise!
The folder is called Expose to the Right.
My public iDisk:
thedigitaldog
Name (lower case) public
Password (lower case) public
Public folder Password is "public" (note the first letter is NOT capitalized).
To go there via a web browser, use this URL:
[a href=\"http://idisk.mac.com/thedigitaldog-Public]http://idisk.mac.com/thedigitaldog-Public[/url]
-
I think my question is misunderstood. I personnaly always shoot to the right using RAW and do not underexpose. However if I do happen to underexpose, it appears that I can recover the situation using the exposure compensation facilities available in the RAW converter.
If this is an acceptable practice then why promolgate the view that you should shoot to the right?
I have read your responses with great interest and look forward to more.
[a href=\"index.php?act=findpost&pid=124223\"][{POST_SNAPBACK}][/a]
The point is, you're not getting the same data. You're making the image appear lighter at an expense of data in the shadows, or shadow detail if you will. There's no free lunch here. When you shot film and over exposed and under developed (chrome) it altered the rendering of the image and in some cases (depending on how far off you are) affected image quality. Its really the same here. We're talking about correct expsoure to contain as much usable data within the full linear encoded capture.
Under exposure isn't an acceptable practice IF your goal is to capture as much usable data as possible.
-
I don't expect to accurately represent specular highlights on a print. I expect to have usable detail there that would otherwise wash to a paper white blotch. Even if it's all compressed down to 240-255 there's still good stuff there that makes an image look better to me. And you might be surprised what the right algorithm can do for shadows.
I shoot digital at least a stop down so I have the luxury of highlight data that would otherwise be forever gone. It's my shoulder. And maybe my toe has a bit more noise. But I'm cool with that. The right algorithm can take care of most of it anyway. Highlight recovery is never more than a sop; shadow recovery can do much more.
-
I don't expect to accurately represent specular highlights on a print.
Why not? You're not supposed to blow out highlight data with the correct exposure. The correct exposure is about capturing the highlight data below clipping but as close to that as possible.
I shoot digital at least a stop down so I have the luxury of highlight data that would otherwise be forever gone.
Its only gone if you expose improperly, no one is suggesting that!
It's my shoulder. And maybe my toe has a bit more noise. But I'm cool with that. The right algorithm can take care of most of it anyway. Highlight recovery is never more than a sop; shadow recovery can do much more.
[a href=\"index.php?act=findpost&pid=124228\"][{POST_SNAPBACK}][/a]
If you look at how a sensor, which is just a photon counter captures this linear data, you'll see you're not doing yourself any good and some substantial harm. But its your data so by all means.
What is being discussed here is pretty simple to back up both scientifically (mathematically) and visually using actual image examples.
When I did darkroom work at school, there were plenty of others in the darkroom that didn't expose the paper correctly so they just left the paper in the first develoepr a lot longer or tried to rub areas with their fingers or blew on the print. Heck, if it works for you, go for it. But this isnt' really best practices.
The best thing to be said for under exposure is it makes the preview on the LCD of your camera look better.
-
I would rather have an exposure that is predominantly dark but has a complete record of the scene than have one that is easy to process but loses highlight detail. With such an exposure I can create a print I'm pleased with.
I think we differ in our definition of highlights, and that's cool.
-
I've been playing with this concept using my Pentax K100D's in-camera histogram to study how the histogram changes under different exposures of the same scene in the highlite region. Haven't tackled RAW yet.
As I expose toward the right in a scene with say for example a blue sky fading to white toward the horizon, the peak in the highlite region of the histogram will start to gain spikes the closer I make this region of detail brighter through exposure moving the peak closer to clipping. The more I underexpose from this point this highlite peak starts to lose its spikes as it moves farther back from the clipping point. The nice thing is the two shots show very little difference in overall luminance between each other viewed in PS and the shadow regions don't clip to black in either one as well.
-
I shoot digital at least a stop down so I have the luxury of highlight data that would otherwise be forever gone. It's my shoulder. And maybe my toe has a bit more noise. But I'm cool with that. The right algorithm can take care of most of it anyway. Highlight recovery is never more than a sop; shadow recovery can do much more.
[a href=\"index.php?act=findpost&pid=124228\"][{POST_SNAPBACK}][/a]
If you shoot one stop down, you are losing one stop of dynamic range. With a high dynamic range subject, you may be clipping the shadows and losing data. However, if the camera is able to capture the entire dynamic range of the scene (as shown by a histogram that does not occupy the entire scale), shooting one stop down would only cause slightly increased noise in the shadows, which could be handled, especially if you are shooting with a low noise camera such as the Canon 5D. With a P&S camera, I doubt that this luxury would be worth the cost in noise.
I wouldn't underestimate the utility of highlight recovery in ACR. Specular highlights are usually towards white, have little detail, and can be recovered well. Under such conditions with daylight white balance, the green channel may be blown but the red and blue channels may contain good data, permitting ACR to do a good reconstruction.
I don't think there is any major heresy here as I infer that you believe in shooting to the right as much as possible while still leaving some headroom.
Bill
-
I note that the Canon 1dMkIII does what I describe above when "Highlight Tone Priority" is selected. Exposes a stop down (ISO 100->200), then pulls the main body of the exposure up whilst compressing the recorded highlights. It's nice that the camera does the work for you, though.
-
I've got some brackets at high ISO of this Sekonic target (DNG's and rendered examples) on my iDisk if anyone wants to look em over and comment. Using ACR/LR to (as Michael calls it) Normalize the 'over exposed' image you can see the effects on noise compared to the 'normal' exposure. Look at the noise!
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=124224\")
I downloaded the files and did an analysis, which demonstrates good and bad effects of the test series and makes for interesting discussion.
It's nice to have a fancy light meter like Andrew's which gives real time readings, but by looking at the raw files produced by the camera we can also get equivalent exposure data, since the sensor is linear over most of its range. The following analysis involves only the green channel, but could be extended to the others.
Here is a curve from the two exposures spliced together showing the relative exposures of the patches and the resulting values in the raw file which can be decoded with DCRaw, a freeware program much used by digital tinkerers. The raw values are actually 0..4095 but they have been converted to 16 bit format by DCRaw in order to output them by multiplying them by 16.
[attachment=2666:attachment]
The exposure values on the left are all bunched and hard to read, but they can be spread out with a log-log plot which is standard for plotting characteristic curves of film. The pixel value is normalized to one by dividing by 65635 for a 16 bit file (erroneously labeled 65625 on the image). This notation is more confusing at first, but best once you get used to it. Norman Koren uses this format in his Imatest charts. I used 2 base logs for the exposure, so the values correspond to f/stops.
[attachment=2667:attachment]
This curve shows that the brightest patch on the f/16 1/50 sec shot is blown and not on the linear portion of the graph.
Now we can look at the characteristic curves of the rendered images. The f/16 @ 1/50 exposure with default rendering has a blown highlight, but this is largely restored by the normalized rendering (shown in yellow). This normalized rendering is similar to that of the normal exposure of f/15 @ 1/200 sec, but the brightest patch is at the maximum pixel value and still slightly blown as shown below. The midtones match fairly well but have slightly different density and slope.
[attachment=2668:attachment]
Here is the histogram from the blown brightest patch mentioned above. Note that the right side of the bell shaped curve is truncated. This histogram is from the free ware program ImageJ, which does 16 bit histograms, unlike Photoshop.
[attachment=2669:attachment]
Now finally for the noise, which is measured as the standard deviation of the pixel values in the patches. Actually some of this variation is in the target and possibly in nonuniform illumination, but most of it is likely random noise from the camera. Such noise is primarily photon sampling noise (shot noise), but read noise enters into the equation at low exposure values (see [a href=\"http://www.clarkvision.com/imagedetail/evaluation-1d2/index.html]Roger Clark[/url] for explanation and a better way to measure noise). This noise is proportional to the square root of the number of photons captured by the sensor and is shown in this plot from the raw data. The noise is actually higher in the file with more exposure and worse in the highlights, perhaps contrary to conventional wisdom which associates noise with the shadows:
[attachment=2670:attachment]
However, what we are interested in more is the signal to noise ratio. This varies with the square root of the number of photons captured and is greater in the highlights because it is related to the number of photons actually captured (signal) to the square root of the number of photons captured (noise), which reduces mathematically to the square root of the number of photons captured [N/sqrt(n) = sqrt (n)].
[attachment=2671:attachment]
So in the final analysis, we quadrupled the exposure, captured 4 times as many photons and the S:N improved by a factor of 2 (square root of 4), as predicted by theory. However, the highlights were blown and the recovery was less than perfect. All in all, I thought this was a good blend of theory and practice and worth the effort needed to write it up.
Bill
-
Well, I have found the replies very interesting. This is one area that I have been playing about with a little over recent weeks.
With my D2x and nx I find I have just over half a stop of "sensible" highlight recovery. By that I mean, reasonably good detail within clouds etc.
I found by moving the histogram to the right by adjusting the exposure, to the point of allowing highlights to clip, and then overexposing by half a stop provided more details in the shady leaves of the test scene and less noise in some other shadow items.
However when using the histogram one must remember how any WB settings MAY effect it!!!
I often find in my images that I am "highlight limited" and personally prefere expose for the highlights and then to push the curves fairly hard at the expense of noise and some shadow detail in order to retain bright details when presented with that situation. (with the exception of skin tones which in colour do not seem to respond well to much pushing. But I certainly don't want to waste space and data at the "high" end and so knowing how much data is there (and slightly more if going to make a composite blending or single shot HDR) or overexposing scenes with limited DR seems to be proving useful.
As for the question, how do I get more than 2 stops of ev correction on post. If using nx, the design folk may well have decided that they feel that is a sensible limit, capture one proved, I think 2 1/5. Bibble seeoms to get a lot of good comments about highlight recovery.
-
However, the highlights were blown and the recovery was less than perfect. All in all, I thought this was a good blend of theory and practice and worth the effort needed to write it up.
Bill
[a href=\"index.php?act=findpost&pid=124420\"][{POST_SNAPBACK}][/a]
No comments on the analysis?
Bill
-
No comments on the analysis?
Bill
[a href=\"index.php?act=findpost&pid=124962\"][{POST_SNAPBACK}][/a]
Sorry, been on the road (but that doesn't stop others from commenting).
Now that this trip is almost over (writing from lovely Ohare airport), I hope to spend some time looking into this a lot more, talking with the product manager for the new meter and doing many more tests. Got to get to the bottom of this.
-
This is very interesting thread. There is some agreement that one should expose to the right, but not overdo it. I think that is the problem. The histogram is great tool. But as it is derived from the incamera jpeg, it is not very accurate. With the rgb histogram on the Canon 5D, I have a better idea of what is going on, but in some images there are some highlight values that have poor representation in the histogram. You can think of the histogram as the fraction of pixels with intensity between I and I+DI, where I is the intensity value (0-255). There can be a highlight "toe" that is not well defined in the histogram because the fraction of pixels with that value is low. It can be hard to see the toe under contrasty review conditions. It is only later, when we examine the raw image and set the white balance that we really understand if we got "the ideal" exposure. Those that espouse moderation know what they are talking about. Thiis is an area where experience really counts.
-
This is very interesting thread. There is some agreement that one should expose to the right, but not overdo it. I think that is the problem. The histogram is great tool. But as it is derived from the incamera jpeg, it is not very accurate. With the rgb histogram on the Canon 5D, I have a better idea of what is going on, but in some images there are some highlight values that have poor representation in the histogram. You can think of the histogram as the fraction of pixels with intensity between I and I+DI, where I is the intensity value (0-255). There can be a highlight "toe" that is not well defined in the histogram because the fraction of pixels with that value is low. It can be hard to see the toe under contrasty review conditions. It is only later, when we examine the raw image and set the white balance that we really understand if we got "the ideal" exposure. Those that espouse moderation know what they are talking about. Thiis is an area where experience really counts.
[a href=\"index.php?act=findpost&pid=125008\"][{POST_SNAPBACK}][/a]
This is very consistent with my experience with a Hasselblad H3D-39. The histogram on the back does not show any evidence of highlight clipping in many images, but it is there when the file is opened in ACR or Flexcolor, and I really dislike the missing highlight detail. Much prefer black shadows to highlights with no detail. It seems that the highlight clipping display on the LCD is much more reliable in showing clipping at the far ends of the histogram and I am starting to rely on that as an important check on the accuracy of the histogram.
-
You want to place as much data to the right so that you end up with the most data in the last stop of the tone curve (shadows). If you have a 12 bit file that can produce 6 stops, the first half of the data is contained in the first stop of exposure data (2048 levels). The last stop has only 64. See:
http://www.ppmag.com/reviews/200612_rodneycm.pdf (http://www.ppmag.com/reviews/200612_rodneycm.pdf)
[a href=\"index.php?act=findpost&pid=124199\"][{POST_SNAPBACK}][/a]
This is an oft-quoted model of what is going on, but it is not the most accurate one. The number of levels in a stop is not an issue with current cameras. They have too much noise to be limited by posterization, except for a few cameras at their lowest ISO (Pentax K10D at ISO 100, for example), and then, just barely, and in the deepest shadows.
The signal-to-noise ratio is what matters, in the absence of any real posterization threat. In the deepest shadows, dominated by read noises, the SNR doubles with each doubling of exposure (+1 EC). In the midtone and highlight areas, SNR doubles with each quadrupling of exposure (+2 EC), and the in-between zones, in-between. The benefit is strongest in the dark shadow areas.
-
Above unity gain of the camera (which varies from 800 -1600 with most DLSRs--see Roger Clark (http://www.clarkvision.com/imagedetail/digital.sensor.performance.summary/)), if you are strapped for exposure by shutter speed or f/stop necessities, it does not help to raise the camera ISO any further than that of unity gain, and you can increase the exposure in the raw converter.
[a href=\"index.php?act=findpost&pid=124202\"][{POST_SNAPBACK}][/a]
The concept of unity gain, as proposed by Roger, is meaningless, IMO. ADC units are arbitrary except in their ability to posterize. He has no evidence for that conclusion; the camera he based it on has no real ISO 3200 with analog gain - it is just 1600 pushed, so of course there is no real noise benefit in using it.
-
This is very interesting thread. There is some agreement that one should expose to the right, but not overdo it. I think that is the problem. The histogram is great tool. But as it is derived from the incamera jpeg, it is not very accurate. With the rgb histogram on the Canon 5D, I have a better idea of what is going on, but in some images there are some highlight values that have poor representation in the histogram.
[a href=\"index.php?act=findpost&pid=125008\"][{POST_SNAPBACK}][/a]
The camera histograms are derived from the JPEG preview, but you do have some control over their appearance via the camera settings. The contrast control, for example, applies an S curve to the data to lower the quarter tones and raise the three quarter tones. This should not affect the end-points of the histogram but does affect the shape towards the extremes. Therefore, many set the camera to low contrast to get a better view of the histograms.
If the camera permits the uploading of a custom tonal response curve, one can use this to calibrate the histogram so that clipping in the histogram correlates with clipping in the raw file. For example, if the histogram indicates clipping when there is none, one can upload a curve with roll off in the highlights. The RGB histograms reflect the status of the channels after white balance, but one can upload a custom WB to obtain an indication of the contents of the channels prior to WB.
Bill
-
If you're "shooting to the right" in a natural lighting situation by using the camera histogram, you're probably (although not certainly) losing specular highlights -- the upper 0.05% of your image -- that help the scene come alive. I prefer to underexpose significantly (a full stop) to capture these highlights (gaining shutter speed) and relying on post processing to correct exposure in a non-linear space.[a href=\"index.php?act=findpost&pid=124204\"][{POST_SNAPBACK}][/a]
Well, your standpoint does not contradict the shooting to the right idea, as long as you clarify what the right edge of your tonal levels is. If you want specular highlights, then you can expose them as far to the right as possible. "To the right" doesn't necessarily mean positive EC. It just means pushing the tones you wish to preserve just short of clipping. For black spraypaint on on a dark grey wall, ETTR might mean +3 EC. For capturing detail in city lights at night, ETTR might mean -2 EC. The underlying principle is to get the brightest tones you wish to record just short of clipping.
-
I think my question is misunderstood. I personnaly always shoot to the right using RAW and do not underexpose. However if I do happen to underexpose, it appears that I can recover the situation using the exposure compensation facilities available in the RAW converter.
If this is an acceptable practice then why promolgate the view that you should shoot to the right?[a href=\"index.php?act=findpost&pid=124223\"][{POST_SNAPBACK}][/a]
Acceptable and optimum are two different things. If you expose one image a stop more to the right than another, the one exposed more to the right will have usable shadows a stop deeper in real world light, and the acceptable shadows will be good shadows.
-
The camera histograms are derived from the JPEG preview, but you do have some control over their appearance via the camera settings. The contrast control, for example, applies an S curve to the data to lower the quarter tones and raise the three quarter tones. This should not affect the end-points of the histogram but does affect the shape towards the extremes. Therefore, many set the camera to low contrast to get a better view of the histograms.
Good point! This deserves a lot more attention. It would be nice if the camera makers would just allow us to view a linear encoded Histogram (it will take some getting used to, it's all shoved to one side). Then you need to get the brightness on the LCD way down too.
Well, your standpoint does not contradict the shooting to the right idea, as long as you clarify what the right edge of your tonal levels is. If you want specular highlights, then you can expose them as far to the right as possible.
Exactly. We need to know how to nail the highlights.
-
If you shoot one stop down, you are losing one stop of dynamic range.[a href=\"index.php?act=findpost&pid=124245\"][{POST_SNAPBACK}][/a]
Actually, no DR is lost; it is simply shifted from the shadows to the highlights.
-
The concept of unity gain, as proposed by Roger, is meaningless, IMO. ADC units are arbitrary except in their ability to posterize. He has no evidence for that conclusion; the camera he based it on has no real ISO 3200 with analog gain - it is just 1600 pushed, so of course there is no real noise benefit in using it.
[a href=\"index.php?act=findpost&pid=125016\"][{POST_SNAPBACK}][/a]
Yes, John, we have been over this before. However, it does not make sense to quantify beyond 1 electron = 1 ADU. At that point you have completely quantified the number of electrons that have been captured--you have the actual count and that is all you need.
I have yet to hear your refutation of that point.
Bill
-
Actually, no DR is lost; it is simply shifted from the shadows to the highlights.
[a href=\"index.php?act=findpost&pid=125035\"][{POST_SNAPBACK}][/a]
That is assuming that you still have data in all 12 bits of your ADC. If the upper bit is empty, you have lost potential DR. In that case, the maximal:minimal recorded signal drops from 4096:1 to 2048:1 with a 12 bit ADC. With decreased exposure, the noise floor for DR also increases, further limiting actual DR.
Bill
-
That is assuming that you still have data in all 12 bits of your ADC. If the upper bit is empty, you have lost potential DR. In that case, the maximal:minimal recorded signal drops from 4096:1 to 2048:1 with a 12 bit ADC. With decreased exposure, the noise floor for DR also increases, further limiting actual DR.
[a href=\"index.php?act=findpost&pid=125048\"][{POST_SNAPBACK}][/a]
You are talking about the DR of the capture; I tend to think in terms of the medium, as the capture is generally a wild card, as in the specular highlights that the other poster was concerned with.
-
You are talking about the DR of the capture; I tend to think in terms of the medium, as the capture is generally a wild card, as in the specular highlights that the other poster was concerned with.
[a href=\"index.php?act=findpost&pid=125134\"][{POST_SNAPBACK}][/a]
Yes, in a thread about exposure to the right, are not we all talking about the capture and how to optimize the captured data with due consideration given to the limitations of the medium? Specular highlights and other features of the scene are not really wild cards, but are subject to the laws of physics and scientific analysis.
-
Yes, in a thread about exposure to the right, are not we all talking about the capture and how to optimize the captured data with due consideration given to the limitations of the medium?
Many times I have seen people imply that some choice that they make in exposure affects the DR of the camera, and that is what I meant to dispell.
Specular highlights and other features of the scene are not really wild cards, but are subject to the laws of physics and scientific analysis.
[a href=\"index.php?act=findpost&pid=125159\"][{POST_SNAPBACK}][/a]
They are wild because you can not accurately measure them in many situations; you can only gamble.
-
Many times I have seen people imply that some choice that they make in exposure affects the DR of the camera, and that is what I meant to dispell.
They are wild because you can not accurately measure them in many situations; you can only gamble.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=125197\")
In most cases it is not necessary to render specular highlights accurately, but merely place them above the level of diffuse white in the scene and then let them blow out at higher levels. In movies and slides, the specular highlights are often placed at 200% as discussed in this [a href=\"http://www.color.org/iccprofile.html]ICC Paper[/url]. In this case you would allow 1 stop of headroom as suggested early in this thread. For prints, probably a bit less headroom would be advisable.
Bill
-
In most cases it is not necessary to render specular highlights accurately, but merely place them above the level of diffuse white in the scene and then let them blow out at hither levels. In movies and slides, the specular highlights are often placed at 200% as discussed in this ICC Paper (http://www.color.org/iccprofile.html). In this case you would allow 1 stop of headroom as suggested early in this thread. For prints, probably a bit less headroom would be advisable.
[a href=\"index.php?act=findpost&pid=125221\"][{POST_SNAPBACK}][/a]
Nevertheless, it is still up to the individual how much detail they want from these small, brighter areas, and how much they are willing to gamble. There are ways to compress DR locally in images if they want.
-
Yes, John, we have been over this before. However, it does not make sense to quantify beyond 1 electron = 1 ADU. At that point you have completely quantified the number of electrons that have been captured--you have the actual count and that is all you need.
I have yet to hear your refutation of that point.
[a href=\"index.php?act=findpost&pid=125046\"][{POST_SNAPBACK}][/a]
I'm pretty certain I have addressed this before. Regardless, the fact is that cameras are *NOT* counting electrons, even if that is what we'd really want them to do. The discreet electrons come packaged in a bundle of analog noise caused by reading, amplifying, transporting, (possibly amplifying again,) and digitizing the electron charge. This extra read noise is *NOT* in units of electrons; it is analog until digitization.
With customized circuitry, Canon has been able to get the level of read noises at the highest ISO down to the equivalent of a few electrons (not a few discreet electrons!). They have been able to do this at the *highest* amplification used in the cameras. Nothing that Roger writes on his website addresses what may or may not happen with more and better amplification; he simply jumps to the conclusion that nothing is gained, and uses the fact that his 1Dmk2 has the same total read noise in electrons at ISO 3200 as it does at ISO 1600. That is not any real support for his conclusion, because ISO 3200 *IS* ISO 1600 on that camera. Had he used a Minolta K7, which uses real amplification at ISO 3200, he would have measured slightly less noise at ISO 3200, and if he had actually looked at the shadows, there would be slightly less line noise at 3200, and less chromatic noise in a RAW at 3200 than 1600 pushed to 3200. I'd offer the 1Dmk3's ISO 3200 as additional support for my claim, but the fact that it is 14 bit may make you feel that the goal post for unity gain has moved (despite the fact that mk3 ISO 3200 quantized to 8 bits is still far less noisy than the mk2's ISO 3200).
I'm sure I have shown you this chart before, in previous refutations of the "unity gain" limit:
(http://www.pbase.com/jps_photo/image/65737967/original.jpg)
That is the total read noise, and the isolated horizontal and vertical line noises.
The total read noise, the yellow line, is scaled to 10% to fit in with the others, and the vertical axis is the read noise normalized to ISO 100 for all other ISOs, as standard deviation in ADUs (which can be considered arbitrary units of electrons). The noises clearly show no sign of flatlining completely by 3200, as far as the trends up to 1600 are concerned, especially the line noises, which are far more visible than their statistical strength suggests. Horizontal line noise *is*, without a doubt, the most troublesome aspect of high-ISO shadow areas in Canon cameras.
Roger has nothing to really support his unity gain hypothesis; he is simply applying the concept of one equals one, but these ones are really apples and oranges; one is discreet integer values, and the other is discreet multiples of a single value, with variance at a finer degree. The ADC in these cameras can *NOT* count electrons. They can only get so close to counting them, and by all appearances, with Canon's technology, the more you amplify the signal, the more you can reduce the inaccuracy, which is why it is illogical to declare that something as arbitrary as the ADU unit is a meaningful limit to practical amplification. And the ADU truly *is* arbitrary when it is fine enough not to cause posterization of RAW data. Only when it is coarse enough to cause posterization does the actual absolute meaning of the ADU have any value (the ability to posterize).
-
We need to know how to nail the highlights.
[a href=\"index.php?act=findpost&pid=125034\"][{POST_SNAPBACK}][/a]
Even without direct support on the camera, it could conceivably be done now with a computer hooked up to the camera; a program could look for new RAW files on the card or in the computer, and display a RAW image and/or histogram on the screen. Not for action shooting, of course!
-
The camera histograms are derived from the JPEG preview, but you do have some control over their appearance via the camera settings. The contrast control, for example, applies an S curve to the data to lower the quarter tones and raise the three quarter tones. This should not affect the end-points of the histogram but does affect the shape towards the extremes. Therefore, many set the camera to low contrast to get a better view of the histograms.
If the camera permits the uploading of a custom tonal response curve, one can use this to calibrate the histogram so that clipping in the histogram correlates with clipping in the raw file. For example, if the histogram indicates clipping when there is none, one can upload a curve with roll off in the highlights. The RGB histograms reflect the status of the channels after white balance, but one can upload a custom WB to obtain an indication of the contents of the channels prior to WB.
Bill
[a href=\"index.php?act=findpost&pid=125019\"][{POST_SNAPBACK}][/a]
Modifying the contrast seems like a good idea. But in the final analysis, the camera manufacturers have let us down. There are not enough bins in the histogram and it comes from the jpeg, not the raw. Because there are so few bins in the histogram, its shape is poorly defined. This matters most in the highlights. In some case parts of the histrgram can be distorted becasue the bin-width is too large. One experiment for someone with ambition and too much time is to measure the in-camera histogram and compare its shape to the one in photoshop. It would be good to do this experiment for several lighting conditions
-
No comments on the analysis?
Bill
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=124962\")
Thanks for sharing this analysis. This whole discussion is helping me to see some of the things I am doing wrong. I think I resist going up to a higher ISO fearing noise, but I end up pushing it up on the computer and maybe coming up with even more noise (I use a 20D primarily)
[span style=\'font-size:9pt;line-height:100%\']______________________
"You cannot escape the responsibility of tomorrow by evading it today." Abraham Lincoln
[a href=\"http://www.iriselements.com]Company[/url] – Projects (Affordable Stock Photos (http://www.imagetrail.net) – Sell Your Photos (http://www.microstockforum.com))[/span]
-
Modifying the contrast seems like a good idea.
[a href=\"index.php?act=findpost&pid=125328\"][{POST_SNAPBACK}][/a]
It is a good idea, but you have to experiemnt with whatever camera you are using. Try different settings of jpeg contrast and see how 'no clipping' or 'moderate clipping' on the camera's LCD screen (flashing red patches)translates to the degree of recoverable highlights in your RAW converter.
I've got my 5D set so that I know a small amount of red flashing in the brightest parts of the sky is recoverable in ACR, but not if a large area of the sky is flashing.
I think that setting is minimum contrast (it's a long time since I set it). At maximum contrast, I'd have a situation where the histogram and blown highlight warning would have the whole sky flashing, causing me to underexpose.
-
recoverable highlights in your RAW converter.
"Highlight recovery" is a usual term when talking about RAW developing, but it's most of the times wrong. What you get in ACR pushing down the exposure slider is not any highlight recovering at all. If your previously blown highlights become not blown through underexposure tuning, is because they were actually never blown on the RAW file. It was YOU in the developing process, commonly with the white balance (which implies heavy channel scaling by factors usually greater than 2.0 in linear, i.e. +1EV correction, and easily beyond 2.5, nearly +1.5EV), who blowed those areas.
ACR does not apply any real recovery of highlights. DCRAW in its -H2 to -H9 modes does; DCRAW interpolates truly blown pixels to values close to those in the boundaries; it "invents" colours where they were previously blown.
I just wanted to point this as most people talk about hightlight recorvery when thery are just simply referrering to "no-blowing underexposure".
Here is a sample that demonstrates how white balance can blow image areas that were not in the original RAW file. This image was developed without applying any white balance (hence the greeny colour):
(http://img164.imageshack.us/img164/173/sobrese6.jpg)
If you look into the window area and compare that result (ANTES) to the one obtained with an ACR type white balance applied (con balance de blancos) you can see how much information YOU are blowing just with your white balance:
(http://img248.imageshack.us/img248/2407/wbuu2.gif)[/quote]
-
I've read this topic over and over and only grasp about a quarter of what you folks are saying. And I keep asking myself, so what! Are you only discussing theory or is there some real world benefit to your debate.
I ask simply because I'm so new to photography and because of my advancing years, don't have time to theorize. If I'm unsure about exposure, I bracket. If I still think that I still have not gotten the whole range, I bracket and merge to HDR.
Please don't think that I'm being disrespectfull, because that is not my intent.
To give you an idea where I'm coming from, my work is for personal enjoyment, don't sell or show. But rest assured, I do have a very demanding audiance.
FWIW
-
I'm pretty certain I have addressed this before. Regardless, the fact is that cameras are *NOT* counting electrons, even if that is what we'd really want them to do. The discreet electrons come packaged in a bundle of analog noise caused by reading, amplifying, transporting, (possibly amplifying again,) and digitizing the electron charge. This extra read noise is *NOT* in units of electrons; it is analog until digitization.
With customized circuitry, Canon has been able to get the level of read noises at the highest ISO down to the equivalent of a few electrons (not a few discreet electrons!). They have been able to do this at the *highest* amplification used in the cameras. Nothing that Roger writes on his website addresses what may or may not happen with more and better amplification; he simply jumps to the conclusion that nothing is gained, and uses the fact that his 1Dmk2 has the same total read noise in electrons at ISO 3200 as it does at ISO 1600. That is not any real support for his conclusion, because ISO 3200 *IS* ISO 1600 on that camera. Had he used a Minolta K7, which uses real amplification at ISO 3200, he would have measured slightly less noise at ISO 3200, and if he had actually looked at the shadows, there would be slightly less line noise at 3200, and less chromatic noise in a RAW at 3200 than 1600 pushed to 3200. I'd offer the 1Dmk3's ISO 3200 as additional support for my claim, but the fact that it is 14 bit may make you feel that the goal post for unity gain has moved (despite the fact that mk3 ISO 3200 quantized to 8 bits is still far less noisy than the mk2's ISO 3200).
I'm sure I have shown you this chart before, in previous refutations of the "unity gain" limit:
(http://www.pbase.com/jps_photo/image/65737967/original.jpg)
That is the total read noise, and the isolated horizontal and vertical line noises.
The total read noise, the yellow line, is scaled to 10% to fit in with the others, and the vertical axis is the read noise normalized to ISO 100 for all other ISOs, as standard deviation in ADUs (which can be considered arbitrary units of electrons). The noises clearly show no sign of flatlining completely by 3200, as far as the trends up to 1600 are concerned, especially the line noises, which are far more visible than their statistical strength suggests. Horizontal line noise *is*, without a doubt, the most troublesome aspect of high-ISO shadow areas in Canon cameras.
Roger has nothing to really support his unity gain hypothesis; he is simply applying the concept of one equals one, but these ones are really apples and oranges; one is discreet integer values, and the other is discreet multiples of a single value, with variance at a finer degree. The ADC in these cameras can *NOT* count electrons. They can only get so close to counting them, and by all appearances, with Canon's technology, the more you amplify the signal, the more you can reduce the inaccuracy, which is why it is illogical to declare that something as arbitrary as the ADU unit is a meaningful limit to practical amplification. And the ADU truly *is* arbitrary when it is fine enough not to cause posterization of RAW data. Only when it is coarse enough to cause posterization does the actual absolute meaning of the ADU have any value (the ability to posterize).
[a href=\"index.php?act=findpost&pid=125309\"][{POST_SNAPBACK}][/a]
I provided a link of John's post to Roger Clark and invited him to respond, and he was kind enough to provide the following explanation.
Hi Bill,
You can post this response if you wish to the luminous-landscape forum
as I have not registered to post there.
John Sheehy posted a plot of noise versus ISO for a Canon 20D.
The way I read the plot is that the noise at ISO 1600 is the
same as ISO 3200 and the noise is greater at ISO 800.
This is the same as other people's measurements.
The Unity Gain ISO for the 20D is 1200, so one would expect
a slight improvement from ISO 800 to 1600 (ISOs in between
these factor of 2 values are reportedly scaled and not true gains).
Amateur astronomers are pushing these cameras to their limits
and everyone I know has concluded that there is no benefit in
going to ISO 3200 (query the literally thousands of people
on the digital_astro yahoo group). Most are using ISO 1600
and some ISO 800. So I don't see John's plot in conflict with
the Unity Gain implications. I also have never seen a clear demonstration
that you could actually get more out of an image at ISO 3200
versus 1600 on the current suite of 12-bit/pixel cameras.
A number of amateur astronomers have tested this too and
come to this conclusion (that ISO 1600 "gets it all").
In fact you actually lose a stop of dynamic range in going from
ISO 1600 to 3200.
Regarding the 1D Mark III, part of the "read noise" is A/D converter
noise. The use of a 14-bit A/D reduces that noise and Canon reportedly
has a half to one stop better noise floor performance.
More importantly, Canon has reportedly reduced the fixed-
pattern noise (e.g. line noise). That improves the
perception of noise making the higher ISO images look
better. Until these cameras get into the hands of people
who run rigorous tests we really won't know the
implications of ISO, 14-bits, and noise floors.
Roger
The implications for shooting in very dim light are that it does not really make much sense to exceed the unity gain under these conditions. You get no more information and lose one f/stop of dynamic range. The unity gain of the Nikon D200 is 800 according to Roger's tests. According to the unity gain theory, when shooting in dim light with this camera in raw mode, it would be best to set the camera to ISO 800 rather than 1600. If there is enough light to expose at ISO 800, you get better dynamic range and less noise. If "underexposure" occurs at ISO 800, you merely use the exposure control of ACR (or whatever raw converter you are using) to brighten the image.
Bill
-
I provided a link of John's post to Roger Clark and invited him to respond, and he was kind enough to provide the following explanation.
Amateur astronomers are pushing these cameras to their limits
and everyone I know has concluded that there is no benefit in
going to ISO 3200 (query the literally thousands of people
on the digital_astro yahoo group). Most are using ISO 1600
and some ISO 800. So I don't see John's plot in conflict with
the Unity Gain implications. I also have never seen a clear demonstration
that you could actually get more out of an image at ISO 3200
versus 1600 on the current suite of 12-bit/pixel cameras.
A number of amateur astronomers have tested this too and
come to this conclusion (that ISO 1600 "gets it all").
In fact you actually lose a stop of dynamic range in going from
ISO 1600 to 3200.
He seems to have missed the point.
Perhaps you should ask him to re-read John's post, particularly the part about the ISO modes above 1600 on pre-1D MkIII Canon cameras.
-
I've read this topic over and over and only grasp about a quarter of what you folks are saying. And I keep asking myself, so what! Are you only discussing theory or is there some real world benefit to your debate.
[a href=\"index.php?act=findpost&pid=125381\"][{POST_SNAPBACK}][/a]
wmchauncey -- Here's the real deal: Current megapixel digital sensors are like film with no shoulder at all. When someone says "this looks digital" they really mean that the highlights are clipped hard. No one exposes film to clip highlights hard; they haven't for decades. I'm not sure why we're supposed to expose our digital sensors that way, even though they have the smoothest response right up near that wall of clipped highlights. So what if they are? Am I going to regret a minor amount of additional more noise by boosting my RAW file if doing so makes a scene look natural? No. I am not, my curators are not, and my customers are not. They will look at my photos and say, "wow."
The dumb thing is that most digicams push the metering right up near that wall so that signal/noise ratios are as low as possible (probably for marketing's sake) -- at the cost of hard clipped highlights. Well, I don't know about you, but I don't sell "signal/noise ratios". I sell pictures. And I've found that I can make the kind of pictures I want by shooting "underexposed" according to the camera, then "push-processing" the resulting RAW file. I get a little bit more noise, but a much more natural looking result.
(http://photi.ca/photos/167154168-L.jpg)
This is an extreme example. Exposed -2EV, shadows recovered up to 4 stops. Detail visible in the shadowed brush on the left and around the telegraph pole on the right. And the highlights on the rails look natural, and the sky isn't blown out, and it feels just like it felt when I was there -- a bright, clear, early morning.
Remember that Canon has added a "highlight tone priority" mode to the 1DmkIII -- which does exactly what I describe above. If you don't trust me, trust Canon. The era of hard clipped digital highlights is finally coming to an end. It's been a long time coming. And if you don't have a 1DmkIII, you can still do this by "underexposing" in camera, and "push-processing" in RAW.
-s
[edit -- removed unnecessary editorializing -s]
-
The implications for shooting in very dim light are that it does not really make much sense to exceed the unity gain under these conditions.
Really? Where is the proof for that?
You get no more information and lose one f/stop of dynamic range.
You get no more information BECAUSE THEY REALLY AREN'T TRUE AMPLIFIED 3200. How many times do I have to repeat this extremely relevant fact? Roger's conclusions, and those of the astronomers he's asked, are all based on ISO 3200s that are really ISO 1600s under-exposed, with the RAW values doubled and almost a stop of hightlights clipped away for no good reason.
The unity gain of the Nikon D200 is 800 according to Roger's tests. According to the unity gain theory, when shooting in dim light with this camera in raw mode, it would be best to set the camera to ISO 800 rather than 1600. If there is enough light to expose at ISO 800, you get better dynamic range and less noise. If "underexposure" occurs at ISO 800, you merely use the exposure control of ACR (or whatever raw converter you are using) to brighten the image.[a href=\"index.php?act=findpost&pid=125384\"][{POST_SNAPBACK}][/a]
What does that have to do with unity gain?
It is true for most Nikon cameras more than a year or two old, because most of the read noise occurs at the initial read on the sensor, and only a clean, low-gain amplifier is used to feed the ADC. Total read noise in electrons is very similar at all ISOs, and only varies because the absolute_signal-to-ADC_noise is different.
The same principle can apply between ISO 200 and 400, as well as 400 and 800 with cameras that do not have lower absolute (electron) noise at higher ISOs (like the Pentax K10D). Nothing special happening at unity gain. Even when the ADC noise makes a difference between ISOs, it makes the most difference between lower ISOs, explaining why there is less loss pushing 800 to 1600 than 200 to 400. The higher the noise before the ADC, the less increase there is in total noise from the ADC, because of the non-linear way in which noise sums.
Only poor circumstantial evidence exists for the unity gain theory. An ADU:electron ratio is only completely relevant if the total read noise is low enough so that no two quantities of electrons are digitized as one. That actually requires far greater than 1:1 with even 0.1 adu of analog read noise, for total accuracy in counting.
As far as 14 bits are concerned, they do nothing for IQ at ISO3200 with the mk3. Truncated to 8 bits, and then converted to RGB, the mk3 has less noise than the 1dmk2 with 12 bits. Even 12 bits at high ISOs is overkill for 99.99% of uses.
Roger's idea of testing this type of thing is to take a linear conversion, and then quantizing it. That is nothing at all like quantizing the RAW data, and then interpolating/demosaicing it and performing WB.
-
Modifying the contrast seems like a good idea. But in the final analysis, the camera manufacturers have let us down. There are not enough bins in the histogram and it comes from the jpeg, not the raw. Because there are so few bins in the histogram, its shape is poorly defined. This matters most in the highlights. In some case parts of the histrgram can be distorted becasue the bin-width is too large. One experiment for someone with ambition and too much time is to measure the in-camera histogram and compare its shape to the one in photoshop. It would be good to do this experiment for several lighting conditions
[a href=\"index.php?act=findpost&pid=125328\"][{POST_SNAPBACK}][/a]
I don't have unlimited time on my hands, but have done part of the experiment that you suggest with a Nikon D200. Illumination was 5000K. I took shots of a Stouffer step wedge, increasing the exposure in 0.33 EV increments until I observed clipping on the camera histogram with contrast set to normal. I then examined the contents of the raw files (converted with DCRaw) and the preview in Adobe Camera Raw. The displayed results are for the last exposure without clipping.
Here is the camera histogram. Note that the head of the wedge (which includes step 1) is just short of clipping and is the large spike towards the right. The small spike to the extreme right is blown background without the base density of the wedge.
[attachment=2711:attachment]
Here are the pixel values in the raw file (in 8 bit notation). Note that step 2 (down 0.3 EV) is not blown in any channel, whereas the green is in step 1 is near maximum.
[attachment=2712:attachment]
And this is how ACR views the file:
[attachment=2713:attachment]
Since the highlights are 255, I used the exposure control to decrease exposure until the value fell below 255, which took place at -0.2:
[attachment=2714:attachment]
With my particular camera, the camera histogram from the JPEG preview gives an accurate indication of clipping. For most exposures, I like to have the exposure just short of clipping. If the highlights are not critical and I want better shadow detail, then I may allow some highlight clipping and perform recovery in ACR.
Bill
-
Roger's idea of testing this type of thing is to take a linear conversion, and then quantizing it. That is nothing at all like quantizing the RAW data, and then interpolating/demosaicing it and performing WB.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=125401\")
John,
I don't really feel qualified to comment on the fine points of the work that you and Roger have done. At this point, I would look at the qualifications and background of the authors. Roger has a PhD in astrophysics from MIT and is professionally involved in various imaging projects at NASA and has 179 peer reviewed scientific papers ([a href=\"http://www.clarkvision.com/rnc/index.html]Bio RN Clark[/url]). What are your qualifications?
Bill
-
I don't really feel qualified to comment on the fine points of the work that you and Roger have done.
I see. Your role is to quote and link without any personal understanding.
At this point, I would look at the qualifications and background of the authors.
Really? A discerning reader would look at the logic and quality of the arguments, critically. Test these things for yourself. Why would you want to trust someone else for something you can test yourself?
Roger has a PhD in astrophysics from MIT and is professionally involved in various imaging projects at NASA and has 179 peer reviewed scientific papers (Bio RN Clark (http://www.clarkvision.com/rnc/index.html)).
That doesn't impress me to the same level it apparently does you. The world is full of people who fudge their way through their careers. Statistically speaking, he is likely to know more than someone who has not taken such a career path, but it does not license him to guaranteed truth.
I am impressed by quality of argument, and depth of thought. Do you think that there are no errors in 179 scientific papers? Do you think that no one objected to any of his ideas?
What are your qualifications?
[a href=\"index.php?act=findpost&pid=125415\"][{POST_SNAPBACK}][/a]
The quality of my arguments is my qualification. There is nothing mysterious, either in what I say. You can test most of it for yourself.
If you can't understand them, then go idolize an "expert" who jumps to conclusions.
-
I may allow some highlight clipping and perform recovery in ACR.
ACR does not do any highlights recovery. Correcting the Exposition down and therefore apparently recovering blown areas is not recovery at all, is simply compensate the blowing effect of white balance scaling, with the scaling provided by Exposition compensation to the left.
Your camera's histogram is pesimistic, as it will show blown areas where in the RAW file they have not still begin to blow. Reason? it's an histogram calculated on a white balanced RAW file using greater than 1.0 multipliers.
I see you use DCRAW to develop, just do this experiment: take the first RAW file in your series that started to appear blown on some channel in your camera0's histogram. Now develop it with -H 1 or better -H 2 options in DCRAW that ensure through smaller than 1 multipliers not to blow anything that was not already blown in the RAW file. Surely there will be nothing blown.
So if your camera's histogram shows nothing blown, no channel is. If it does, you don't know what you will find in the RAW file until you develop it in your PC.
-
ACR does not do any highlights recovery.
[a href=\"index.php?act=findpost&pid=125479\"][{POST_SNAPBACK}][/a]
Maybe we have to define what highlight recovery means.
ACR can build data in highlights IF one of the three color channels has data. If all three are blown out, nada.
-
I see. Your role is to quote and link without any personal understanding.
Really? A discerning reader would look at the logic and quality of the arguments, critically. Test these things for yourself. Why would you want to trust someone else for something you can test yourself?
That doesn't impress me to the same level it apparently does you. The world is full of people who fudge their way through their careers. Statistically speaking, he is likely to know more than someone who has not taken such a career path, but it does not license him to guaranteed truth.
I am impressed by quality of argument, and depth of thought. Do you think that there are no errors in 179 scientific papers? Do you think that no one objected to any of his ideas?
The quality of my arguments is my qualification. There is nothing mysterious, either in what I say. You can test most of it for yourself.
If you can't understand them, then go idolize an "expert" who jumps to conclusions.
[a href=\"index.php?act=findpost&pid=125476\"][{POST_SNAPBACK}][/a]
John,
Thus far all the data you have presented is one graph showing noise characteristics of a camera with results similar to those obtained by others. Then you draw conclusions that, in my view, are not supported by the graph. Any scientific paper has a materials and methods section where the experimental technique is described in sufficient detail to allow others to replicate the experiment and confirm the results. If you look at Roger's web site, he describes his methods in detail and gives references in the scientific literature to support his conclusions. Please direct us to an explanation of your work.
Science is not logic and the validity of a hypothesis can not necessarily be determined from the quality of the argument and the depth of thought. You need data and need to show how it was obtained.
I have worked through many of Roger's methods with my own camera. However, thus for you not described any methods that I can verify. Thus, while you seem to know quite a bit, you have less credibility with me than does Roger. Let other forum members draw their own conclusions. It they want to set their cameras to ISO 3200 and lose dynamic range.
Bill
-
So if your camera's histogram shows nothing blown, no channel is. If it does, you don't know what you will find in the RAW file until you develop it in your PC.
[a href=\"index.php?act=findpost&pid=125479\"][{POST_SNAPBACK}][/a]
If you have an RGB histogram, and nothing is blown in it, then probably nothing is blown in the RAW, but if the histogram is a luminance one, blown red and blue channels can easily be missed by the histogram, as they have weak luminance weighting.
-
ACR does not do any highlights recovery. Correcting the Exposition down and therefore apparently recovering blown areas is not recovery at all, is simply compensate the blowing effect of white balance scaling, with the scaling provided by Exposition compensation to the left.
[a href=\"index.php?act=findpost&pid=125479\"][{POST_SNAPBACK}][/a]
Whether what ACR does should be called recovery is a matter of semantics. With daylight white balance, the green channel can show severe clipping of the highlights when the blue and red channels are still intact. These tones in the green channel are lost and ACR does more than just scale back the green channel, but rather rebuilds it from intact data in the other channels using an algorithm to prevent color shifts as much as possible. This feat was demonstrated in the analysis I did on Digidog's overexposed file. Bruce Fraser and some other knowledgeable and articulate people have called this recovery. Your mileage may vary.
Your camera's histogram is pesimistic, as it will show blown areas where in the RAW file they have not still begin to blow. Reason? it's an histogram calculated on a white balanced RAW file using greater than 1.0 multipliers.
So if your camera's histogram shows nothing blown, no channel is. If it does, you don't know what you will find in the RAW file until you develop it in your PC.
[a href=\"index.php?act=findpost&pid=125479\"][{POST_SNAPBACK}][/a]
With Nikon cameras it is a simple matter to upload a custom white balance to the camera with the red and blue multipliers set to 1.0. The resulting RGB camera histogram gives a good preview of what is in the raw file without resorting to one's computer. If the camera has several banks in which to store camera settings, one of these can be used for the UniWB (coined by Julia Borg, a Nikon guru). This UniWB does mess up Adobe Camera Raw; when it sees Red and Blue multipliers set to 1.0, it thinks it is dealing with a multiple exposure. This problem can be addressed by making the coefficients nearly 1.
Bill
-
"Highlight recovery" is a usual term when talking about RAW developing, but it's most of the times wrong. What you get in ACR pushing down the exposure slider is not any highlight recovering at all. If your previously blown highlights become not blown through underexposure tuning, is because they were actually never blown on the RAW file. It was YOU in the developing process, commonly with the white balance (which implies heavy channel scaling by factors usually greater than 2.0 in linear, i.e. +1EV correction, and easily beyond 2.5, nearly +1.5EV), who blowed those areas.
ACR does not apply any real recovery of highlights. DCRAW in its -H2 to -H9 modes does; DCRAW interpolates truly blown pixels to values close to those in the boundaries; it "invents" colours where they were previously blown.
I just wanted to point this as most people talk about hightlight recorvery when thery are just simply referrering to "no-blowing underexposure".
[a href=\"index.php?act=findpost&pid=125372\"][{POST_SNAPBACK}][/a]
Let's put it this way. Blown highlights was more of a problem for me before I started using ACR several years ago. Canon's ZoomBrowser and othe popular RAW converters like BreezeBrowser could not do nearly as good a job as ACR in 'so-called' highlight recovery.
I remember being amazed at how much more detail in a grey sky I could recover using ACR. BreezeBrowser at that time could not compete, in my view. I don't know what BreezeBrowser is like now. I'm sure it has improved.
One RAW converter I still use is Raw Shooter Premium. I like the ease with which I can get solid, vibrant colors with almost a painterly effect, yet still retaining full detail. However, when it comes to recovering detail or color in a blown sky, ACR is marginally better than RSP.
It might well be true that DCRAW is even better at reconstructing lost data. I'm led to believe it is, from comments from John Sheehy and others and yourself.
But it's not a programs that's easy to use, is it? I get the impression it's a program that appeals more to computer programmers.
-
Whether what ACR does should be called recovery is a matter of semantics. With daylight white balance, the green channel can show severe clipping of the highlights when the blue and red channels are still intact. These tones in the green channel are lost and ACR does more than just scale back the green channel, but rather rebuilds it from intact data in the other channels using an algorithm to prevent color shifts as much as possible. This feat was demonstrated in the analysis I did on Digidog's overexposed file. Bruce Fraser and some other knowledgeable and articulate people have called this recovery. Your mileage may vary.
With Nikon cameras it is a simple matter to upload a custom white balance to the camera with the red and blue multipliers set to 1.0. The resulting RGB camera histogram gives a good preview of what is in the raw file without resorting to one's computer. If the camera has several banks in which to store camera settings, one of these can be used for the UniWB (coined by Julia Borg, a Nikon guru). This UniWB does mess up Adobe Camera Raw; when it sees Red and Blue multipliers set to 1.0, it thinks it is dealing with a multiple exposure. This problem can be addressed by making the coefficients nearly 1.
Bill
[a href=\"index.php?act=findpost&pid=125525\"][{POST_SNAPBACK}][/a]
Yes, this is what you can read almost everywhere about ACR, but I never saw an example where it was clear that ACR recreated a blown channel (mean blown even in the RAW data) with the information of the other two. "This feat was demonstrated in the analysis I did on Digidog's overexposed file"-> where exactly can I find this analysis? I would be interested in looking at it.
What DCRAW calls recovery, despite the results can be better or worse, is real interpolation of values in a channel according to values in the same channel but in non-blown surrounding pixels.
Something (very argueable) that makes me think DCRAW does something more than ACR is that if you activate recovery on DCRAW you clearly see an increment in the processing time (in fact DCRAW tells you when the recovery stage begins), while ACR always seems to take the same time to develop a RAW, no matter how much highlights were blown or not.
This is a sample of an image with important enough real blown areas (in the RAW data) in the green and blue channels:
(developed with -H 1 in DCRAW to avoid WB clipping and be aware of real blown areas):
(http://img243.imageshack.us/img243/8916/cascadapiedraox2.jpg) . (http://img214.imageshack.us/img214/38/cascadapiedrabmphisku7.gif)
(http://img243.imageshack.us/img243/6511/cascadaaguadj0.jpg) . (http://img156.imageshack.us/img156/9766/cascadaaguabmphisfu0.gif)
ACR produces (look at the water fall and the shiny rock) disgusting final tones. Shouldn'd it recover the blown channels from the red one?
On the other hand -H option in DCRAW produces a pleasant highlight recovery specially in the water fall. In this case -H 9 was used; with -H 6 the tone on the rock becomes more pleasant (less red).
ACR with -3EV exposure correction to minimise highlight blowing:
(http://www.foto-ceif.com/cpg/albums/userpics/10005/normal_IMG72473_ps.jpg)
Developed using DCRAW and -H 6 recovery option:
(http://img156.imageshack.us/img156/7342/cascadah6ib2.jpg)
The 1.0 EB multipliers on the Nikon is a cool option. With no doubt I would make use of it.
Best.
-
One RAW converter I still use is Raw Shooter Premium. I like the ease with which I can get solid, vibrant colors with almost a painterly effect, yet still retaining full detail. However, when it comes to recovering detail or color in a blown sky, ACR is marginally better than RSP.
Have you tried ACR 4.1 with either Photoshop Elements (4.01/Mac or 5.0/Windows), Photoshop CS3 or Lightroom 1.1?
Already in Lightroom 1.0, the "vibrance" slider seems somewhat similar to a Raw Shooter feature with the same name. I can't say whether it works in exactly the same way, because I stopped using RSE shortly after starting to use it.
(I'm not saying that ACR 4.1 replicates what you like about RSP, but I'm wondering whether it does.)
-
Already in Lightroom 1.0, the "vibrance" slider seems somewhat similar to a Raw Shooter feature with the same name. I can't say whether it works in exactly the same way, because I stopped using RSE shortly after starting to use it.
[a href=\"index.php?act=findpost&pid=125629\"][{POST_SNAPBACK}][/a]
Jani,
I have Lightroom 1.0 but don't use it, and I tried the CS3 demo until it expired. I'll eventually upgrade to CS3. I found the vibrancy slider very tame in ACR in CS3, however.
There's something about that combination of 'hot pixel/pattern noise suppression', 'noise suppression', 'detail extraction' and 'vibrance' in RSP that's difficult to emulate in ACR. There's a quality there that seems just right for some images, but not all images.
-
Yes, this is what you can read almost everywhere about ACR, but I never saw an example where it was clear that ACR recreated a blown channel (mean blown even in the RAW data) with the information of the other two. "This feat was demonstrated in the analysis I did on Digidog's overexposed file"-> where exactly can I find this analysis? I would be interested in looking at it.
[a href=\"index.php?act=findpost&pid=125575\"][{POST_SNAPBACK}][/a]
My analysis of Andrew's ETTR files was posted earlier in this thread. One can use the same images to demonstrate highlight recovery in ACR.
Here is a histogram of the green channel of Andrew's image taken at 1/50 sec @ f/16 from the white portion of the target. The image was rendered with DCRaw and the histogram is from ImageJ, which supports 16 bit histograms. The highlights are totally blown.
[attachment=2730:attachment]
Here is the histogram after Andrew's highlight recovery. It is apparent that the green channel has been reconstructed in the highlight area.
[attachment=2731:attachment]
Bill
-
The quality of my arguments is my qualification. There is nothing mysterious, either in what I say. You can test most of it for yourself.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=125476\")
I agree that a reliance on popularity can be misleading. But the fact is that arguments are not enough. Science is based on data. I've read through [a href=\"http://luminous-landscape.com/forum/index.php?showtopic=17706&st=37#]your data post[/url] a few times and must admit I have a hard time deciphering it to reconcile with your conclusions (despite a medical degree and a neurobiology PhD that was founded on digital data acquisition).
Maybe you could expand on the data you collected and explain (or at least link to) how you reached the arguments that you now propound.
-
I agree that a reliance on popularity can be misleading. But the fact is that arguments are not enough. Science is based on data.
Without *any* data from myself, I can still challenge Roger Clarks' conclusions, based on his data. His data does not support his conclusion, IMO, except as weak circumstantial evidence. He sees a tapering off in improvement in read noises as measured in electrons, going from ISO 800 to 1600, and then no improvement at all for ISO 3200 (no surprise there, as the camera doesn't really have an ISO 3200 amplification). Since 1 electron is approximately 1 ADU somewhere just below ISO 1600, he concludes that it is because of "unity gain", a point past which there is no further gain from more amplification. He supports this with "logic" that says that it makes no sense to count beyond unity gain, because electrons are discreet units and you only need one unit to count one. That logic would be fine, if cameras were actually counting photons-turned-electrons, and I wish that were true, but THEY ARE NOT COUNTING ELECTRONS. They are digitizing an analog amplification of the discreet electron counts, blanketed in camera-generated noises with an infinite number of in-between analog values from at least a couple of sources before the signal becomes binary data.
The variance due to read noises in apparent electron counts is tremendous at the ISOs near "unity gain". The situation is not even remotely close to "counting electrons" with any degree of accuracy, at the pixel level. Here is a histogram of a blackframe from the 20D at ISO 1600:
(http://www.pbase.com/jps_photo/image/81688868/original.jpg)
The variance here is due to read noises picked up throughout the signal chain, including noise introduced by the ADC unit. The noise introduced by the ADC unit, alone, would be cut in half, relative to signal, if there were a true gain-based ISO 3200.
I've read through your data post (http://luminous-landscape.com/forum/index.php?showtopic=17706&st=37#) a few times and must admit I have a hard time deciphering it to reconcile with your conclusions (despite a medical degree and a neurobiology PhD that was founded on digital data acquisition).
I can only guess what you're refering to, without your being specific. I have said many things.
The point of my chart, if that is what you are refering to, is that the values at ISO 3200 are *NOT* a natural progression of the curves from ISO 100 through ISO 1600. The curve for "total noise" from 100 to 1600 is the closest one to actually suggesting the ISO 3200 value, but the 3200 value is still an abrupt flattening of the curve by a small degree. Everyone who has ever tried to use the shadows of Canon DSLRs at high ISOs knows that one of the most visible aspects of noise is the banding or line noise. Its statistical strength is very small, compared to its visible strength. Subtracting the banding component from a blackframe decreases the standard deviation by almost nothing. With my 20D, it lowered the standard deviation at ISO 1600 from about 4.7 to about 4.6, yet the visible improvement is much more than the numbers would suggest.
Maybe you could expand on the data you collected and explain (or at least link to) how you reached the arguments that you now propound.
[a href=\"index.php?act=findpost&pid=126507\"][{POST_SNAPBACK}][/a]
It's more a matter of *not* jumping to the conclusions that Roger Clark does, IMO. My interpretation of his data and mine combined suggest to me that it may be possible to get less read noise in electrons with greater and higher quality amplification. None of the data contradicts that possibility. The "unity gain" theory totally ignores the fact that the real limitation to low signal quality is camera-generated read noises, which have been demonstrated to decrease, relative to absolute signal, at higher amplifications and with improved technologies. It is currently of no practical value, whatsoever. If cameras *were* actually counting electrons, then of course, there would be no point in amplifying to an extent, or digitizing to such a depth, where each electron was represented by more than 1 ADU, but this is the real world, and read noises are a real obstacle to counting electrons.
-
The variance due to read noises in apparent electron counts is tremendous at the ISOs near "unity gain". The situation is not even remotely close to "counting electrons" with any degree of accuracy, at the pixel level. Here is a histogram of a blackframe from the 20D at ISO 1600:
(http://www.pbase.com/jps_photo/image/81688868/original.jpg)
The variance here is due to read noises picked up throughout the signal chain, including noise introduced by the ADC unit. The noise introduced by the ADC unit, alone, would be cut in half, relative to signal, if there were a true gain-based ISO 3200.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=126517\")
Your histogram of the dark frame (bias frame) is similar to that determined by others, for example [a href=\"http://www.astrosurf.org/buil/20d/20dvs10d.htm]Christian Buhl[/url]. However, the bias frame does not represent read noise, since it also includes fixed pattern noise derived from non-uniformity of the light field (e.g. due to vignetting in the lens), dust specs on the sensor, pixel to pixel sensitivity variations and other effects. To get the true read noise, you must subtract the fixed pattern noise as Christian points out in the above link. The horizontal and vertical banding noise on which you dwell is one type of fixed pattern noise. Since you do not describe your methods, I have no idea of what you are doing and whether or not any of your conclusions are valid.
The primary component of read noise is is from the on chip amplifier (see here, read noise (http://www.photomet.com/library/library_encyclopedia/library_enc_signal.php)). If you increase amplification, this noise would most likely increase also and you might not gain much. The noise introduced by the A/D converter is relatively minor and can be reduced by increasing the bit depth of the A/D converter (as Roger points out for the new EOS 1D M3, which uses a 14 bit device).
The relative contributions of photon noise to read noise at unity gain vary with signal strength. With higher signals, the noise source is mainly photon noise, while read noise predominates at lower signal strength. You did not state what signal strength was involved in your statement about read noise at unity gain. Of course, with a dark frame with a short integration time, the noise is almost entirely read noise, since there would be no photon noise and thermal noise would be minimal.
To use Roger's data for the 1DM2, full well at ISO 50 is 80,000 electrons. At ISO 1600 (unity gain), full scale on the A/D converter would represent 2500 electrons. The variance of photon noise would be 2500 and the variance due to read noise would be 15.2. If you look at noise in the shadows, for example 6 EV down, one would collect about 40 electrons. The variances of the photon noise and read noise would be 40 and 15.2 respectively. Photon noise still predominates.
Furthermore, when one determines read noise, one is measuring total noise and the various contributing components can not be separated out, and your statement "The noise introduced by the ADC unit, alone, would be cut in half, relative to signal, if there were a true gain-based ISO 3200" does not make much sense since you can't separate out amplifier noise from ADC noise.
Finally, the proof of the pudding is in the eating, and you should measure noise at various ISOs rather than theorizing at what they might be. Astronomers who have done their homework apparently do not bother with amplification much above unity gain, since it has no advantage with respect to noise and merely reduces dynamic range. The unity gain of the Canon cameras is relatively high, and the differences might be demonstrated more easily with a camera with a lower unity gain (such as the Nikon D200 with a unity gain of 800).
-
My analysis of Andrew's ETTR files was posted earlier in this thread. One can use the same images to demonstrate highlight recovery in ACR.
Here is a histogram of the green channel of Andrew's image taken at 1/50 sec @ f/16 from the white portion of the target. The image was rendered with DCRaw and the histogram is from ImageJ, which supports 16 bit histograms. The highlights are totally blown.
[attachment=2730:attachment]
Here is the histogram after Andrew's highlight recovery. It is apparent that the green channel has been reconstructed in the highlight area.
[attachment=2731:attachment]
Bill
[a href=\"index.php?act=findpost&pid=126206\"][{POST_SNAPBACK}][/a]
mmmm I see, you are totally right. ACR recovered both the red and green channels from the blue one. I thought highlight recovery in ACR was a fake from Adobe, glad to see I was wrong.
Thanks for the info Bill.
-
Your histogram of the dark frame (bias frame) is similar to that determined by others, for example Christian Buhl (http://www.astrosurf.org/buil/20d/20dvs10d.htm). However, the bias frame does not represent read noise, since it also includes fixed pattern noise derived from non-uniformity of the light field
"Light field"? Hello?
(e.g. due to vignetting in the lens),
Vignetting in the lens? Hello?
dust specs on the sensor,
How does the sensor know that the specs are there? Are they telepathic?
pixel to pixel sensitivity variations and other effects
Sensitivity to what? Do you know what a "blackframe" is? It is the noise of having to read a sensor, with no light having exposed it.
To get the true read noise, you must subtract the fixed pattern noise as Christian points out in the above link.
No, the read noise of a black frame is the noise of the blackframe. The *RANDOM* component of a blackframe, is the blackframe minus any fixed pattern noise. The Canon 20D has no significant fixed pattern noise in 1/8000 second black "exposures". Stacking 16 of them yields 1/4 of the statistical noise, and no non-random patterns appear. I've taken 16 exposures interleaved with 16 blackframes, stacked all the exposures together, stacked all the blacks together, and subtracted the black result from the exposure result and all I got was a 40% increase in random noise. It takes a couple of seconds at ISO 1600 for fixed-pattern noise to become measurably higher.
The problem with your understanding is that you are not familiar enough with these issues to have a clear picture in your head of what the terms are referring to.
You see me write the word "blackframe" and the only thing you can think of is astrophotography, with its long exposures, and reply as if I were talking about stacking many long exposures, subtracting fixed-pattern noises, and flat fields. None of that has anything directly to do with how much amplification is worth using for readout. In fact, if other noises were present which I needed to subtract out, then the curves in my 20D noise profile graph would be *STEEPER*, not flatter, as the curves approached ISO 1600. Do you understand that? Noises in the sensor itself have the same intensity in "electrons" at all ISOs; unsubtracted fixed pattern noises would compress the differences in measured noise from single frames at different ISOs, as measured in electrons!
Roger often mistakes line noise for fixed-pattern noise; the line noises I speak of *DO NOT* repeat from frame to frame; they are in a different place with a different pattern, in *EVERY* frame. No matter how many times I have told this to him, he goes on again to call the line noises "fixed pattern" noises. They are random 1-dimensional noises. In some cases, there is a periodic component, but the phases are totally random.
The horizontal and vertical banding noise on which you dwell is one type of fixed pattern noise. Since you do not describe your methods,
I have described many times to you what I do. I take blackframes (NOTHING TO DO WITH ASTRONOMY!) at a short exposure at all ISOs, load each RAW file into IRIS, window the display to 0 to 255 ADU, save out into a BMP, load into photoshop. I get the standard deviation of the blackframe from the histogram tool. That is the "total read noise". Then I make a copy of the blackframe, and resize it with bicubic so that it is one pixel wide. Then, I resize it again to the original width (not really necessary, but I may want to subtract it from the original), and get the standard deviation. This is the standard deviation of the horizontal line noise. Do the same vertically for the vertical line noise. Very simple. I would have included noise data for the original with the line noises subtracted, but they are almost exactly the same as the original, and would not be distinct on the graph.
I have no idea of what you are doing and whether or not any of your conclusions are valid.
The primary component of read noise is is from the on chip amplifier (see here, read noise (http://www.photomet.com/library/library_encyclopedia/library_enc_signal.php)).
"The major component of readout noise arises from the on-chip preamplifier."
That's accurate with older CCD cameras. The cameras Roger draws his conclusions from are not older, or CCD.
If you increase amplification, this noise would most likely increase also and you might not gain much.
We are discussing at what point it becomes worthless. Saying that it will become worthless at some point is stating the obvious. I am saying that there is no evidence that read noise can't be reduced further than it is now by higher and/or better amplification.
The noise introduced by the A/D converter is relatively minor and can be reduced by increasing the bit depth of the A/D converter (as Roger points out for the new EOS 1D M3, which uses a 14 bit device).
There is no evidence of this happening in the 1Dmk3. The Ratio of max RAW signal to read noise is almost exactly the same on the mk2 and the mk3 at ISO 100. Low ISOs are the ones with the largest percentage of their noise contributed by the ADC, and ISO 100 has no improvement. That means that it is likely that there is no real improvement in ADC-contributed noise in the mk3. ISO 3200 on the mk3 has a read noise of 6.0 ADU compared to 9.4 ADU on the mk2, but the benefits are fully there even if the RAW data is quantized to 12 or even 10 bits.
-
"Light field"? Hello?
Vignetting in the lens? Hello?
How does the sensor know that the specs are there? Are they telepathic?
Sensitivity to what? Do you know what a "blackframe" is? It is the noise of having to read a sensor, with no light having exposed it.
No, the read noise of a black frame is the noise of the blackframe. The *RANDOM* component of a blackframe, is the blackframe minus any fixed pattern noise.
The problem with your understanding is that you are not familiar enough with these issues to have a clear picture in your head of what the terms are referring to.
[a href=\"index.php?act=findpost&pid=127184\"][{POST_SNAPBACK}][/a]
Your post is very difficult to read, since you did not get the quote/unquotes right.
Anyway, I will respond to a few of your points. If you are determining the total noise in an image, you need to take the shot noise, read noise, and dark current (thermal noise) into account. With short exposures (up to several seconds), the dark current is negligible and may be ignored. You dwell on read noise, but photon noise is dominant in most situations. Photon noise may not be a wild card, but it needs to be taken into account. If if overshadows read noise by a large margin in a particular situation, read noise can be neglected for practical purposes, just as we are ignoring thermal noise.
To analyze the photon noise, you need some light falling on the sensor. If you merely take a picture of a supposedly uniform field, the light falling on the sensor may vary due to the factors I mentioned (taken from Roger's web site). These factors are fixed, and can be eliminated by subtracting images taken under identical conditions as Roger explains. For read noise, lens vignetting, dust on the sensor, etc do not apply since the image is taken in the dark with the lens cap on (and I should not have mentioned them in the context of read noise).
If you are analyzing a black frame for read noise (yes, I know they are different from a dark frame to determine the dark current in the long exposures used in astronomy), you must subtract fixed pattern noise. Your understanding of read noise is different from what Roger uses or from Christian Buhl's concept: "Subtraction of an offset image and of the fixed pattern structure for the EOS 10D. It is the true readout noise "image" of the camera." You allege that I don't have a clear concept of the terms involved, but you make up definitions on your own.
I have described many times to you what I do. I take blackframes (NOTHING TO DO WITH ASTRONOMY!) at a short exposure at all ISOs, load each RAW file into IRIS, window the display to 0 to 255 ADU, save out into a BMP, load into photoshop. I get the standard deviation of the blackframe from the histogram tool. That is the "total read noise". Then I make a copy of the blackframe, and resize it with bicubic so that it is one pixel wide. Then, I resize it again to the original width (not really necessary, but I may want to subtract it from the original), and get the standard deviation. This is the standard deviation of the horizontal line noise. Do the same vertically for the vertical line noise. Very simple. I would have included noise data for the original with the line noises subtracted, but they are almost exactly the same as the original, and would not be distinct on the graph.
[a href=\"index.php?act=findpost&pid=127184\"][{POST_SNAPBACK}][/a]
This is the first time I have seen an explanation of your methods in such detail. I don't know why you save in 8 bit format, since you lose precision. I multiply the results by 16 to convert 12 bit data to 16 bit format. The standard deviation of the black frame is not the read noise, since it includes fixed pattern noise. To get read noise, you must subtract a bias frame as Christian explains. That is also what Roger does.
You've just described my blackframes! Why all the confusion at the beginning of your reply?
[a href=\"index.php?act=findpost&pid=127184\"][{POST_SNAPBACK}][/a]
You jumped to the wrong conclusion.
A chain is no stronger than its weakest link, and the red channel is 1 stop less sensitive in daylight WB, so Sat-6EV is only 1.5 stops below "middle red". Change the WB to tungsten, and the blue channel is only 0.5 stops below "middle blue", with Sat-6EV!
[a href=\"index.php?act=findpost&pid=127184\"][{POST_SNAPBACK}][/a]
Yes, I was using only the green channel to simplify things. A full analysis would require all channels to be evaluated.
The bottom line, which you seem to ignore, is that the steps of the ADU are arbitrary relative to the analog equivalent of an electron.
[a href=\"index.php?act=findpost&pid=127184\"][{POST_SNAPBACK}][/a]
Not really. The ADU is related to the number of electrons as described by the camera gain, which is in units of electrons/ADU.
-
What do you guys think of this?
[attachment=2848:attachment]
Top, minus 1 stop over flash meter recommendation (ISO 400)
Middle Normal
Bottom Plus 1.5 stops, normalized with exposure slider in Lightroom (-1.46).
Oh, I should add, small section of a big image with many different targets. New images, shoot with strobe (very even lighting according to flash meter). Did ISO 100/400 and 800 (this is but one small example). Bracketed with strobe back NOT aperture (that affected the results too much).
On my 5D, 2 stops over was blown out, couldn't recover less than 100% on whites with exposure slider in LR. 1.5, over, no problem.
-
What do you guys think of this?
[attachment=2848:attachment]
Top, minus 1 stop over flash meter recommendation (ISO 400)
Middle Normal
Bottom Plus 1.5 stops, normalized with exposure slider in Lightroom (-1.46).
[a href=\"index.php?act=findpost&pid=129007\"][{POST_SNAPBACK}][/a]
Andrew,
I've always found ACR can do a spectacular job recovering greys. A grey sky or a grey concrete wall with fine texture seems to be able to accept more exposure before clipping than a scene with color. A particularly common indication of clipping is a deep blue sky that shows an obvious shift towards cyan.
It was mentioned earlier in the thread that ACR always applies a white balance to the conversion and that this apparently is responsible for a degree of clipping. There's no option, as far as I see, for 'no color balance' in ACR.
I'm not sure if the following is relevant, but recently whilst experimenting with a calibration of my scanner with an old IT8 target using Vuescan software, I noticed a dramatic change in the histogram when selecting 'no color balance'.
The following 2 previews of a slightly overexposed Ektachrome compare 'no color balance' with 'white balance'. Whatever characteristic of WB I select, neutral or landscape etc, the histogram is always pushed to the extreme right, with this particular overexposed slide, but with no WB the histogram is very far from the right.
[attachment=2852:attachment] [attachment=2853:attachment]
-
This is the second half of a reply I made last week; for some reason, the Quote feature won't work with the entire post, but does broken in into two halves. Apparently there is a limit on the number of Quote delimiters in the software.
The relative contributions of photon noise to read noise at unity gain vary with signal strength. With higher signals, the noise source is mainly photon noise, while read noise predominates at lower signal strength. You did not state what signal strength was involved in your statement about read noise at unity gain.
Perhaps that's because it is irrelevant. Shot noise is not a wild card; shot noise is dependent only on the geometry of photon collection and quantum efficiency, including the effects of filters. Read noise is the thing that varies greatly from one camera to another, and its relationship to ISO varies greatly on any given camera. None of the things that you do to improve read noise have any effect on shot noise. The less read noise you have, the more usable your shadows become.
Of course, with a dark frame with a short integration time, the noise is almost entirely read noise, since there would be no photon noise and thermal noise would be minimal.
You've just described my blackframes! Why all the confusion at the beginning of your reply?
To use Roger's data for the 1DM2, full well at ISO 50 is 80,000 electrons. At ISO 1600 (unity gain), full scale on the A/D converter would represent 2500 electrons.
No; that's not Roger's data. Roger is well aware (pun intended) that RAW saturation at ISO 100 is not half as many photons as well saturation at ISO 50. ISO 50 is really only capable of about ISO 70 to 75 on the 1Dmk2, but is exposed like it is actually ISO 50, leaving less highlight headroom than the other ISOs, relative to metering (a major engineering blunder, IMO). RAW saturation at ISO 1600 on the 1Dmk2 is approximately 52300 (by his data) divided by 16 = 3269 electrons. I don't know if his values are correct, though, because he may assume that the 1Dmk2 has 4095 RAW levels to use, but signal is only represented by ADU 128 through about 3700+. He seems to derive either ADU:electron "gain" or max photon count from the other, using the uncertain figure.
The variance of photon noise would be 2500 and the variance due to read noise would be 15.2. If you look at noise in the shadows, for example 6 EV down, one would collect about 40 electrons. The variances of the photon noise and read noise would be 40 and 15.2 respectively. Photon noise still predominates.
No. Six stops down from saturation would be 3269/(2^6) = 51.1 electrons, which would have a shot noise of 7.15 electrons. Total read noise is about 4.19 electrons. Read plus shot would be (51.1 + (4.19)^2)^0.5 = 8.29 electrons, or about 0.21 stops more noise. How far down is Sat-6EV, though? There is about 3.5 stops of headroom above metered grey in most Canon DSLRs (4.0 for the XTi). That puts Sat-6EV about 2.5 stops below middle grey; not very far into the shadows at all, and that's just the green channel. A chain is no stronger than its weakest link, and the red channel is 1 stop less sensitive in daylight WB, so Sat-6EV is only 1.5 stops below "middle red". Change the WB to tungsten, and the blue channel is only 0.5 stops below "middle blue", with Sat-6EV!
Now, let's talk about some *real* (but still not extreme) shadows: 4 stops below middle blue in tungsten lighting - that's Sat-9.5EV. 3269/2^9.5 is 4.51 electrons. That's a shot noise of 2.2, against a read noise of 4.19. Total noise is (4.51+(4.19)^2) = 4.7, or 1.09 stops more total noise.
Not to mention the fact that the read noise has highly-perceptible line noises not present in shot noise of equal intensity.
Furthermore, when one determines read noise, one is measuring total noise and the various contributing components can not be separated out, and your statement "The noise introduced by the ADC unit, alone, would be cut in half, relative to signal, if there were a true gain-based ISO 3200" does not make much sense since you can't separate out amplifier noise from ADC noise.
It is not clear how much of the noise is due to the ADC, but ADC noise is real, and whatever part it is, would be reduced 50% relative to signal, a small, but real improvement. Total noise at ISO 3200 would be (non-ADC-1600-noise^2+(ADC-1600-noise/2)^2)^0.5. You don't even need any more at-sensor gain to get this small benefit, so the issue of the point of diminishing returns of sensor-read amplification is another issue. Some manufacturers use amplification of some kind for ISO 3200; the Minolta K7 a case in point; it's ISO 3200 has less noise than 1600 under-exposed by a stop.
Finally, the proof of the pudding is in the eating, and you should measure noise at various ISOs rather than theorizing at what they might be.
I did; what are you talking about?
Astronomers who have done their homework apparently do not bother with amplification much above unity gain, since it has no advantage with respect to noise and merely reduces dynamic range.
That's a very vague statement, and it may apply to old technology, and/or equipment that doesn't have any gain above unity gain, because the manufacturers believed the same myth, or technology wasn't ready.
The bottom line, which you seem to ignore, is that the steps of the ADU are arbitrary relative to the analog equivalent of an electron.
The unity gain of the Canon cameras is relatively high, and the differences might be demonstrated more easily with a camera with a lower unity gain (such as the Nikon D200 with a unity gain of 800).
[a href=\"index.php?act=findpost&pid=126528\"][{POST_SNAPBACK}][/a]
Go ahead, demonstrate it, and I will offer alternate explanations for your results.
-
Your post is very difficult to read, since you did not get the quote/unquotes right.
It is fixed now. I had to break the post in half for the Quote feature to work properly. I did have a pair of quote delimiters reversed originally, but even after making sure they were all correct, it still wouldn't post correctly.
Anyway, I will respond to a few of your points. If you are determining the total noise in an image, you need to take the shot noise, read noise, and dark current (thermal noise) into account.
I was determining the total noise of a blackframe at a short exposure. There is no shot noise, and there is infinitessimal dark current noise in a short exposure.
With short exposures (up to several seconds), the dark current is negligible and may be ignored. You dwell on read noise, but photon noise is dominant in most situations.
Photon noise is dominant in the highlights, and read noise is dominant in the shadow areas. Read noise is dominant in determining ultimate DR.
Photon noise may not be a wild card, but it needs to be taken into account. If if overshadows read noise by a large margin in a particular situation, read noise can be neglected for practical purposes, just as we are ignoring thermal noise.
Dark current noise is infinitessimal in a short exposure, compared to read noise.
To analyze the photon noise, you need some light falling on the sensor.
To measure read noise in a blackframe? Blackframe read noise is a noise that has no relationship to signal, whatsoever, other than the fact that clipping the signal will also clip away the read noise (and any other noise).
If you merely take a picture of a supposedly uniform field, the light falling on the sensor may vary due to the factors I mentioned (taken from Roger's web site). These factors are fixed, and can be eliminated by subtracting images taken under identical conditions as Roger explains. For read noise, lens vignetting, dust on the sensor, etc do not apply since the image is taken in the dark with the lens cap on (and I should not have mentioned them in the context of read noise).
If you are analyzing a black frame for read noise (yes, I know they are different from a dark frame to determine the dark current in the long exposures used in astronomy), you must subtract fixed pattern noise. Your understanding of read noise is different from what Roger uses or from Christian Buhl's concept: "Subtraction of an offset image and of the fixed pattern structure for the EOS 10D. It is the true readout noise "image" of the camera." You allege that I don't have a clear concept of the terms involved, but you make up definitions on your own.
This is the first time I have seen an explanation of your methods in such detail. I don't know why you save in 8 bit format, since you lose precision..
Only the 8 LSBs are used in a 20D blackframe. You could probably count on one hand the number of pixels above 255 ADU in a blackframe from a 12-bit RAW.
BMP is the only lossless format that IRIS outputs without loss in a file format understandable and unmangled by photoshop.
I multiply the results by 16 to convert 12 bit data to 16 bit format.
The standard deviation of the black frame is not the read noise, since it includes fixed pattern noise.
In the Canon 20D, fixed pattern noise is infinitessimal in short exposures. That is one reason why Canons are so relatively noise-free in the shadows at high ISOs; the readout circutry is aware of any *consistent* significant offsets from black.
To get read noise, you must subtract a bias frame as Christian explains. That is also what Roger does.
Does for what?
I've been playing with Canon RAW data long enough to know that there is no fixed pattern noise in any short exposure that has any visible effect, even in very under-exposed images pushed.
You jumped to the wrong conclusion.
Yes, I was using only the green channel to simplify things. A full analysis would require all channels to be evaluated.
Not really. The ADU is related to the number of electrons as described by the camera gain, which is in units of electrons/ADU.
[a href=\"index.php?act=findpost&pid=127288\"][{POST_SNAPBACK}][/a]
You can measure any two quantities and calculate their ratio, and give that ratio a name. That doesn't mean, however, that there is some special significance to "Unity" .
Here are some parallels:
"It's not worth driving at a speed of less than a mile a minute; you won't get anywhere that way."
"All resistances in the circuit less than 1 ohm should be removed, as they offer no resistance".
I could go on and on, but the point is that there is no empirical relationship between ADU steps and amplified electron counts clouded by analog read noise. All you would need to do is change the bit depth of the camera's RAW output to determine the maximum usable amplification at the initial readout. The 1Dmk3 does just that; it gives a 4x boost to the "ADU:electron gain". If "unity gain" was at ISO 1200 on the 1Dmk2, then it is now at ISO 300 on the 1Dmk3! If you think about nothing else at all, think about that! By the same token, just dropping bits off of any ADC would make it worthwhile to use greater amplification at the inital read!
There is nothing about the amplification levels at the photosite (CMOS) that has anything directly to do with electrons per se, nor with the gain of the ADU. Somehow you think that the camera knows about electrons, and their signal magnitude. The ADC is only taking an analog signal which has no spiking in its virtual analog histogram, and digitizing it. In order for the original signal to be preserved as best as possible, the noise needs to be so small that individual electron counts would have spikes; then we could talk about actually counting electrons. Since we can't count electrons, what we need is amplification that gives the least noise, no matter how many ADU that turns out to be.
Unity gain is an arbitrary ratio with current levels of analog noise.
-
I could go on and on, but the point is that there is no empirical relationship between ADU steps and amplified electron counts clouded by analog read noise.
Here are some parallels:
"It's not worth driving at a speed of less than a mile a minute; you won't get anywhere that way."
"All resistances in the circuit less than 1 ohm should be removed, as they offer no resistance".
[a href=\"index.php?act=findpost&pid=129173\"][{POST_SNAPBACK}][/a]
Please don't, since your parallels are not particularly well chosen
All you would need to do is change the bit depth of the camera's RAW output to determine the maximum usable amplification at the initial readout. The 1Dmk3 does just that; it gives a 4x boost to the "ADU:electron gain". If "unity gain" was at ISO 1200 on the 1Dmk2, then it is now at ISO 300 on the 1Dmk3! If you think about nothing else at all, think about that! By the same token, just dropping bits off of any ADC would make it worthwhile to use greater amplification at the inital read!
[a href=\"index.php?act=findpost&pid=129173\"][{POST_SNAPBACK}][/a]
You do not seem to understand the concept of unity gain. As Roger explained, if you use unity gain to compare the sensitivity of sensors, you must express it in the same bit depth.
Yes, with a unity gain at ISO 300 with the 1D M3 you could leave the ISO at 300 and shoot in dim light without any need to adjust the ISO to a higher value. If you increased the bit depth to 17, you wouldn't even need to adjust the ISO at all, since you would have captured all the information at base ISO. This assumes that the read noise at base ISO is kept under control. With the 1DM2, Roger's data show that read noise increases considerably at at base ISO as compared to ISO 1600. With the Nikon D200, the increase is less marked (up to 10 electrons from 7.4).
-
First off, I'd like to point out that John Sheehy is absolutely correct in that the camera sensors mentioned by Mr. Clark do not count individual electrons.
Counting single electrons is so hard that it was first done just over two years ago by Swedish physicists. (http://physicsweb.org/articles/news/9/3/11)
If John Sheehy doesn't understand the concept of unity gain, it may be because it's practically impossible to have a setting in the camera where it can count single electrons.
If bjanes wants to prove Roger Clark right (or he wants to do so himself), then my guess is that there's a Nobel Prize in physics waiting; doing this at room temperature would be a significant leap forward.
-
If bjanes wants to prove Roger Clark right (or he wants to do so himself), then my guess is that there's a Nobel Prize in physics waiting; doing this at room temperature would be a significant leap forward.
[a href=\"index.php?act=findpost&pid=129357\"][{POST_SNAPBACK}][/a]
I expect lots of density in discussion of things like politics or religion, but stuff like this is puzzling. How can anyone, aware that the noise in the cleanest of systems varies in a smooth, analog manner over a couple dozen electrons, think that counting of electrons is occuring.
On top of the lack of any logic in the concept of unity gain as a limit in a noisy analog system, the 1Dmk3 totally contradicts the concept of unity gain. Even if all ADUs must be converted to 12-bit to qualify, the 1Dmk3 at ISO 3200 has an ADU_12:electron ratio ("gain") of 2.18; far above "unity gain", yet the amplification is immensely useful.
-
Please don't, since your parallels are not particularly well chosen
All they are chosen for is their demonstration of how arbitrary unity can be. They are good examples.
You do not seem to understand the concept of unity gain.
I do understand it. I understand it to be false and simplistic.
As Roger explained, if you use unity gain to compare the sensitivity of sensors, you must express it in the same bit depth.
So, what's the magic bit depth? 12, because it's one of God's favorite numbers?
Maybe we can get back to ancient Greek cosmologies while we're at it; Pythagorean harmony between bits and electrons.
Yes, with a unity gain at ISO 300 with the 1D M3 you could leave the ISO at 300 and shoot in dim light without any need to adjust the ISO to a higher value. If you increased the bit depth to 17, you wouldn't even need to adjust the ISO at all, since you would have captured all the information at base ISO. This assumes that the read noise at base ISO is kept under control. With the 1DM2, Roger's data show that read noise increases considerably at at base ISO as compared to ISO 1600.
The point that you and Roger seem to ignore is that it is the increased amplification at the photosites that reduces the noise in electron units.
Obviously, just counting electrons would be the ideal; ISO would only be a way to tell the camera how to expose, and how bright to make the review image or the default JPEG.
With the Nikon D200, the increase is less marked (up to 10 electrons from 7.4).
[a href=\"index.php?act=findpost&pid=129247\"][{POST_SNAPBACK}][/a]
You could double-check yourself. Just shoot short blackframes and load them into IRIS, and issue the "stat" command. Look at the Sigma value. That is the standard deviation of clipped black. To get the true noise, multiply by 1.725, which will result in what a symmetrical, unclipped standard deviation would be.
Take each of these values, and multiply them by the number of maximum electrons at RAW saturation, and divide by 4095 (yes, the D200 uses all 4096 values). I don't know the exact saturation of the D200, but I would guess about 30000 electrons at base ISO. The correct number is not required to compare ISOs, though.
-
To get the true noise, multiply by 1.725, which will result in what a symmetrical, unclipped standard deviation would be.
What is the origin of this number?
-
What is the origin of this number?
[a href=\"index.php?act=findpost&pid=129536\"][{POST_SNAPBACK}][/a]
I took a bunch of symmetrical gaussian noise samples (and fabrications in PS), measures their standard deviations, clipped them in the center, and measured again. 1.725 was typical of the ratios in the useable range.
-
With the Nikon D200, the increase is less marked (up to 10 electrons from 7.4).
[a href=\"index.php?act=findpost&pid=129247\"][{POST_SNAPBACK}][/a]
I've just been looking at some D200 RAW files, ISO 100 and 1600, and it doesn't seem like those values are likely. It would be interesting to see what values you get for blackframes from the D200.
Or anyone with a D200.
-
First off, I'd like to point out that John Sheehy is absolutely correct in that the camera sensors mentioned by Mr. Clark do not count individual electrons.
Counting single electrons is so hard that it was first done just over two years ago by Swedish physicists. (http://physicsweb.org/articles/news/9/3/11)
If John Sheehy doesn't understand the concept of unity gain, it may be because it's practically impossible to have a setting in the camera where it can count single electrons.
If bjanes wants to prove Roger Clark right (or he wants to do so himself), then my guess is that there's a Nobel Prize in physics waiting; doing this at room temperature would be a significant leap forward.
[a href=\"index.php?act=findpost&pid=129357\"][{POST_SNAPBACK}][/a]
No one has claimed that the camera is counting electrons individually. However, at high ISO we are working with relatively few electrons in the shadows. As an example, the table below is for the Nikon D200 and is derived from Roger Clark's sensor analysis of that camera.The table shows data for ISO 1600 from highlights to shadows in decrements of 1 f/stop. The number of electrons collected, the shot noise, the read noise, and total noise are shown along with the signal to noise ratio. All noise values are in electrons.
The full well capacity of the camera is about 33,000 electrons. However, at ISO 1600 the full well is not utilized and only about 2043 electrons are collected at full scale on the ADC. (output value = 4095 or thereabout). Seven stops down in the shadow area, only 16 electrons are collected. The total noise is 8.4 electrons and this represents the noise fog to which John Sheehy refers. Since the gain is 0.5 electrons per 12 bit ADU, the ADU output would be 16, corresponding to an 8 bit gamma 2.2 value of 28. This is Zone 8, and near where the blackpoint would be set in a normal photograph. We are not counting individual electrons, but are coming close. Since unity gain for the D200 is at ISO 800, we could have left the camera at ISO 800 and increased exposure in the raw converter. That wouldn't win a Nobel prize, but it might us to make better use of the camera.
(http://bjanes.smugmug.com/photos/174715818-O.gif)
-
No one has claimed that the camera is counting electrons individually.
That would be necessary, though, for Unity Gain to be fully meaningful. With a great deal of uncertainty in the counting of electrons, the only thing that should limit the usefulness of amplification is no further decrease in analog noise, as measured in units of electrons.
However, at high ISO we are working with relatively few electrons in the shadows. As an example, the table below is for the Nikon D200 and is derived from Roger Clark's sensor analysis of that camera.The table shows data for ISO 1600 from highlights to shadows in decrements of 1 f/stop. The number of electrons collected, the shot noise, the read noise, and total noise are shown along with the signal to noise ratio. All noise values are in electrons.
It would be nice if someone measured D200 blackframe noise at all ISOs in IRIS, which loads literal RAW data. I am skeptical about the software and flow that Roger uses for measuring RAW statistics.
10.0 to 7.4 electrons seems pretty low; that would make the D200 as clean as the 1Dmk3 at black at ISO 100, and I just don't see that in the RAW images. As I zoom into the D200 shadows, I see the noise become visible much quicker, and in out-of-focus deep shadow areas, the read noise seems to be about 3 ADU, which is 33000*3/4095 = about 24.2 electrons, a more typical value for a DSLR at ISO 100.
The full well capacity of the camera is about 33,000 electrons. However, at ISO 1600 the full well is not utilized and only about 2043 electrons are collected at full scale on the ADC. (output value = 4095 or thereabout). Seven stops down in the shadow area, only 16 electrons are collected. The total noise is 8.4 electrons and this represents the noise fog to which John Sheehy refers. Since the gain is 0.5 electrons per 12 bit ADU, the ADU output would be 16, corresponding to an 8 bit gamma 2.2 value of 28. This is Zone 8, and near where the blackpoint would be set in a normal photograph. We are not counting individual electrons, but are coming close.
I don't see how you come to that conclusion. Do you know what a sigma of 7 or 8 electrons means? It doesn't mean that most counts are right, and occasionally one will be 3.5 - 4 electrons off. If the average signal is 15 electrons, you will have many pixels clipping at zero, and going as far off as 40 or 50 electrons worth of analog signal. It's a big mess.
And as I've said earlier, these RAW levels are not what you might think they are. The top stop isn't even used in the green channel with matte subjects in even lighting, at the rated ISO, and even more isn't used in the blue and red in daylight, and three stops get clipped away normally in the blue channel in incandescent. That puts middle blue at 6.5 stops below saturation in incandescent light. Sure, you can expose a stop to the right, but you are no longer actually at the stated ISO. Every camera has a highest ISO, and to use Av and Tv values that are needed may mean shooting way down in the RAW signal, in at least one color channel.
Since unity gain for the D200 is at ISO 800, we could have left the camera at ISO 800 and increased exposure in the raw converter. That wouldn't win a Nobel prize, but it might us to make better use of the camera.
[a href=\"index.php?act=findpost&pid=129758\"][{POST_SNAPBACK}][/a]
You could probably push ISO 400 to 1600 as well, with little loss, but the converter must not treat the midtones as shadows, as ACR does, for example, if you leave the Shadows slider at 5.
I don't see "unity gain" explaining anything here. If the results are the same (except for mild quantization with the pushes), then the real gain at the first stage may very well be the same for all ISOs, and the D200 doesn't do unity gain, but only 1:8 gain.
-
Conventional wisdom is that for digital cameras it is best to expose so that the right side of the histogram is as far to the right as possible without blowing the highlights.
However; the raw converter I use allows exposure compensation of +/- 2 stops.
If I under expose does this mean I can bring out detail in the shadow areas or is this a false move. Having preached the conventional wisdom I was asked this question during a talk and I could not answer. Can anyone help.
[a href=\"index.php?act=findpost&pid=124188\"][{POST_SNAPBACK}][/a]
Within limits I think expose for the shadows and dev for the highlights works best for me.
Kevin.
-
Conventional wisdom is that for digital cameras it is best to expose so that the right side of the histogram is as far to the right as possible without blowing the highlights.
However; the raw converter I use allows exposure compensation of +/- 2 stops.
If I under expose does this mean I can bring out detail in the shadow areas or is this a false move. Having preached the conventional wisdom I was asked this question during a talk and I could not answer. Can anyone help.
[a href=\"index.php?act=findpost&pid=124188\"][{POST_SNAPBACK}][/a]
Back to the beginning .
I think you're asking whether shadow detail can be brought out, in an underexposed image, by using the +/- exposure sliders in ACR.
I think the answer is both yes and no. The appearance of the shadows can be lightened and more detail made visible by using the Exposure Compensation slider during conversion, just as it can with many other techniques in Photoshop, such as use of curves.
But as far as I know, there is no recovery of shadow detail in the way that a minus adjustment of the EC slider can bring out highlight detail.
In order to get the most shadow detail in the RAW conversion it's necessary to have the 'shadows' and 'contrast' sliders at zero in order to minimise shadow clipping, and of course convert into 16 bit. Having done that, it doesn't make much difference if the EC slider is at +1 or -4. All the shadow detail that's there will be converted in both cases, but the conversion with a -4 setting will need to be lightened by other methods, such as use of curves or the shadow/highlight tool.
(Notice I wrote it doesn't make much difference. It probably makes some difference as a result of quantization issues, but nothing outside the realm of extreme pixel peeping, that I can see.)
Exposing to the right is basically just a technique of making the most of the dynamic range of your camera. If the scene you are shooting is of low contrast, it's not such a big deal. If the scene has a greater dynamic range than that of your camera, then it's very important to correctly expose to the right in order to minimise shadow noise. The alternatives would be exposure bracketing on a tripod and digitally blending the different exposures.
-
Exposing to the right is basically just a technique of making the most of the dynamic range of your camera. If the scene you are shooting is of low contrast, it's not such a big deal. If the scene has a greater dynamic range than that of your camera, then it's very important to correctly expose to the right in order to minimise shadow noise. The alternatives would be exposure bracketing on a tripod and digitally blending the different exposures.
[a href=\"index.php?act=findpost&pid=130060\"][{POST_SNAPBACK}][/a]
My take is this. Expose to the right places the most actual data in the last stop as the camera can record instead of noise. I'm not sure how that equates to dynamic range which I would have to believe is fixed.
If you have a camera that can record 6 stops from shadow to highlight and you have a scene that's less than 6 stops, this is pretty easy to capture, however, you still want to place as many levels in the last stop as possible. So with a 12 bit file, that means 64 levels can possibly be utilized for data. If you under expose, you'll get less than 64 levels and if you go way too far, you'll lose shadow detail of course (and get muddy highlights you'll have to adjust).
IF you have a 6 and a half stop range, well you better decide what half a stop you're not going to capture on one end or the other. Or play with fill or other lighting techniques to adjust the scene dynamic range. If shadow detail is key, you're going to lose ½ stop of highlight detail; you can't fit the entire range, the sensor can't handle it. If you go for the highlights, you lose half a stop of shadow detail. But in either case, you want to expose to get as much data into that last stop, which means not under exposing or all the levels get shifted into the wrong direction.
Does this sound reasonable?
-
I'm not sure how that equates to dynamic range which I would have to believe is fixed.........
.......Does this sound reasonable?
[a href=\"index.php?act=findpost&pid=130062\"][{POST_SNAPBACK}][/a]
Sure does. I was just trying to simplify matters. If your camera has a dynamic range of 7 stops and you give 2 stops less exposure than you could have given (without blowing highlights), then it's equivalent to using a lesser camera with a lower dynamic range of 5 stops in which the shot has been correctly exposed to the right.
Dynamic range limitations are often a major concern with digital cameras. If you don't expose to the right, you are not making full use of the dynamic range capability of your camera.
-
Do you know what a sigma of 7 or 8 electrons means? It doesn't mean that most counts are right, and occasionally one will be 3.5 - 4 electrons off. If the average signal is 15 electrons, you will have many pixels clipping at zero, and going as far off as 40 or 50 electrons worth of analog signal. It's a big mess.
Yes, I do know what a standard deviation is, even though you seem to think you are the only one who understands anything. First of all, your method of determining the read noise is non-standard. The standard method used by Roger and also illustrated at RIT (http://spiff.rit.edu/classes/phys445/lectures/readout/readout.html) is to subtract the dark frame from a bias frame, determine the SD, and divide the result by 1.414. This eliminates any clipping at black and your crude corrections for clipping are not needed.
For example, with the D200, here is a dark frame of the green channel at ISO 1600 as produced by Iris:
[attachment=2897:attachment]
And here is the result of adding 200 to one dark frame (the bias) and then subtracting a second dark frame. Note that one gets a smooth symmetrical bell shaped curve as one should for a normal distribution.
[attachment=2898:attachment]
Note that in both cases the standard deviation is 22.63. In the second case, the standard deviation of the subtraction is 32.304 and dividing by 1.414 gives 22.84.
In the first case, the mean is 20.469 and the standard deviation is 23.881; the usual 95 confidence interval is the mean ± 2 SD or 20.369 ± 22.84 * 2. This means that 95% of all determinations will be within this interval. I did not determine the camera gain, but using Roger's data, it is 0.5 electrons/ADU. Therefore, the SD expressed in electrons would be 11 electrons, which is close to the value Roger obtained in his analysis. 20.369 ± 22.84 * 2 gives negative counts, which is an impossibility, and results in "clipping". However with the bias frame subtraction frame one has all positive numbers. Your fudge factor does not seem necessary.
-
What is the origin of this number?
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=129536\")
See my [a href=\"http://luminous-landscape.com/forum/index.php?showtopic=17706&view=findpost&p=130247]Post[/url] for comments on this fudge factor.