Luminous Landscape Forum
Equipment & Techniques => Digital Cameras & Shooting Techniques => Topic started by: Ian Lyons on July 16, 2003, 06:19:12 pm
-
Michael,
My tutorial should NOT be read as the way of obtaining the "ideal exposure" for anything other than capturing the profiling target. I only quoted it so that folk could see that even when they think the highlight has been blown it very likely hasn't.
For real world images use your camera and normal methods to obtain the "ideal exposure" - capture the scene and read the histogram - if the highlights fall short of the right-hand end point increase exposure, but not so much as to blow the highlights.
Pleass don't make ass of yourself waving a white/grey card around
-
the values returned by the sensor are linear, and need to have a (logarithmic) gamma curve applied to "look right" to human eyes, since human vision responds to light logarithmically rather than linearly.
I slightly disagree: gamma rescaling (which is a power law, not quite logarithmic) is not directly to correct for the eye's response, and in fact it gets undone on output: gamma transformed values are something like
recorded level = (relative luminance) raised to the power 1/gamma
["contraction"], and on output this is undone by raising the level to the power gamma ["expansion", e.g. monitor gamma]. The virtue is that low luminance levels are "encoded" at larger numerical values, and so any subsequent noise, discretization error and such are a smaller fraction of the signal level (better S/N), and this noise gets shrunk in the expansion step. (TV used it first to deal with good old fashioned transmission noise.)
I think this is similar to the "bias" used in analog audio: when recording, boost the high frequencies, which suffer most from noise; on playback, scale them back down, reducing any hiss that accumulated in between.
This does relate indirectly to the eye's "logarithmic response" by giving smaller absolute noise levels in the shadows, where the eye can detect a lower level of noise than in highlights.
-
Bernard, the camera histogram is after the gamma has been applied which is why Michael says "if you do not use the right-hand fifth of the histogram for recording some of your image you are in fact wasting fully half of the available encoding levels of your camera." Without the gamma applied the rightmost stop would be half the histogram.
All this aside, although the article contains some truth in the value of getting data up out of the noise band (improving S/N), it's full of so much mis-information concerning DR (eg. it does *nothing* to increase your DR unless you're currently underexposing) that I don't think it even warrants discussion. Even the bit talk is misleading. Only 128 possible values for the lowest stop! Horrors! but when you think about it, at print time you've only got 256 values for the entire 5 or 6 stop range, so 128 is MORE than adequate for the bottom 1/5 of the image (which will be represented by 50 or so values unless you're wildly pulling shadows up).
- DL
-
Michael,
Please not so, I would like to know on what you base this opinion “Most readers of this board are sophisticated enough to know that prints are done in 8 bit mode” or do you believe you are above qualification?
Further to the article, I see no device engineers, lets hear from some one who understands how the device functions, not how it is treated in software. I am sure you know the difference!
Maybe the article falls because it is neither; an expert assessment of hew the sensor device works or, an advice to photographers on how to get the results they desire.
I have to wonder if you understand photography, art and science?
Victor
PS – It is the accepted practice to provide qualification for your (or others) opinions.
-
I'm fascinated by this discussion. Unfortunately, I recently joined this forum and did not read the original article by Michael. Can anyone point me in the direction of finding it? Thanks,
Ken Weissblum
-
Looks like we are making progress, however, a little confused about gamma in the histogram! How about a few examples...
-
> My understanding of Gamma is a measure of contrast
Victor, I'm not an expert on these matters and was hoping someone would jump in, but lacking that I'll give you some layman's comments.
I wouldn't consider gamma a measure of contrast. Technically it is a transfer function (and strictly speaking a power function) but if you need an everyday word to describe it I think 'brightness' would be closer than 'contrast'.
> Now I have a question on how the histogram is showing gamma, surly it is just a measure intensity by tone. So if the contrast is there (high to low) it is not representative of the contrast or gamma in the picture. Did I miss it?
The histogram doesn't directly show gamma although it certainly reflects it. 'Gamma' adjustments are made because of different characteristics of devices and generally the reason is to provide similar appearance on the different devices.
But the word 'gamma' is also sometimes used in a loose sense to mean brightness (midtone) adjustment represented by pulling the center of a curve up or down. This obviously affects contrast also but is not usually the primary aim of the adjustment. Sometimes it's used in an even more general way to refer to all tonal adjustments used to render an image appropriately.
Someone please jump in and correct me if I'm too far off base.
- DL
-
Bruce,
I hope all those gamma references don't just obfuscate the issue! :-) But seriously, considering the work you've done and your mathematical bent, I'm eager to hear more of your views on gamma.
Enjoy!
-- Jim
-
Michael's interesting article on dynamic range and exposure with dSLRs does not seem to me to go far enough. I have noticed that even though my exposure is accurate enough so that the histogram is similar to those pictured in the article, often one of the color channels is blown out. This leads to unacceptable oversaturation that cannot be corrected in PS. So, the question is how do we gauge the exposure in situations where one color predominates? My solution, based on experience, is to reduce the exposure.
-
I always keep my grey card hidden under my raincoat.
-
I am not getting this, the image is captured on an array of sensors, each sensor has a specification equal to the other millions. Now that means the shadow detail of an image is captured at 12bits, so can an expert please explain why the captured image would only reproduce the limited 'number of levels'.
Because shadows are a very limited subset of the total number of values. So if you underexpose, then you are further limiting an already small set of values representing shadow tones, thus causing blocky shadows and posterization.
The rest of your post is pretty meaningless. Duh, the whole point if an image sensor is to measure light. And water is wet, too. Yes, each pixel in a Bayer array is color masked, but no, that doesn't mean it has only 4 bits of accuracy. Each pixel is a 12 bit light measuring device in a specific color band. Silicon has a linear response to light intensity, therefore the values returned by the sensor are linear, and need to have a (logarithmic) gamma curve applied to "look right" to human eyes, since human vision responds to light logarithmically rather than linearly.
-
Don,
I'm not interested in debating the article with you. It stands on its own. You are free to agree or disagree, of course.
But I'd like to take you up on your last point...
"...at print time you've only got 256 values for the entire 5 or 6 stop range, so 128 is MORE than adequate for the bottom 1/5 of the image "
Most readers of this board are sophisticated enough to know that prints are done in 8 bit mode. The point that you appear to overlook with the comment is with regard to the advantages of working in a larger bit space prior to printing. That's why an understanding of this issue is worthwhile. (The fact that 6 or 7 ink dithering simulates a wider space is another issue).
People who are new to digital image processing, and even many of us who aren't, need to understand the underpinnings of our craft. Thomas and I were trying to shed some light on how imaging chips work so that photographers can have an understanding of why certain things happen.
Indeed, I suggested in the article that people try a simple test themselves, not so much to give them a tool that they might use in the future (though that too) but so that they could see first-hand what's going on "behind the curtain". Based on the huge e-mail feedback that I've received thus far I believe that this has been accomplished.
Michael
-
No. I think not.
The article stands as is and is supported by expert opinion. I have no inclination to enter into an exhaustive debate.
If you find it unsatisfactory, so be it.
Michael
-
Lots of good information in this thread!
Schewe (Jeff?),
You're right - it would be nice to have the option to see a linear histogram vs. a gamma-adjusted one. But as long as we treat the (Canon) histograms as representing the gamma-adjusted scene, can't we make make judgements based on the histogram?
Something I'd like to learn more about is the process of adjusting the image in Camera Raw (or other RAW converter) to take advantage of the linear data, while minimizing noise. I see relationships between the "exposure", "lightness", "darkness", and contrast controls, but I'm still unsure what, exactly, they're doing to my data. Yes, I've tried manipulating the sliders and watching the histogram. I still can't generalize to understand what's happening, beyond noticing the changes to the image.
Ray,
You raise a good point about the interconnectedness of dynamic range and signal-to-noise. And I agree with your description of how to handle the case where the camera has less dynamic range (with acceptable signal-to-noise) than the scene.
As for your other example where the camera has more dynamic range than the scene, It seems that the "best" approach would be to overexpose the image, while being careful not to blow out the highlights. That way the majority of the pixels are up out of the noise. If I understand this correctly, more bits are used to represent the image data the farther to the right the data is in the histogram. This should be the case whether the histogram represents gamma-corrected data or linear data.
In Photoshop, (or RAW converter) I suppose one goal would be to adjus the image to look, as much as possible, like the original scene. (Yes, many photographers prefer to take liberties with the image, and that's okay too.) Whatever the goal, if you wish to portray lots of deep shadows with little or no detail, then moving the left slider to the right will accomplish that. If you want to preserve lots of detail at the bottom-end, then perhaps adjusting the middle slider would be more appropriate.
These steps would tend to "undo" the overexposure, and possibly "stretch" the image to fill the histogram. But it seems that one result would be to avoid the "noisy" left side of the historgams, possibly resulting in cleaner shadows.
Okay, I'm all ears if I've bolluxed any of this up!
Enjoy!
-- Jim
-
> a little confused about gamma in the histogram! How about a few examples...
Ok, http://www.lashier.com/home.cfm?dir_cat=25027&gal_col=5 (http://www.lashier.com/home.cfm?dir_cat=25027&gal_col=5)
I'd estimate that the 'bars' on these histos are roughly 2/3 stop apart. In particular note the linear histo. Even though this shot was only 1/3 stop away from blowout the linear data barely reaches midpoint. The Canon RIC histo is probably similar to what's shown on the camera. Also note the compression on the ends of the 'non-linear' conversions. These represent the toe and shoulder rolloffs in the TRC's.
ps I've added another page showing the non-linear to linear and vice versa transformations.
- DLhttp://www.lashier.com/home.cfm?dir_cat=25028&gal_col=5 (http://www.lashier.com/home.cfm?dir_cat=25028&gal_col=5)
-
> I take it EC stands for exposure compensation, but not sure about TRC.
Ray, "Tonal Reproduction Curve" - see http://www.lashier.com/home.cfm?dir_cat=14477&gal_col=4 (http://www.lashier.com/home.cfm?dir_cat=14477&gal_col=4)
> There's a price to pay. Does this make sense?
Yes. Trade-offs seem to abound in this game, which is what I was trying to point out about the 'expose right' theory. For me the trade-off usually isn't good. If there's any "rule of thumb" I've discovered it's "the less pushing and shoving the better". This means I generally aim for an exposure that requires the least adjustment over the range of the image. But the corollary is "if you must adjust it's better to adjust down than up" which means that in some cases you may expose for the shadows.
- DL
-
Yes Don, I never think about the histogram, expose for the subject and compensate for the scene, sounds crazy but, I meter with my eye and then check with a spot meter.
The runners were shot on raw mode and I used two different converters (Fuji / Bibble), both the same magenta. I do think the software should do what film would and burn white here! What do y'all think?
Victor
-
I am not getting this, the image is captured on an array of sensors, each sensor has a specification equal to the other millions. Now that means the shadow detail of an image is captured at 12bits, so can an expert please explain why the captured image would only reproduce the limited 'number of levels'.
Now what I think is meant by this is the exposure value only turns the luminosity level to a particular number, but this will work on a binary scale based on how bright the light falling on the sensor is. In this context the sensor is a measuring device! Except there is another limitation in that each sensor is masked with a color, so you are only measuring 33% of the light, is that 4 bits of the 12?!
Then the solution must be for the next generation of sensors to measure color temperature (being RGB) and luminosity at a analogue to digital conversion level that creates clean tone across the dynamic range!
-
Even getting up from the current 8 to 12 bit D/A conversion to full 16-bit per color accuracy would require sensor S/N ratios of about 65000:1, about ten times higher than anything available now in either film emulsions or digital camera sensors. The best performance so far is probably good negative film, equivalent maybe to up to 12 or 13 bits depth.
Most current digital sensors do not have good enough S/N ratio to give full accuracy down as far as 12 bits; the least significant bit is probably all noise in many DSLR's, as indicted be the fact that the dynamic range of such cameras is distinctly less than 12 stops.
If noise levels could be reduced enough to give 16 significant bits/color, it would give close to a 16 f-stop range and almost a hundred luminance levels per f-stop even way down in deep shadows ten stops below maximum brightness. From what I have read about the eye's capabilites, this would go well beyond what the eye can handle in both dynamic range and distinction of luminance levels. It would certainly far exceed the luminance range of any image display method; slide, video or print.
In summary, full 16-bits per channel luminance resolution would already be more than we could ever need, (though people who love having big numbers displayed on decals on their camera bodies might want and pay for more!)
-
If only cash flow was not a constraint!
Victor
-
Jim,
Sorry to cause you such distress, however the article is just a rehash of standard transparency technique, and I am pleased you will gain from the technique. You can try this simply, on a sunny day pick a place with some good dark shadow area and then roll of the exposures. ( Good example - http://www.contaxg.com/document.php?id=10120 (http://www.contaxg.com/document.php?id=10120) ). If the shadow is 60th f2 and the highlight 2000th f16 begin shooting at a 15th and go as dark as you lens camera will go. Now take a look at the images and make your own assessment on the effect of the exposures. Repeat the exercise for different lighting like snow and water. I hope you will learn that there is only one correct exposure, for the subject your photographing. And the more subjects you include in the picture the bigger the compromise you make. Simple huh!
I have been on this site for a long time, have seen a lot of his work and think that Michael should know better! Maybe he needs to work for a good editor.
Victor
-
As for your other example where the camera has more dynamic range than the scene, It seems that the "best" approach would be to overexpose the image, while being careful not to blow out the highlights. That way the majority of the pixels are up out of the noise. If I understand this correctly, more bits are used to represent the image data the farther to the right the data is in the histogram. This should be the case whether the histogram represents gamma-corrected data or linear data.
Jim,
Well this is precisely what Michael is recommending in his article and it certainly makes sense to me. But supposing the dynamic range of the scene is only 2 or 3 stops and the histogram shows the image to consist mainly of mid-tones, ie. the histogram is not close to either the left or right side, but slap in the middle. I presume in those circumstances noise is not a problem, yet the advice would still be to overexpose in order to achieve greater tonal subtlety. Right?
-
EC does not drop stuff off the left, but compresses it into the toe of the TRC.
There are so many acronyms in this thread I don't know whether I'm coming or going. I take it EC stands for exposure compensation, but not sure about TRC.
One thing I've noticed when converting RAW images in Breezebrowser (sorry! BB), is that exposure compensation of a seriously underexposed image (say 1.5 to 2 stops) at the time of conversion, results in an image with considerably less noise in the shadows than any levels adjustment in PS can achieve with a straight non-EC conversion. However[/i], the EC conversion has slightly MORE noise in the rest of the tonal range. There's a price to pay. Does this make sense?
-
> You can read more here
Thanks Jim, thought I halfway understood this stuff but now I'm really confused
:Victor
> First sample shows the sky going magenta in the top right corner!?
Looks like a channel blown. What processing/conversion? (Interesing photo btw)
> exposing to the left and see how the highlights have blown out!
Looks like good exposure to me. Sometimes you just have to choose and I think you chose right (whoops! meant 'think you chose correctly') in this case. BTW, this does show that you can sometimes blow highlights gracefully with digital.
> Finally a film example, so far I have failed to get this much from a digital camera, I'll keep playing!
Oh I think digital will handle that, albeit possibly with a slightly different effect. Just watch out for artifacting in the highlights.
Victor, sounds like your experience mirrors mine in that I rarely have the luxury of deciding where to place the 'hump' but rather am pushing both ends and may have to choose which end to blow.
- DL
-
Here's what Thomas has to say on the issue...
Yes, the problem of only one or two of the channels being clipped is
sometimes a problem. With raw capture it is not as bad as with JPEG
capture, since with raw what you care about is clipping in the native
camera color space, rather than in the working RGB space (since
conversion to working space happens after the raw converter's tone
adjustments). Lots of colors clip in sRGB or even Adobe RGB that
don't clip in camera native space.
The ideal fix would be for the camera makers to change the
overexposure clipping warning to flash based on the native color
space channels, rather than on luminosity. This is one of reasons
why Camera Raw's histogram is an overlay of the color channel
histograms, rather than a luminosity histogram.
In practice, this is usually not that big of an issue, since the
brightest objects in a scene tend to be nearly neutral (often white
clouds). When in doubt, you can always bracket and check the
resulting color histograms to pick which frame to work with.
Thomas.
-
Actually this is well known fact, that digital doesn't handle traditional "underexposure" well and I'm using this "Right" approach with digital from the beginning. I even did some measures for this:
http://www.robgalbraith.com/ubbthre....pe=post (http://www.robgalbraith.com/ubbthreads/printthread.php?Board=UBB8&main=137186&type=post)
If you take a look at noise graph it's easy to see. Sorry for using stddev as noise mark, dE would be better, but even this should be enough to show tendency. Don't ask me about shape of graph - I have no clue why it's like this
So, the article gives me theoretical base for my practical observations and this is very nice!
Thank you, Michael and Co for this!
Andrey
-
Victor -
Quite a dream! Forget 512 bit...Just get us to 32bit!
At 32-bits:
- we'd have 2.1 billion unique levels in the 1st f/stop.
- we'd have 4.2 million unique levels in the 10th f/stop.
That'd be enough to work with, right?
- Dave
-
Dear all,
I just read the very interesting article of Michael on the topic, and there are a few things I have a hard time understanding. Thanks for any explanation on the topic:
- I read recently (poster was Thom Hogan on dpreview I believe) that, on the D100 at least, the histogram that is displayed is computed based on the thumbnail image displayed on the screen of the camera, and not on the actual RAW image.
If this were correct, it would mean several things:
1. it is probably only a 8 bit projection of the 12 bit raw space,
2. the Y axis of the histogram only represents the amount of pixels of a given darkness in the thumbnail image. This means that the rightmost fifth of the histogram actually represents one fifth of the dynamic range, and therefore only a part of the brightmost stop.
The vertical lines draw on the screen are just misleading since they lead the user to think that that one fifth corresponds to one stop, but this is not the case since, as Michael points out, one stop should correspond to half of the screen.
I guess that whether I am right or wrong depends on what is actually displayed in the histogram.
At least, isn't there a need to confirm this on a per camera basis?
Thank you for your feedback on this.
Best regards,
Bernard
-
Victor,
You're new here. Perhaps, rather than repeatedly challenging Michael, you would be well served to "lurk" a bit. Read what Michael has to say, both in the message boards and in his articles. As you learn more about Michael's work and see his photographs, you may develop some respect. If not, well, there are plenty of other destinations on the web where you might find things more to your liking. As it is, your tone is disrespectful. While you had some interesting questions, it's Michael's option to debate these issues with you - or not.
You say the article "falls because it is neither; an expert assessment of hew the sensor device works or, an advice to photographers on how to get the results they desire." On your second point, I disagree. Speaking for myself, I enjoyed the article in question, and feel I learned some things about histograms that hadn't occured to me before. I will adjust my shooting accordingly. I think my response to the article was just as Michael would have hoped for many of us.
As for your first point, if you're looking for expert assessments of how the CMOS works, why berate an article that is intended to do something different? Why not look elsewhere - or offer to do your own research and post an article with the sort of information you're interested in?
Sorry for the rant - I simply felt your last posting was unfair to Michael.
Best Regards,
Jim
-
Ray,
Yes, my understanding, based on Michael's article and the follow-up discussion here, is that image data to the right side of the histogram is of higher quality (less noise, defined by more bits) than image data to the left. As you say, we ought to get better tonality from a low-dynamic range image if we shift it in-camera to the right, then correct later in PS.
Enjoy!
-- Jim
-
> In brief, it is due to the extra accuracy allowed by the extra four significant bits used in 16-bit editing.
Ok, I see this. It depends upon at what stage you do the EC. My assumption was that EC would be done on the linear data but I just did a quick experiment and that doesn't seem to be the case, with C1 anyway. -EC does not drop stuff off the left, but compresses it into the toe of the TRC.
But I would still make the point that gradient resolution usually only becomes an issue when you raise shadows and in a compressed image shot to more or less center the histo, there's nothing in the shadows to be raised. More than likely you're going to push stuff down by setting BP or increasing contrast, and as long as you do this in the raw converter (or convert to 16 bits and do it in PS) you're not going anywhere close to running short of values.
Do I really need floating point to balance my checkbook? Maybe BG, but not me :(
- DL
-
Thanks Jim for all those references on gamma; this seems to be another subject that many of us need to understand better than we do in order to get the best looking images (sorry, I mean "to optimize our digital workflow").
Perhaps it would be nice for this site to have an article on Understanding colour spaces and gamma, but then again, perhaps one of the above references already does the job; I will start reading.
-
Michael,
As digital photography evolves, I hope we see more incremental development of tools to improve the acquisition process. Thomas's suggestion to have camera manufacturers have blown pixels flash in the color of the channel that has clipped is an example of this and I hope he is heard by folks like Canon. A couple of thoughts from my recent noodling :
Since we are concerned about tonal gradations and smoothness (in addition to the relentless quest for resolution :p ) it seems that an alternative (i.e. selectable by a button when the histogram is displayed) histogram display which is linear, rather than logarithmic, might be useful. This would facilitate assessment of whether the areas of the image where that might be critical do indeed have enough "levels" to achieve the desired image. In this space, one would be aiming for the centered histogram (again!).
A few posts have mentioned the possibility of a "live" histogram in the viewfinder (based on a small - few enough pixels to be processed on the fly) image sensor in the pentaprism somewhere, as being something that would be helpful - another incremental aid.
Also, now the LCD can be zoomed somewhat, it would be nice to be able to see clipped pixels in the zoomed version. I often know that to get the shot I want, I will accept "blown" highlights - they may actually be "blown" in reality - specular highlights for example. So some clipping in the histogram will be expected, but it's hard to know that only those are blown, and not some others for which we want to retain some levels information. Examination of a zoomed section for blown pixels would be another incremental help. This would be evn more important if we start to adopt the suggestion that started this thread, of moving the histogram to the right.
None of these is real urgent, but they might be considered analogous to the increments in automotive technology such as ABS, airbags and satellite navigation systems - all incrementally beneficial, do-able as technology moves forward and , not least of all, sales tools as incentives for manufacturers and all that implies!
Just a little ramble!
Andy
-
Thank you Jim
So the what now that seems to be missing is the ability of the sensor to represent low light levels. Rather than over exposing, would it not be better to have a sensor that has a better range! If I may dream for 18 months, maybe we will get a full color sensor that has 512 or 1024bits and s/n problems fixed.
Why full color, go and understand how film works then you wont ask the question. There is a simple rule to remember capture the real color info from the lens, not what the software thinks it should be. If you think Bayer is OK, then good for you!
-
Do tell me what improvement you would prefer?
I was not looking for 500khz but covering a greater range than is required makes good quality sense. Most modern technology is built to the average need, this is not OK for serious photography. Some how there are boundaries to define what is acceptable level of quality in the representation of changes in tone (regardless of illumination). I hope we see sensor makers delivering in excess so we can produce pictures that represent the subjects we photograph. However the assessment of tone, like lens quality is perceptive making it difficult to decide how much is needed tonal definition is needed.
On a sunny 5500 kelvin day the contrast can be in excess of 5000:1. Today the compromise gets me a max of 500:1 (if I'm lucky) there is no reason we cannot capture closer to the real detail in future. Then we can photograph the muscle contours on a black horse against white sand, and see the grains! Today you have to choose which you want.
I'll keep dreaming :D
-
> Most readers of this board are sophisticated enough to know that prints are done in 8 bit mode. The point that you appear to overlook with the comment is with regard to the advantages of working in a larger bit space prior to printing. That's why an understanding of this issue is worthwhile. (The fact that 6 or 7 ink dithering simulates a wider space is another issue).
Michael, I agree that working in higher bit space is desirable prior to printing, but my point was that in most cases of compressed light where this technique is applicable that the first thing you're going to do (probably in the raw converter) is to -EC to get midtones back in line, then most likely set BP and WP to expand tonality. In the process you lose back all the gains in bit depth (and more) from over-exposing. If you don't lower the shadows back down then gains are preserved but in that case I would argue that the higher exposure is the 'proper' exposure to begin with, so you're really not applying this technique.
That's not to say that I don't see merit in the technique in special situations, but I'd say those situations are the exception rather than the rule for most photographers. And the gains are in reduced shadow noise rather than increased dynamic range.
- DL
-
Ken,
It's right here: http://www.luminous-landscape.com/tutorial...ose-right.shtml (http://www.luminous-landscape.com/tutorials/expose-right.shtml).
Enjoy!
-- Jim
-
With subjects of modest contrast range, exposing to the right is probably not of much practical value:
on one hand,
it will give more levels at each subject brightness level, and these extra gradations will then be preserved when one does compensating EC downwards in 16-bit mode,
but on the other hand,
all the important brightness levels in a low contrast subject are high enough to get plenty of levels of gradation when one does a normal "18% gray" metering, so the extra effort proabbly does not pay off in any visible improvement in final image quality.
I would suggest as a rule of thumb that the extra effort of pushing the histogram to the right is worthwhile in roughly the same cases as when careful spot metering of highlights and shadows are worth it. Perhaps comparing these two approaches in the field is preferable than endless attempts to anwer the question theoretically.
-
"on one hand, it will give more levels at each subject brightness level, and these extra gradations will then be preserved when one does compensating EC downwards in 16-bit mode,"
Please explain how the extra gradations will be preserved when you do compensating EC downwards. There must be something I'm missing here.
OK Don, here is my reasoning, let me know what you think.
In brief, it is due to the extra accuracy allowed by the extra four significant bits used in 16-bit editing.
At great length,
a) Suppose that a one stop subject brightness range down in the shadows is placed by traditional exposure metering at -8 stops (8 below the maximum brightness handled by the A/D converter) and so 12-bit D/A conversion give you only eight levels over that one stop.
Increasing the exposure by two stops puts it at -6 and gives you 32 levels over the same one stop range of subject brightness.
c) Converting to 16 bit linear increases the number of numerical levels 16-fold, so those 32 values are scattered over a range of 512 (15 as yet unused levels have been added in between each level output by the D/A converter.)
d) If you then correct downwards by the same two stops, dividing each of these value by four, the range is compressed to 128 levels; still a gap of four between each recorded level: no distinctions of level are lost. In other words, each full stop of downward EC discards one bit at the least significant end of the 16-bit number, but since originally the last four bits were meaningless zeros anyway, you can go down four stops before arithmetic round-off loses any read information from the A/D converter. EC by other than whole stop amounts will make the numbers more complicated, but the rounding error from the 16-bit arithmetic will not be as much as the A/D discretisation error until the drop is four stops (a factor of 16) or more.
e) Gamma compression increases the number of levels in each f-stop range in the shadows, so the situation is even better if EC is applied after gamma.
Another point: as Don has mentioned in a different way, if you simply EC back down and do a straight print, these deep shadows will be off the bottom of the scale of any print anyway; the only point to making this effort to accurately record these shadow levels is to keep them somewhat above their natural level, by contrast reduction or whatever. So if you are wanting to print those shadow details, they should not in the end be EC'd all the way back to where they came from.
(I sometimes teach numerical computing, so I hope I have got at least this stuff about rounding, bits and errors right, but I could more easily be wrong about the many other aspects of this situation, and in particular I make no claim to any expertise about the ultimate aesthetic value of the different methods.)
-
Michael's interesting article on dynamic range and exposure with dSLRs does not seem to me to go far enough. I have noticed that even though my exposure is accurate enough so that the histogram is similar to those pictured in the article, often one of the color channels is blown out.
I think with digital ACCURATE is the wrong term: try IDEAL!
Basically the ideal exposure is as Michael describes: get your histogram as close to the right side as possible but not so close as to cause the over exposure indicator to flash. The ideal exposure ensures that you have maximum number of levels describing your image without loosing important detail in the highlights. The closer you get to this ideal then the more of those levels are being used to describe your shadows. If you underexpose an image to the extent that the shadows block, which is often what folk do to protect their highlights; then you will need to open them again to ensure the final image is as you require. The problem with this approach is that we only have 128 levels available to the shadows. You start pulling curves, etc to open the shadows and you'll get posterisation, etc.
We need to get away from the concepts of exposure that have served us well with film. The CCD/CMOS isn't film and does not react like film in the highlight shadow regions. Exposure on film tends to roll-off smoothly in the shadows and highlights. With digital the capture is linear and there is no roll-off. Unfortunately (as Thomas has indicated) the behavior of these sensors isn't perfect and we can (often do) get one or two channels going into saturation (blowing). This was a major problem with the Canon D30 and the current version of Adobe Camera Raw simply can't handle such images. Thomas knows my thoughts on this and I know his.
Its worth noting that I've not yet seen a "linear" raw image from a Canon EOS D30/60 or 10D that went full-scale, close but not all the way. This means that even though existing conversion apps might not meet your needs. So far as the raw conversion apps go I am confident that things will improve, but in the meantime I really do urge folk to look at this technique. Remember that you will likely still have the CRW file for a long time. It really is your equal of the negative; don't trash it. Even if the current crop of conversion apps can't handle the blown highlights future apps will. However, NOTHING will ever get you back the lost shadow detail.
You can see in the example shown on the following page how much info still exists in the highlights of an image that current raw convertors say is blown. This technique is virtually redundant now but should still give help you better understanding of just how mcuh info exists in yor highlights.
http://www.computer-darkroom.com/d30-profiling/d30_1.htm (http://www.computer-darkroom.com/d30-profiling/d30_1.htm)
-
Now that sensor location gets translated or processed to represent a pixel which holds an RGB value - if that value is not based on 12bits for each color (=36bits), we lose some detail, 20bits if we only use 16bits!
Victor,
In general I think you're correct - but the way you put it could be misleading. You say that an RGB value must be based on 12 bits for each color = 36 bits. I would add "at least" 12 bits per color.
But then you say we lose some detail ... if we only use 16 bits. Again, true - but probably off-base. When we talk about 16-bit color, we're generally referring to 16 bits *per channel* - or 48 bits per pixel. Representing an RGB value in 16 bits would result in, perhaps, 5 bits per color, with one bit wasted.
In practice, photographers are generally using 24 bits (8 per channel) or 48 bits (16 per channel) to represent each RGB pixel. If each photosite captures 12 bits of data, you can see that 24-bit color loses some data, while 48-bit color does not.
Hope This Helps!
-- Jim
-
Do tell me what improvement you would prefer?
To start with, I want something that you, I and many others in this forum probably agree on: sensors that can handle a greater dynamic range and S/N ratio than currently: even a very high end product like the Kodak 16MP sensor in their MF backs has a S/N ratio of about 4000:1 or about 12 stops from blown highlights down to pure noise, and so distinctly less than 12 stops if you stay far enough above noise to get good looking shadow details. That is currently the one area I know of where 35mm film (print film at least) most clearly offers more than even high end digital, and dealing with high contrast subjects seems to be the one area where digital photographers seem to have to work hardest, with techniques like blending two images.
There is a lot of room for improvement (about 4 stops, or up to about 65,000:1 luminance range) before the capabilities of 16-bits per channel format are exhausted; that is one reason why talk of 512 bits does not interest me.
After that, my second priority is a "smaller number": bringing the price of that envisioned high end quality down to my price range!
-
Yes, broad statements like "it's full of so much mis-information ...that I don't think it even warrants discussion" are useless to our ongoing conversation. If you disagree, explain why.
Dan, Michael has graciously retitled the article and removed or edited the references to which I objected. My problem was tying it to dynamic range and while there may be a tenuous connection it isn't the primary possible benefit of the technique. Note the sort of statements that appeared at dpr: "Michael Reichmann stated in his article that if you don't use the right hand (highest) 1/5 of the histogram, you lose 1/2 the dynamic range." (http://forums.dpreview.com/forums/read.asp?forum=1019&message=5626152) which I'm sure will make even Michael smile a bit. ::
Dynamic range and bit depth are two of the most mis-understood and confused concepts in digital photography and although I'm sure Michael understands them, his loose use of the terminology only sows seeds for ill-advice. Now if Michael had said 'minimize your noise levels', 'maximize your adjustability' or 'maximize your gradient' or such I wouldn't have had any argument, and now that things have been clarified I'm more interested in discussing other ramifications.
- DL
-
> It’s my speculation that the histogram is displaying a gamma encoded interpretation of a capture. . .at least for Canons
Jeff, Chuck confirmed this over at RG, and also note that the TRC is involved so the ends are compressed a little.
> So, the question still remains. . .should one expose raw captures like a chrome or a neg?
I shot chrome for years and I think that all that's meant when folk say 'expose like chrome' is that you've got to pay more attention to exposure because processing is less forgiving, and this applies to digital also. But I agree that you can't expose digital exactly like chrome - you've got to pay more attention to highlights because of the hard landing compared to chrome. In my normal outdoor (high dynamic) situation I spot meter almost exclusively on the highlights letting everything else fall where it may, but sometimes checking midtones and shadows. These images rarely require much if any EC. In compressed situations I pay more attention to the midtones.
> Some early digital shooters tended toward underexposing to avoid hilight blooming and blowing out specular detail. . .but the habit of underexposing also leads to a lot of shadow noise.
No doubt, and Michael's article makes a very good case as to why you shouldn't underexpose, I'm not so sure about the case the other way.
> When confronted with such a histogram in PS in 'levels', to keep it simple, what sort of broad adjustments would be recommended? I mention this because Don seems convinced that any 'overexposure' will need at some stage to be corrected and the correction will 'undo' those advantages of having initially more levels to describe the image.
Ray, to me overexposed images need to have values lowered by definition. ;-) As to whether you spread compressed shots and how far are artistic decisions, but for me normally yes. But I spread them in the raw converter so we're still in 16 (ok, 12) bits, left values are well up off the noise floor, plus I'm pushing them down, etc.
- DL
-
>Perhaps comparing these two approaches in the field is preferable than endless attempts to anwer the question theoretically.
And WTF do you think some of us have been this past year and half or more!
-
Great curves Don, that is what we need to map the histogram and the number of digital sample points to.
My understanding of Gamma is a measure of contrast, and checking myself http://www.wordreference.com/english/definition.asp?en=gamma (http://www.wordreference.com/english/definition.asp?en=gamma) would confirm my thinking. So I understand how gamma effects the display of an image...
Now I have a question on how the histogram is showing gamma, surly it is just a measure intensity by tone. So if the contrast is there (high to low) it is not representative of the contrast or gamma in the picture. Did I miss it?
-
so, we can blow huge sums on a DSLR happily thinking we'll never have to worry about spotting scans again, and that we can take advantage of all the time we saved to do a part time degree in non-linear algebra and rocket science so that we can end up with a photo which looks almost as good as film!
(is there a smilie for tongue-in-cheek anywhere ?)
But seriously, does anybody actually enjoy all this stuff ?
-
Let me put this a different way, if the sensor has a 12 bit register that means it can only give you a 12 bit value! So the measurement is the intensity of the color it is filtered to - seems simple enough!
How about this as a non-technical explanation! Each photodetector is like a bucket that can hold a certain finite number of photons. At full exposure (highlights), the bucket is full. At close to zero exposure (deep shadows) the bucket is empty. The size of the bucket represents dynamic range. Let's say we have a 10 litre bucket and each litre represents an F stop of dynamic range.
Let's assume we have a completely noise-free system. We're shooting a high contrast scene that has a dynamic range of 10 stops, but we fail to get the histogram as far to the right as possible without overstepping the mark. Result? Something less than 10 stops and loss of detail in the shadows. We're throwing water away. Wasting it.
Now, in practice we don't have noise-free systems. The 10 litre bucket contains 3 litres of sludge. By exposing as far to the right as possible, we're getting as much of the image as possible away from the sludge.
-
Yes Dave, I agree that would be a good start.
It is however BJL, not about big numbers it is about having enough detail to show the subjects we photograph. The tones we see in shadows are a curve not a numbererd steps. The only limitation from your list to consider is the eye, many of us got the same story when CD's arrived but musicians knew the difference. Most current display methods fail to show what is on a good kodak/fujichrome image, which in turn fails to show what the photographer wanted. So we should set the objectives high, higher than film which will inturn challange the the display industries!
Victor...
One. Good is the Enemy of Great - Jim Collins http://shopping.yahoo.com/shop....7047419 (http://shopping.yahoo.com/shop/__yltc=s:2065001512,d:46101228,p:s,w:Yahoo_Top_Sellers,a:r?d=b&id=1977047419)
-
Yes, broad statements like "it's full of so much mis-information ...that I don't think it even warrants discussion" are useless to our ongoing conversation. If you disagree, explain why.
-
That's not to say that I don't see merit in the technique in special situations, but I'd say those situations are the exception rather than the rule for most photographers. And the gains are in reduced shadow noise rather than increased dynamic range.
Seems to me (and I always stand to be corrected) signal-to-noise and dynamic range are inter-dependent. You can't have one without the other - a bit like resolution and contrast.
There always seems to be a certain vagueness in describing dynamic range. Is the true DR of the 1Ds, D60/10D 5, 6 or even 7 f stops? Well, it depends on how much noise you're prepared to accept in the shadows.
If we describe the DR of the 1Ds as being 6 f stops with a noticeable but not objectionable amount of noise in the lowest stop, then clearly, when shooting a scene that has higher than a 6 stop dynamic range (which would apply to most outdoor scenes on a sunny day) one needs to make sure the histogram reaches all the way to the right. If it doesn't, the result will be LESS dynamic range than the camera is capable of. Instead of 6 stops with an acceptable amount of noise in the shadows, we'll have say 5 or 5 1/2 stops with an acceptable amount of noise in the shadows. Or, to look at it another way, we'll still have 6 stops DR, but now with an UNACCEPTABLE degree of noise in the shadows.
Unfortunately, in such circumstance where the DR of the scene is higher than the DR of the camera, using the evaluative metering setting is quite likely to result in blown highlights (at least that's my experience with the D60). Yet the exposure is still in a sense 'correct'. The waterfall might look like a blank sheet of paper, but the person standing next to it looks good. Skin tones are correctly balanced, good detail in the shadows. The picture's perfect, except for the waterfall. The plain fact is, the camera has insufficient dynamic range for the scene.
If I want to retain detail in the waterfall, I have no choice (with a single shot) but to 'underexpose' other elements in the picture. That means I have to work on those underexposed elements in post processing to correct them. Lighten the mid-tones with the levels slider, or whatever.
I'd rather not have to do this because the person standing next to, or in the waterfall might be equally important, but I have to make a choice. In such circumstances, getting the histogram to touch the right side without clipping (or flashing) would seem to be the goal and the best compromise.
But what happens when the dynamic range of the scene is LESS than the camera's capability? The histogram might be the shape of a conical mountain slap in the middle of the range, not even nearly touching either the right side or the left side. Do we then deliberately 'overexpose' so the base of the mountain touches the right of the histogram screen and the gap between the left side of the mountain and the left side of the histogram screen actually widens?
When confronted with such a histogram in PS in 'levels', to keep it simple, what sort of broad adjustments would be recommended? Should one bring the left slider to the base of the left side of the mountain and then move the middle slider leftwards to lighten the mid-tones, or should one leave the left slider as it is and concentrate on the middle slider?
I mention this because Don seems convinced that any 'overexposure' will need at some stage to be corrected and the correction will 'undo' those advantages of having initially more levels to describe the image.
He has a point and it needs to be either corroborated or demolished.
-
So, if the histogram is representative of a Canon SDK "default" conversion, I would be inclined to say the histogram is even more usless since it will not be a predictor of either C1 or CR's conversion. . .
If one takes the care to spot meter various tones in the scene, then clearly one can determine the scene luminance range and how best to set the exposure. . .but I suspect that many photographers don't do that. I often don't either because of lot of what I shoot is in the studio where I can control the scene's contrast.
But I still think that Michael's contention that the linear raw capture has the widest tone levels in the upper part of the scene and less in the lower part (as would be the case with a linear 12 bit capture) might explain te problems a lot of shooters who tent toward underexposing have with shadow noise.
and yes, it's Jeff. . .
-
Don,
I'm struggling a bit, myself, with this whole topic. I found the articles on your web site, notably the histogram comparisons of the various raw converters, to be quite interesting. I look forward to the examples you mention.
Enjoy!
-- Jim
-
Thank you Don and Jim, It means we keep taking photos...
Don, That was kind of my thinking also. I had only considered gamma as an adjustment for display (CRT's)...
I would be interested to see some real photos, with explanations of how they got them. In my experience getting the exposure right has always been better than pushing to the right! I have even got highlights burning to a complimentary color (magenta is common) see below.
First sample shows the sky going magenta in the top right corner!?
(http://members.shaw.ca/aberdeenz/eagle/HMC-Y049.jpg)
Next is an example of exposing to the left and see how the highlights have blown out! Both on an Fuji S2 pro (RAW).
(http://members.shaw.ca/aberdeenz/eagle/DS-A0019.jpg)
Finally a film example, so far I have failed to get this much from a digital camera, I'll keep playing!
(http://members.shaw.ca/aberdeenz/eagle/LD-A0748.jpg)
Victor
-
> sounds crazy but, I meter with my eye and then check with a spot meter.
Doesn't sound crazy at all Victor. The first ten years or more that I shot chrome I usually didn't carry a meter at all so developed a pretty good eye and mental calculator. When I got my first camera with a built-in meter (Canon FT) my exposures actually went downhill for a bit! I use nothing but spot on my 1D. Matrix is just too vague for me.
> I do think the software should do what film would and burn white here! What do y'all think?
Yes. C1 seems to do a better job in this regard but I don't think it'll handle your S2 raws.
- DL
-
Ian,
So this sounds too simple. All I need to do is use a WHITE card (in the same light as the subject) to adjust the histogram so that the spike is all the way to the right, but not touching. Am I missing something?
Thanks,
Michael
-
Doh!
Let me put this a different way, if the sensor has a 12 bit register that means it can only give you a 12 bit value! So the measurement is the intensity of the color it is filtered to - seems simple enough! Now that sensor location gets translated or processed to represent a pixel which holds an RGB value - if that value is not based on 12bits for each color (=36bits), we lose some detail, 20bits if we only use 16bits!
Also the value representing RGB surly has a relationship to the color temperature, and luminance? So lets stop thinking that the Bayer mask is acceptable, that representing the light in 12 bits is acceptable and set a high expectation for the quality of image we require.
-
Victor, the limitations of the eye were my main point: over one hundred levels within a single f-stop in the deep shadows (and well over a thousand levels in the f-stop around middle gray) is, as fas as I know, vastly finer tonal disinctions than they eye can distinguish: in your audio analogy, it would be comparable to the 500khz frequency response boasted by a few specialist amplifiers, not the quibbling over whether 22kHz is enough. Dwelling on going vastly beyond what is visually relevant in one aspect of performance would, it seem to me, just take resources and market pressure away from making other far more relevant improvements.
-
Michael, can you please provide some references for your statements. I have found in the past that such comments are often not supported by the data! Your digital argument is lacking in detail. Now draw the characteristic curve for a sensor and show where the binary values fall, now draw the steps - I think you will smile! This is why we need more numbers on the curve.
Your article is IMHO only suggesting that you over-saturate, or expose for the shadows - yawn.
Victor :)
-
Well. . .inspite of all the back and forth (particularly aggressive from newcomers I might add), I think Michael’s basic premise is being ignored.
If you expose digital the way you exposed film, you are 1) making a mistake, and 2) failing to fully exploit the full potential of sensors.
You can argue the merits. . .in point of fact there may indeed be confusion since Michael has blended traditional analog nomenclature with digital terms-which to be honest is entirely legit since we’re all struggling to blend analog with digital as well-you can choose to try to grasp the fundamental differences, or not.
The only thing I might argue about is trying to tell anything substantial from looking at a camera’s histogram in the first place.
It’s my speculation that the histogram is displaying a gamma encoded interpretation of a capture. . .at least for Canons, a raw capture and a jpg shot at the same exposure looks like the same histogram to me.
So, if that is indeed the case, you can’t really ever examine the linear capture histogram. I’m assuming here that the histogram function is using Canon’s default jpg gamma conversion for the purpose of determining the histogram information. Too bad really because if you are shooting raw, it would be useful to examine REAL information in the histogram instead of a gamma encoded interpretation. . .particularly if you aren’t using the Canon SDK for conversion (as would be the case using Camera Raw).
So, the question still remains. . .should one expose raw captures like a chrome or a neg? My contention is neither. . .one should learn to expose digital for the chip, your raw linear to gamma encoded conversion and the aesthetics of the image you are trying to create. Which I think is what Michael was trying to get across. . .
I might also take issue with pegging the dynamic range of digital captures at 5 stops. . .I would argue that the usable dynamic range of raw captures with Canons at least (10D and 1Ds) might be more in the 8-9 stop range. . .The BetterLight backs are getting in the 11 stop range. . .yes, one must still be careful of individual channel clipping even if the other channels don’t clip.
Some early digital shooters tended toward underexposing to avoid hilight blooming and blowing out specular detail. . .but the habit of underexposing also leads to a lot of shadow noise.
Personally, I’m still learning to understand exactly how sensors and conversions are effecting the images I try to create. I think it’s useful to examine Michael’s article for the underlying concepts and the impact they have on capture & conversion.
-
> I would be inclined to say the histogram is even more usless since it will not be a predictor of either C1 or CR's conversion. . .
Jeff, I rarely take a *close* look at the histo on camera, only using it as a sanity check (primarily watching for under-exposure as the flashing alerts me to possible over). For one thing the thing's darn hard to see out in the sun!
> If one takes the care to spot meter various tones in the scene,
That's the only method I use anymore, and after using it for a while you get surprisingly good at running with just a single spot reading in fast paced situations, knowing from experience where you should place it and where the others will fall. When leisure permits I'll meter both ends and probably the prominent subject also.
> I shoot is in the studio where I can control the scene's contrast.
Wish I had that luxury. My version is to wait for just the right thickness of cloud or fogbank to pass giving me the contrast I want, or the right time of day.
- DL
-
>With subjects of modest contrast range, exposing to the right is probably not of much practical value:
on one hand, it will give more levels at each subject brightness level, and these extra gradations will then be preserved when one does compensating EC downwards in 16-bit mode,
Please explain how the extra gradations will be preserved when you do compensating EC downwards. There must be something I'm missing here.
> Perhaps comparing these two approaches in the field is preferable than endless attempts to anwer the question theoretically.
My observations here are based on comparing exposure approaches in the field and references to theory are only to understand what I saw in the field.
> is that image data to the right side of the histogram is of higher quality (less noise, defined by more bits) than image data to the left.
Less noise, yes, but higher quality overall? It depends. I actually do use this technique on occasion but it involves shots where I'm not pulling the shadows back down, only the highlights. But my 'motivation' in these cases is not to arbitrarily push data to the right but to expose for the shadows (taking care to not blow highlights) in split tonality situations. This means that later I can pull highlights down rather than pushing shadows up resulting in cleaner shadows. I'll post an example later.
- DL
-
Victor, Don,
According to Chris Cox (Photoshop guru): Gamma encoding makes use of the eye's known sensitivity to store the data in a way that minimizes the visible difference between the reproduced and original scenes with a limited number of values (bits). In other words, the image is stored in a way that is perceptually uniform -- an equal number of pixel values are used for equal areas of perceived lightness.
You can read more here http://chriscox.org/gamma/ (http://chriscox.org/gamma/).
Here are additional sites with interesting information on gamma:
http://research.microsoft.com/~hollasch/cg...olor/gamma.html (http://research.microsoft.com/~hollasch/cgindex/color/gamma.html)
http://www.graphics.cornell.edu/~westin/gamma/gamma.html (http://www.graphics.cornell.edu/~westin/gamma/gamma.html)
http://www.aim-dtp.net/aim/evaluation/gamma_error/index.htm (http://www.aim-dtp.net/aim/evaluation/gamma_error/index.htm)
http://www.poynton.com/notes/colour_and_gamma/GammaFAQ.html (http://www.poynton.com/notes/colour_and_gamma/GammaFAQ.html)
http://www.cgsd.com/papers/gamma.html (http://www.cgsd.com/papers/gamma.html)
Here are a couple web sites that allow you to evaluate your monitor's gamma:
http://www.cs.berkeley.edu/~efros/java/gamma/gamma.html (http://www.cs.berkeley.edu/~efros/java/gamma/gamma.html)
http://www.aim-dtp.net/aim/evaluation/gamma_space/index.htm (http://www.aim-dtp.net/aim/evaluation/gamma_space/index.htm)
So what does it all mean?
Enjoy!
-- Jim
-
> Finally a film example, so far I have failed to get this much from a digital camera, I'll keep playing!
Oh I think digital will handle that, albeit possibly with a slightly different effect. Just watch out for artifacting in the highlights.
I have found the Fuji S2 sensitive to overexposure, and recovery using Raw files is limited.
I sill have my S2, but it has been joined by a Kodak 14n. I have been stunned by the Kodak Raw and ERI Jpeg +/- 2 stop exposure recovery. Goodbye blown highlights, hello incredible dynamic range and after the event exposure recovery, plus (with the latest software) great colour. This is how digital is meant to be. Its like bracketing with one shot. You hear a lot (reasonably) about the negatives (noise mainly) with the 14n, but (a) its not that bad, even at high-ish ISO and ( the detail plus this exposure thing is a wonder to behold.
I never believed I'd buy this camera, but I am glad I did. And I have the S2 to fall back on for low light.
Quentin :)