Luminous Landscape Forum

Equipment & Techniques => Digital Cameras & Shooting Techniques => Topic started by: wynpotter on June 19, 2006, 10:33:36 am

Title: Shooting to the right
Post by: wynpotter on June 19, 2006, 10:33:36 am
I have a CAnon 350d and if I shoot the meter bar at (0) then I seem to be underexposing. The histogram has about 1 stop more to the right
What is the general concensus on exposure  0,+.5, +1, ?
What seems to work best for you. Wyndham
Title: Shooting to the right
Post by: DarkPenguin on June 19, 2006, 10:44:27 am
Anything from -2 to +2.  Depending on the situation.
Title: Shooting to the right
Post by: wynpotter on June 19, 2006, 10:52:48 am
Another thought , are we calibrating our eye/brain to the image sensor?
Because it seem that each persons taste or perception of light would differ, but the sensor is a standard in the output, so to get the most pleasing image, we create our own subjective profile
Title: Shooting to the right
Post by: bjanes on June 19, 2006, 11:03:25 am
Quote
Another thought , are we calibrating our eye/brain to the image sensor?
Because it seem that each persons taste or perception of light would differ, but the sensor is a standard in the output, so to get the most pleasing image, we create our own subjective profile
[a href=\"index.php?act=findpost&pid=68540\"][{POST_SNAPBACK}][/a]

I would submit that exposure to the right has little to do with percieved lightness or darkness of the image, but rather with maximizing image information and minimizing noise. A good start on understanding ETTR can be realized by reading Michael's essay on the subject on this site.

If the ETTR exposure looks too light as shot, then one uses negative exposure adjustment to get the proper balance.
Title: Shooting to the right
Post by: Anon E. Mouse on June 19, 2006, 11:40:28 am
Finally, all images are subjective. The "exposing to the right" idea is just a rule of thumb (it used to be called "exposing for the highlights"). It ain't a hard an fast rule. It isn't a great way taken literally - the reasons camera metering does not use it. You may find the easiest "rule of thumb" is "expose for the subject." The imortant thing is your images look good.

In a sense, you are right that you need to learn to see how your camera sees. The human visual system plays a lot of tricks where your camera is very strict. Knowing how a scene will be reproduced by the camera is really important for determining exposures. There are no real rules for getting good exposures. The way the metering system works by using or averaging to a middle grey is about the closest thing to a "rule." But there are situations where that won't work. Experience is the best teacher.
Title: Shooting to the right
Post by: wynpotter on June 19, 2006, 11:49:47 am
I agree that experience is the best teacher and the feedback from others down this path sure helps.Wyndham
Title: Shooting to the right
Post by: bruce fraser on June 19, 2006, 12:29:03 pm
Quote
Another thought , are we calibrating our eye/brain to the image sensor?
Because it seem that each persons taste or perception of light would differ, but the sensor is a standard in the output, so to get the most pleasing image, we create our own subjective profile
[a href=\"index.php?act=findpost&pid=68540\"][{POST_SNAPBACK}][/a]

There is a huge variation in the sensititivity of the sensors in just about any DSLR model line. Nominal ISO 100 can really be anything from about 70 to about 170. The camera meter doesn't know this.

You need to learn the behavior of your camera and not worry about what other people are doing with exposure, because their cameras are almost certainly different from yours.
Title: Shooting to the right
Post by: Gary Ferguson on June 19, 2006, 12:42:57 pm
Quote
There is a huge variation in the sensititivity of the sensors in just about any DSLR model line.


That's interesting, and something I've never heard stated so clearly. Could this explain why my Canon 1Ds often suffered from burn't out highlights, where as my 1Ds MkII hardly ever has this problem?
Title: Shooting to the right
Post by: Gordon Buck on June 19, 2006, 12:43:04 pm
Interesting to me is that, as I push exposure to the right, ACR tends to adjust oppositely; that is, underexposes.

I realize that this overexposure/underexposure is some sort of combination of my own particular camera, my own adjustments -- not to mention eyesight and small screen, and ACR auto exposure.  Nonetheless, it does happen.

Sometimes I just overdo the "push to the right" concept but I'm becoming better calibrated with practice.

Sometimes the "problem" is obviously in ACR; or rather, a difference of opinion between me and ACR as to how much clipping to permit in the highlights.
Title: Shooting to the right
Post by: wynpotter on June 19, 2006, 01:08:55 pm
Quote
There is a huge variation in the sensititivity of the sensors in just about any DSLR model line. Nominal ISO 100 can really be anything from about 70 to about 170. The camera meter doesn't know this.
[a href=\"index.php?act=findpost&pid=68552\"][{POST_SNAPBACK}][/a]

So this opens another possibility. I started this thread based on the idea that the sensor was a calibrated ref point and I was the subjective variation. How do we find out what the true (reasonably) iso scale is for a sensor. Gray scale card and external light meter compared to profiled monitor? This is a bit "Alice in Wonderland" but interesting.
Wyndham
Title: Shooting to the right
Post by: bruce fraser on June 19, 2006, 01:25:56 pm
Quote
That's interesting, and something I've never heard stated so clearly. Could this explain why my Canon 1Ds often suffered from burn't out highlights, where as my 1Ds MkII hardly ever has this problem?
[a href=\"index.php?act=findpost&pid=68553\"][{POST_SNAPBACK}][/a]

If you haven't taken steps to nail down, and compensate for, the systematic inaccuracies in the camera's metering system then yes, that's entirely possible.
Title: Shooting to the right
Post by: abaazov on June 19, 2006, 02:05:39 pm
bruce what are you referring to when you say to take steps to nail down and compensate for the systematic inaccuracies in your camera's metering system?

amnon
Title: Shooting to the right
Post by: bjanes on June 19, 2006, 02:08:59 pm
Quote
There is a huge variation in the sensititivity of the sensors in just about any DSLR model line. Nominal ISO 100 can really be anything from about 70 to about 170. The camera meter doesn't know this.

You need to learn the behavior of your camera and not worry about what other people are doing with exposure, because their cameras are almost certainly different from yours.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=68552\")

With more advanced models from Nikon and Canon, I doubt there is that much variation. The Nikon guru Thom Hogan states that there is a nearly 0.5EV tolerance in manufacturing standards, but he states that in his evaluation of literally dozens of Nikons, he rarely seen any significant deviation between bodies.

Nonetheless, it is advisable to check the meter and DPReview does this routinely. In the D200 test, they stated that the meter was right on as is usual with Nikon cameras. They noted that Canons tend to give about 0.3 EV more exposure.

An easy way to check the meter is to take a picutre of a gray card. With a gamma of 2.2 and bit depth of 8 bits, the observed pixel value in the file should be 114. (Actually, the reflectance of the card makes little difference)

[a href=\"http://www.normankoren.com/digital_cameras.html#ISOspeed]http://www.normankoren.com/digital_cameras.html#ISOspeed[/url]

For exposure to the right, most authorities recommend that the highlight should be placed close to saturation and you can test the exposure compensation from the indicated meter reading to accomplish this at the same time.
Title: Shooting to the right
Post by: dlashier on June 19, 2006, 02:37:04 pm
Quote
There is a huge variation in the sensititivity of the sensors in just about any DSLR model line. Nominal ISO 100 can really be anything from about 70 to about 170. The camera meter doesn't know this.

You need to learn the behavior of your camera and not worry about what other people are doing with exposure, because their cameras are almost certainly different from yours.
[a href=\"index.php?act=findpost&pid=68552\"][{POST_SNAPBACK}][/a]

I recall reports from people who had two identical bodies that metered up to a 1/2 stop different due to sloppiness in factory calibration. In the case of owning two bodies it may be worth sending them in to have them equalized, but for the typical user you just need to learn the behaviour of your's and adjust accordingly. Matrix mode will also be fooled by some situations and underexpose as when taking a picture of a large white object (eg house or boat). You need to recognize this and compensate, either with +EV or by metering to the side and recomposing. I just avoid the whole issue by metering manually. In the end it's easier and more consistent and predictible.

- DL
Title: Shooting to the right
Post by: John Sheehy on June 19, 2006, 02:48:20 pm
Quote
Another thought , are we calibrating our eye/brain to the image sensor?
Because it seem that each persons taste or perception of light would differ, but the sensor is a standard in the output, so to get the most pleasing image, we create our own subjective profile
[a href=\"index.php?act=findpost&pid=68540\"][{POST_SNAPBACK}][/a]

That is an outlook that really applies to JPEG more than RAW.  The JPEGs out of the camera have compression of the highlights, where they are not necessarily clipped but are pale and posterized, so exposing too far too the right, even if it doesn't result in clipping, can compromise the quality of the highest highlights.

For RAW, however, most cameras are linear or very close to it right up to clipping (if this isn't true for the lowest ISO, it is still true for the other ISOs).  You can't expose RAW too far to the right, except for clipping, and that occurs a bit higher than JPEG clipping.
Title: Shooting to the right
Post by: John Sheehy on June 19, 2006, 02:51:47 pm
Quote
There is a huge variation in the sensititivity of the sensors in just about any DSLR model line. Nominal ISO 100 can really be anything from about 70 to about 170. The camera meter doesn't know this.

You need to learn the behavior of your camera and not worry about what other people are doing with exposure, because their cameras are almost certainly different from yours.
[a href=\"index.php?act=findpost&pid=68552\"][{POST_SNAPBACK}][/a]

So, when are we going to get RAW RGB histograms in our cameras?  As a RAW shooter, I find the histograms on current cameras only slightly better than worthless.
Title: Shooting to the right
Post by: bruce fraser on June 19, 2006, 02:58:31 pm
Quote
With more advanced models from Nikon and Canon, I doubt there is that much variation. The Nikon guru Thom Hogan states that there is a nearly 0.5EV tolerance in manufacturing standards, but he states that in his evaluation of literally dozens of Nikons, he rarely seen any significant deviation between bodies.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=68564\")

I shoot Canons so I can't speak for Nikon, and the only people who have experience with statistically significant numbers of units are the vendors, who aren't saying. But Tom Knoll, who has had to shoot a lot of cameras to build ACR, has seen the same kind of variations I have. Jeff Schewe and I compared Canon 20Ds and 300Ds (3 of each) and found differences of about 2/3 of a stop. Greg Gorman had two IDs's that were about a half stop different.

Quote
Nonetheless, it is advisable to check the meter and DPReview does this routinely. In the D200 test, they stated that the meter was right on as is usual with Nikon cameras. They noted that Canons tend to give about 0.3 EV more exposure.

An easy way to check the meter is to take a picutre of a gray card. With a gamma of 2.2 and bit depth of 8 bits, the observed pixel value in the file should be 114. (Actually, the reflectance of the card makes little difference)

[a href=\"http://www.normankoren.com/digital_cameras.html#ISOspeed]http://www.normankoren.com/digital_cameras.html#ISOspeed[/url]
[a href=\"index.php?act=findpost&pid=68564\"][{POST_SNAPBACK}][/a]

With all due respect to both you and Norman, a gray card isn't a terribly reliable method. On a linear capture, 18% reflectance is 18% of the way from black towards white—it's awfully dark. You really need to meter the highlight, preferably with an external meter that lets you bracket ISO's. Then you can compare which ISO is actually closest to camera behavior.

Canons have more headroom than Nikons, but a good chunk of that "headroom" is driving the chip into the nonlinear range where the captured pixels may or may not be useful. That's a different phenomenon to the one under discussion.

Quote
For exposure to the right, most authorities recommend that the highlight should be placed close to saturation and you can test the exposure compensation from the indicated meter reading to accomplish this at the same time.
[a href=\"index.php?act=findpost&pid=68564\"][{POST_SNAPBACK}][/a]
Title: Shooting to the right
Post by: John Sheehy on June 19, 2006, 03:10:00 pm
Quote
So this opens another possibility. I started this thread based on the idea that the sensor was a calibrated ref point and I was the subjective variation. How do we find out what the true (reasonably) iso scale is for a sensor. Gray scale card and external light meter compared to profiled monitor? This is a bit "Alice in Wonderland" but interesting.
Wyndham
[a href=\"index.php?act=findpost&pid=68559\"][{POST_SNAPBACK}][/a]

There isn't a lot out there to help people see what the camera is capturing; at least not with the typical software.  There are only 3 programs I am aware of that can show you the actual RAW capture; IRIS, ImagesPlus, and DCRAW with the right command line arguments.  Trying to surmise RAW exposure from the RAW converters of choice is really futile.

If you're dealing with matte subject in consistent lighting, you can generally expose a grey card for +1 EC, use those manual settings, and expose without clipping.  With my Canon 10D, I could meter for incident light with my Sekonic L-558 at ISO 32, and shoot the Gretag-Macbeth color checker with the meter settings with the camera set to ISO 100, and the white square would not clip in the RAW data.  I believe metering for ISO 25 just barely clipped the green channel.

It's scary when you consider that people could take important shots, going through all the trouble of accurately metering for ISO 100, and getting 3x the shadow noise possible metering for ISO 32.  Even ISO 200 metered for ISO 64 would have much less shadow noise.

A camera, set to an ISO, basically has two different ISO sensitivities; the ISO of average metered value, and the ISO of maximum unclipped white highlights.
Title: Shooting to the right
Post by: John Sheehy on June 19, 2006, 03:26:26 pm
Quote
Canons have more headroom than Nikons, but a good chunk of that "headroom" is driving the chip into the nonlinear range where the captured pixels may or may not be useful. That's a different phenomenon to the one under discussion.
[a href=\"index.php?act=findpost&pid=68573\"][{POST_SNAPBACK}][/a]

If true for a given camera, that would only apply to the lowest ISO.

My 20D has a non-linear RAW response in the very top of the highlights, only at ISO 100.

What would be RAW level 3550 at any other ISO, gets stretched up to 4095, and the stretch starts somewhere around 3000 (anything below about 3000 is the same at all ISOs).  This, however, is exactly the opposite of what you'd expect from sensor saturation; you'd expect a lack of response to further photons.  Canon seems to be intentionally boosting limited sensor highlights to give the illusion of full DR at ISO 100.  IMO, they should have just used a little more amplification so that the 4095 point would be just below saturation.  This non-linearity can easily cause a converter to get the WB of extreme highlights wrong.

I don't use ISO 100 for anything with important highlights.  200 is my target ISO, when lighting permits.
Title: Shooting to the right
Post by: bruce fraser on June 19, 2006, 03:47:34 pm
Quote
If true for a given camera, that would only apply to the lowest ISO.
[a href=\"index.php?act=findpost&pid=68576\"][{POST_SNAPBACK}][/a]

Good point.

Quote
My 20D has a non-linear RAW response in the very top of the highlights, only at ISO 100.
[a href=\"index.php?act=findpost&pid=68576\"][{POST_SNAPBACK}][/a]

Mine too.

Quote
What would be RAW level 3550 at any other ISO, gets stretched up to 4095, and the stretch starts somewhere around 3000 (anything below about 3000 is the same at all ISOs).  This, however, is exactly the opposite of what you'd expect from sensor saturation; you'd expect a lack of response to further photons.  Canon seems to be intentionally boosting limited sensor highlights to give the illusion of full DR at ISO 100.  IMO, they should have just used a little more amplification so that the 4095 point would be just below saturation.  This non-linearity can easily cause a converter to get the WB of extreme highlights wrong.

I don't use ISO 100 for anything with important highlights.  200 is my target ISO, when lighting permits.
[a href=\"index.php?act=findpost&pid=68576\"][{POST_SNAPBACK}][/a]

I use ISO 200 if the sun is in the frame, But I've found that for more reasonable DR subjects, If I spot-meter the highlights and dial in +2.3 stops, I rarely get badly-blown highlights at 100 ISO, and I get a fair bit less noise. Your mileage may vary in that you may need to dial in a different compensation depending on your camera's real ISO sensitivity.
Title: Shooting to the right
Post by: bjanes on June 19, 2006, 05:39:32 pm
Quote
I shoot Canons so I can't speak for Nikon, and the only people who have experience with statistically significant numbers of units are the vendors, who aren't saying. But Tom Knoll, who has had to shoot a lot of cameras to build ACR, has seen the same kind of variations I have. Jeff Schewe and I compared Canon 20Ds and 300Ds (3 of each) and found differences of about 2/3 of a stop. Greg Gorman had two IDs's that were about a half stop different.

[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=68573\")

Since I shoot Nikon and have never made an exposure with a Canon digital camera, I accept the observations of the above quoted experts for Canon metering. I don't want to start a flame over the relative merits of Canon or Nikon, but Thom Hogan has had quite a bit of experience with Nikons and states in his e-books that he has rarely found it necessary to adjust an exposure reading because of metering inaccuracy. Of course, one often needs to override the metering in evaluative modes.

Quote
With all due respect to both you and Norman, a gray card isn't a terribly reliable method. On a linear capture, 18% reflectance is 18% of the way from black towards white—it's awfully dark. You really need to meter the highlight, preferably with an external meter that lets you bracket ISO's. Then you can compare which ISO is actually closest to camera behavior.

[a href=\"index.php?act=findpost&pid=68573\"][{POST_SNAPBACK}][/a]

It's not Norman and me, but rather the ISO standard for determination of sensitivities in digital cameras that gives rise to the above method. The camera does not know the difference between a gray card in sunlight or a white card in room light. It merely causes the exposure to result in the mid gray value for which the meter is calibrated. If you want a higher luminosity, take the reading in bright sunlight or even use a white card in sunlight. You still should get a value of 114.

As I stated, in ETTR I agree one should use a highlight reading and not a mid gray reading. In this case, one needs to use an offset from the ISO as determined from a gray card reading.

Quote
Canons have more headroom than Nikons, but a good chunk of that "headroom" is driving the chip into the nonlinear range where the captured pixels may or may not be useful. That's a different phenomenon to the one under discussion.

[a href=\"index.php?act=findpost&pid=68573\"][{POST_SNAPBACK}][/a]

I've heard this mentioned many times, but my experiments with the Nikon D70 and D200 show that at base ISO these cameras are linear right up to clipping. Roger Clark has noted similar behavior for the Canon EOS ID Mark II at ISO 50 (see figure 3)

[a href=\"http://www.clarkvision.com/imagedetail/evaluation-1d2/index.html]http://www.clarkvision.com/imagedetail/eva...-1d2/index.html[/url]

I have noted that when Adobe Camera Raw indicates clipping of highlights with no exposure compensation the raw file as shown by a conversion with DCRaw does show head room of 0.6 EV or so. How does the camera communicate its desired headroom to the raw converter?
Title: Shooting to the right
Post by: bjanes on June 19, 2006, 10:18:08 pm
Quote
So, when are we going to get RAW RGB histograms in our cameras?  As a RAW shooter, I find the histograms on current cameras only slightly better than worthless.
[a href=\"index.php?act=findpost&pid=68572\"][{POST_SNAPBACK}][/a]

I once suggested a raw histogram (on the Adobe ACR forum, I think), and Bruce Fraser replied that it would not be useful because everything would be shifted so far to the left that it would be hard to read. He has a good point there, and a histogram with a gamma of 2.2 applied to the raw data might be more useful.

Since most cameras allow some headroom, would you want clipping in the histogram to coincide with clipping in ACR without exposure adjustment or would you want it to show clipping only when the sensor starts to clip or the ADC starts to overflow? Once you decide what you want, it is a relatively simple matter to upload a custom tone curve to the camera to get the desired results.

White balance is another consideration. If you are shooting with daylight color balance, the gain in the red and blue channels has to be increased by a factor of 1.83 and 1.36 respectively for a Nikon D200. Other Nikons and many Canons have similar multipliers. You may expose so the green channel is fully to the right, but the red and blue channels will be relatively underexposed. Do you want your histogram to show the true status of the color channels without white balance? If so you can load a special white balance into the camera such that the red and blue channel multipliers are 1.0 (UniWB).

Do you want the channels with daylight to be better balanced? If so, put a magenta filter over the lens to hold back some of the green light.

In sumary, a raw histogram is already available but it is not worth the trouble for most people. For practical shooting it is often sufficient to know how the camera histogram with default tonal settings relates to the ACR or other raw converter rendering at default settings. This can be determined by a bit of testing. Similarly, most photographers do not bother with a magenta filter. I have UniWB uploaded to bank 4 of my D200, but don't use it very often.
Title: Shooting to the right
Post by: Gregory on June 20, 2006, 01:43:14 am
Quote
I have a CAnon 350d and if I shoot the meter bar at (0) then I seem to be underexposing. The histogram has about 1 stop more to the right
What is the general concensus on exposure  0,+.5, +1, ?
What seems to work best for you.[a href=\"index.php?act=findpost&pid=68538\"][{POST_SNAPBACK}][/a]
I use a 350D too.

I use Full Frame exposure sensing which might affect the results we see. I use full frame because I hope to get as much of the histogram in the picture as possible.

I have found however that my 350D is prone to highlight blowouts so I typically shoot with EC of -1/3 and sometimes -2/3. if the lighting changes radically, I review the camera histogram, make adjustments accordingly and shoot another 1 or 2 photos if the subject is still there.

it's a balancing trick though. sometimes, preserving the highlights (which I can always blow out manually in Aperture if I choose to up the exposure or change the curve) will sacrifice detail in the low light areas depending on the overall contrast of the scene.

I guess though that if it was easy, it wouldn't be so much fun.
Title: Shooting to the right
Post by: Jonathan Wienke on June 20, 2006, 09:37:03 am
Quote
I have noted that when Adobe Camera Raw indicates clipping of highlights with no exposure compensation the raw file as shown by a conversion with DCRaw does show head room of 0.6 EV or so. How does the camera communicate its desired headroom to the raw converter?

It doesn't. The headroom setting is built into ACR's built-in profiles.

A RAW histogram would be trivially simple to implement. You wouldn't need to do any Bayer interpolation, white balance adjustments, or any color processing whatsoever. Simply create a bar graph with 32 segments from left to right (32 vertical bars going left to right) with each bar representing 1/3 stop of exposure. The rightmost bar should be red, and its height should represent the number of clipped pixels in the RAW data. The next two bars should be yellow, and their heights represent pixels that are within 1/3 stop of clipping, and between 2/3 and 1/3 of a stop from the clip value, respectively. The remaining bars should represent successively decreasing 1/3-stop exposure intervals, with the leftmost bar or three colored yellow to indicate the possibility of increased noise levels in those exposure levels. A simple lookup table in the camera firmware would suffice to indicate which bar any given RAW value should be assigned, to make the 1/3 stop per bar paradigm work properly. This would require far less firmware programming than the current converted RAW-to-JPEG bastardgram, work perfectly with any sensor regardless of whether it was Bayer-pattern or monochrome, and indicate to the user the exact exposure adjustment necessary to achieve ideal exposure with a single "polaroid" exposure test shot. What's not to like?
Title: Shooting to the right
Post by: Ray on June 20, 2006, 10:37:44 am
You sometimes come up with some good ideas, Jonathan. It would be interesting to get some input from people at the coal face to see if there are any insurmountable technological obstacles to implementing your idea.
Title: Shooting to the right
Post by: wynpotter on June 20, 2006, 11:46:06 am
I wish to thank everyone for the ideas and comments. It has opened more points to understand and also starting to realize some of the limitations that the histogram and other feature have.
I'm sure as time and new cameras evolve into 16 and 24 bit, we will look back at these days much  the same as the first PCs  w a 10 meg hard disk.

I love the ride, thanks for the time. Wyndham
Title: Shooting to the right
Post by: bjanes on June 20, 2006, 12:45:32 pm
Quote
It doesn't. The headroom setting is built into ACR's built-in profiles.

A RAW histogram would be trivially simple to implement. You wouldn't need to do any Bayer interpolation, white balance adjustments, or any color processing whatsoever.

This would require far less firmware programming than the current converted RAW-to-JPEG bastardgram, work perfectly with any sensor regardless of whether it was Bayer-pattern or monochrome, and indicate to the user the exact exposure adjustment necessary to achieve ideal exposure with a single "polaroid" exposure test shot. What's not to like?
[a href=\"index.php?act=findpost&pid=68628\"][{POST_SNAPBACK}][/a]

Well, I for one think it would be an interesting option that would be useful to knowledgable photographers. I don't know what Bruce would think of a gamma 1 histogram-- he didn't like the idea previously.

Without any white balance the red and blue histograms could be difficult to interpret since they would be to the left with daylight white balance.

I think most users would prefer a white balanced gamma 2.2 histogram that showed clipping accurately.
Title: Shooting to the right
Post by: John Sheehy on June 20, 2006, 01:34:05 pm
Quote
I once suggested a raw histogram (on the Adobe ACR forum, I think), and Bruce Fraser replied that it would not be useful because everything would be shifted so far to the left that it would be hard to read. He has a good point there, and a histogram with a gamma of 2.2 applied to the raw data might be more useful.

Either way would be an improvement.  I look at RAW linear histograms, and don't find them hard to read, except that the upper stop is spread wide enough that the values are so spread out that they don't stack up.  That could be easily addressed, though, by scaling the vertical more, to the right.  Any histogram, IMO, should have a thick bar for the clipping value, so it is easy to see.  The ones in my Canons are only thin dotted lines (not even solid).

A RAW RGB histogram need not be linear to be useful.  It just needs to be RAW, without all the nonsense introduced by conversions that have nothing to do with exposure.

If consistency with the traditional histogram is a concern, I really don't think that this is useful for RAW shooting.  Traditional histogram checking is mainly for checking the tones in your jpeg.  I am only interesting in how much I'm clipping, or how far away I am from it.  I am not looking for an ideal hump in a certain shape in a certain place.

Quote
Since most cameras allow some headroom, would you want clipping in the histogram to coincide with clipping in ACR without exposure adjustment or would you want it to show clipping only when the sensor starts to clip or the ADC starts to overflow?

Clipping is rarely a sensor event; it is usually a digitization event.

If the histogram shows you RAW r, g, and b separately, then you have the information you need for many purposes.  You could determine where on the histogram your converter is reliable; you can avoid clipping all channels, or avoid clipping all but one if that's what you want.

I want to see the real RAW histogram (gamma is relatively irrelevant).  If current converters can't use the data properly, then they need to be updated.  Every RAW converter should have a true linear gain stage before any other processing, just like the EC dial on the camera.  Unfortunately, some drag curves around along with the exposure (ACR nails extreme RAW highlights to 255 in the output, while the midtones go almost black with -4).  There should be total linear freedom before any curves are applied; most converters act as if exposing to the right is not really a viable option, and RAW highlights are specular in nature.  A converter also needs to know where clipping occurs in the RAW data.  This does not always happen at 4095.  Many Nikon and Canon cameras clip at different levels in different vertical lines of pixels (10D, 30D, and D200).  The converter should detect clipping patterns, at least as an option, and clip uniformly.  It should also look for patterns of different amplification in different lines.  The are numerous artifacts in RAW data which can easily be circumvented.

Quote
Once you decide what you want, it is a relatively simple matter to upload a custom tone curve to the camera to get the desired results.

There's no such thing with Canons.  All you can do is try to bring the JPEG parameters as close as possible, but they are still far off.  The frequency response of the red, green, and blue filters in the cameras are not the receptive equivalent of the output of a calibrated monitor, even after WB.  The correlation between RAW and an RGB output image is very weak with many saturated colors; saturation changes and hue shifts are performed on the review image, upon which the histogram is based.  You could have red RAW values in the same image of 700 in both a grey and a red flower, and the red flower's 1000 could clip the review and red histogram, but be only 150 (out of 255) in the grey.

Quote
White balance is another consideration. If you are shooting with daylight color balance, the gain in the red and blue channels has to be increased by a factor of 1.83 and 1.36 respectively for a Nikon D200. Other Nikons and many Canons have similar multipliers. You may expose so the green channel is fully to the right, but the red and blue channels will be relatively underexposed. Do you want your histogram to show the true status of the color channels without white balance? If so you can load a special white balance into the camera such that the red and blue channel multipliers are 1.0 (UniWB).

I don't want any white-balancing in my RAW histogram; then it is no longer a RAW histogram.

Quote
Do you want the channels with daylight to be better balanced? If so, put a magenta filter over the lens to hold back some of the green light.

I've been doing that for a couple of years, but the histogram is still a JPEG histogram.  There is much more difference between RAW and JPEG than just WB and gamma.

Quote
In sumary, a raw histogram is already available but it is not worth the trouble for most people.

I don't think so.

And what "trouble", if the camera actually had a RAW histogram option?  You could see the review with the selected (or auto) WB, but see the histogram in all its RAWness.  No need to discolor the JPEGs for a not-quite-RAW histogram.

Quote
For practical shooting it is often sufficient to know how the camera histogram with default tonal settings relates to the ACR or other raw converter rendering at default settings. This can be determined by a bit of testing. Similarly, most photographers do not bother with a magenta filter.

99.9% of digital photographers don't even know that daylight isn't the native WB of the camera.  When I first posted the idea of using a magenta filter a few years ago, googling showed no one ever mentioning it before me.  So, it is safe to assume that very few people are even aware of the possibility of improving the red and blue channels in daylight shooting.

Support is slow.  I don't recall if the recent versions improve on it, but at one point ACR couldn't even WB an area where RAW R=G=B; it was outside its tint range.  The CC30M filter I was using, however, works in daylight with ACR 3.3 (it's not a perfect daylight correction, though).  I would use a magenta filter more often if I had a RAW RGB histogram.  It's hard enough trying to guess the relationship of histogram to RAW in daylight WB; WBing for the magenta filter in the camera gives another relationship to know, so bracketing is probably the best policy.

Quote
I have UniWB uploaded to bank 4 of my D200, but don't use it very often.[a href=\"index.php?act=findpost&pid=68604\"][{POST_SNAPBACK}][/a]

Well, it's inconvenient to have your JPEG thumbnails with a color cast, and dark.  And what you get is still probably very different from RAW.

I want a RAW histogram (linear or gamma-adjusted) with separate R, G, and B, and a review image that is sRGB, but flashes where the RAW is clipping.  The flashing could be black alternating with the channel or channels that are clipped; for example, alternating between black and yellow where the red and green channels are clipped; white where they all are clipped, blue where only the blue is clipped.
Title: Shooting to the right
Post by: John Sheehy on June 20, 2006, 01:45:58 pm
Quote
You sometimes come up with some good ideas, Jonathan. It would be interesting to get some input from people at the coal face to see if there are any insurmountable technological obstacles to implementing your idea.
[a href=\"index.php?act=findpost&pid=68633\"][{POST_SNAPBACK}][/a]

I'm sure that the real obstacle is marketing, combined with a lack of vision or demand.

There are so many things that could be implemented in digital cameras at almost no extra manufacturing cost, that we never see.

How can I tell my camera to adjust the shutter speed automatically in Tv-priority mode, based on how I zoom the focal length, when shooting in low light?  How hard could something that useful be?  How to tell it to under-expose by up to a stop befpre opening the lens all the way up on those fast lenses, with their horrible wide-open optics?
Title: Shooting to the right
Post by: John Sheehy on June 20, 2006, 01:59:09 pm
Quote
Well, I for one think it would be an interesting option that would be useful to knowledgable photographers. I don't know what Bruce would think of a gamma 1 histogram-- he didn't like the idea previously.

Jonathan's histogram model is not a gamma-adjusted one; it is logarithmic.  2^x, as opposed to x^2.2 (or some other gamma value).

Quote
Without any white balance the red and blue histograms could be difficult to interpret since they would be to the left with daylight white balance.

A RAW RGB histogram serves a different purpose than a JPEG histogram.  There is nothing to interpret about the red and blue channels, if they are to the left, except that you will blow the green channel by exposing much more, or that a purple, red, or magenta filter might help even the channels out a bit.  It's not like you need to see minute details of the red and blue curves.

Quote
I think most users would prefer a white balanced gamma 2.2 histogram that showed clipping accurately.
[a href=\"index.php?act=findpost&pid=68650\"][{POST_SNAPBACK}][/a]

Showed clipping of *what* accurately?  You would need 3 different right edges, one for each channel, to show the RAW clipping points, if there is WB applied.  That would be visually complex.

If you clipped everything where the most sensitive channel clipped after WB, then you won't see your highlight headroom for red highlights.  Should one under-expose a red highlight because the histogram clips away the red highlights?
Title: Shooting to the right
Post by: bjanes on June 20, 2006, 03:04:51 pm
Quote
I want a RAW histogram (linear or gamma-adjusted) with separate R, G, and B, and a review image that is sRGB, but flashes where the RAW is clipping.  The flashing could be black alternating with the channel or channels that are clipped; for example, alternating between black and yellow where the red and green channels are clipped; white where they all are clipped, blue where only the blue is clipped.
[a href=\"index.php?act=findpost&pid=68654\"][{POST_SNAPBACK}][/a]

John,

All in all, a brilliant post and I hope Thomas Knoll is reading it. Although I agree with most all of your suggestions, I do not think that they will be implemented in the near future since most users would not know how to make use of these advanced features. The work arounds that I suggested can be put in use immediately if your camera supports them.

Bill
Title: Shooting to the right
Post by: Anon E. Mouse on June 20, 2006, 11:49:17 pm
Quote
You sometimes come up with some good ideas, Jonathan. It would be interesting to get some input from people at the coal face to see if there are any insurmountable technological obstacles to implementing your idea.
[a href=\"index.php?act=findpost&pid=68633\"][{POST_SNAPBACK}][/a]

There is one little problem. RAW data does not make any pictorial sense. Nor can you get any infomation off it as it is RAW - Jonathan's 1/3 stop increments are a cute idea, but how can you do that without a gamma correction and Bayer interpretation? If you make those adjustments then it is not RAW. Without Bayer intrepretation you end up with some strange data. Even RAW converters need to convert the data in order for you to preview it. The tools given on cameras are sufficient for proper exposures. Simply learn how to use them.

I am glad cameras are not designed on forums. And if a "RAW" histogram were possible and useful, the camera companies would have done them - it is a tough market and they are all looking for an edge no matter how small.

- From one who has just retired from the "coal face."
Title: Shooting to the right
Post by: Jonathan Wienke on June 21, 2006, 03:36:06 am
Here's some examples of what my proposed RAW histogram would look like:

Ideal exposure (highlights not clipped, but close):
(http://www.visual-vacations.com/images/RAW-Histogram-Ideal.gif)

2/3 stop overexposure (some highlight clipping):
(http://www.visual-vacations.com/images/RAW-Histogram-Overexposed.gif)

2/3 stop underexposure:
(http://www.visual-vacations.com/images/RAW-Histogram-Underexposed.gif)

Quote
There is one little problem. RAW data does not make any pictorial sense. Nor can you get any infomation off it as it is RAW - Jonathan's 1/3 stop increments are a cute idea, but how can you do that without a gamma correction and Bayer interpretation? If you make those adjustments then it is not RAW. Without Bayer intrepretation you end up with some strange data.

None of these objections have any relevance whatsoever. You certainly can get information from a RAW file; if you couldn't it would have no usable image data. The whole point of the exercise is to tell if the RAW data is clipped; something you can't really tell once Bayer interpolation and all other normal conversion processing has been done. It's no harder to build a histogram off RAW data than interpolated RGB data; the process is exactly the same--grouping data values together into a given number of "bins" going left to right as the values increase, and assigning a height to each "bin" based on the number of pixels that fall into each "bin". The fact that RAW pixels have a color mask over them is irrelevant to the creation of a RAW histogram. When shooting a white card, you may see three individual spikes in the RAW histogram that will move around depending on the color temp of the lighting, but regardless of where they are, the goal is to get the rightmost one just barely out of the clipped zone. It doesn't matter whether the rightmost spike represents red-masked pixels, blue-masked pixels, or green-masked pixels.

Each bar represents 1/3 of a stop's worth of exposure range. The gray and white horizontal lines at the top and bottom of the histogram represent 1 stop exposure increments. Assigning a RAW data value to a particular bar in the histogram is done by a lookup table in the camera that ensures that a 1/3-stop exposure change will move the histogram one bar to the right or left. Assuming 12-bit RAW values, 4095 would be assigned to the rightmost bar, indicating clipping. Levels 3250-4094 are assigned to the next bar to the left, the next bar would get levels 2580-3249, the next would get levels 2048-2579, the next would get levels 1625-2047, etc. The linear behavior of the CCD and ADC makes this easy to do.

There's no real need for an RGB RAW histogram unless the camera has a Foveon sensor. The whole point of a RAW histogram is to see the data before color interpolation and other processing has been done. If any color channel is being clipped, the RAW histogram will reflect this wthout any color interpolation. The RAW histogram will also accurately warn of over and underexposure in lighting conditions with extremely high or low color temperatures.
Title: Shooting to the right
Post by: Anon E. Mouse on June 21, 2006, 04:52:35 am
Quote
None of these objections have any relevance whatsoever. You certainly can get information from a RAW file; if you couldn't it would have no usable image data. The whole point of the exercise is to tell if the RAW data is clipped; something you can't really tell once Bayer interpolation and all other normal conversion processing has been done.

Naturally you can get information from a RAW but it must be converted if you want an image.

Since you have to convert RAW data, and that converted data is the data you are going to use, then it is more important to see the converted data and how that affects the illuminance distribution. RAW data in and of itself is pretty unless it is converted and especially without Bayer interpretation - the Bayer pattern is dominant in green so color dominance of a scene will not affect the RAW histogram in a linear fashion, but it will with a converted histogram. While the RAW data technically has no color information, it is still affected by the tri-stimulous affect of the Bayer pattern. Until that information has been interpreted, you have very strange data. The histogram of the JPG thumbnail is far more useful.

Also, highlight and shadow warnings are already in cameras. They can simply show when pixels approach the white and black points. They cannot show when they exceed them (nor can your RAW histogram). You cannot show what does not exist.

I think the lack of any RAW histogram in cameras today speak of its importance. This is certainly not the first time the idea has surfaced on the net or been proposed to manufacturers, but no one bites. As I said before, the tools on cameras is sufficient to get good exposures. Just like in the past, photographers need to master their craft rather than depend on technology to do it for them.
Title: Shooting to the right
Post by: John Sheehy on June 21, 2006, 08:39:56 am
Quote
There is one little problem. RAW data does not make any pictorial sense.

That's not really true, it's just a different way of storing an image.

Quote
Nor can you get any infomation off it as it is RAW - Jonathan's 1/3 stop increments are a cute idea, but how can you do that without a gamma correction and Bayer interpretation?

Personally, I want more than 1/3 stop resolution.  The distribution within the last 1/3 stop can make the difference between me increasing exposure or not.  I also would like to see all three colors super-imposed.  His idea, however, is quite do-able.  All you have to do to the RAW data is subtract the blackpoint, and then divide the logarithm of the adjusted RAW values by the logarithm of 2, to get relative values in stops, and scale by a fixed amount to correspond to the bar number.

Jonathan's histogram has nothing to do with gamma.  It is stop-based, and therefore logarithmic.  The histogram is a logarithmic representation of linear data.  That doesn't change its RAWness, just the charting.  Gamma serves no purpose in this context.

Bayer demosaicing is totally unnecessary for a RAW histogram.  A RAW histogram is for displaying what was captured; not what it is going to become.

Quote
If you make those adjustments then it is not RAW. Without Bayer intrepretation you end up with some strange data.

Not at all.  Without interpolation, you have basically three greyscale captures.  One at 1/2 the total megapixels with a green filter, and blue-filtered and red-filtered images at 1/4 the total MP each.  There is nothing mystical or magical about RAW data.  In its most literal display, it is a greyscale linear image that has a checker-board texture based on the different sesnitivities of the color filters, and the colors of the subject.  I've looked at thousands of RAW images this way, and many other ways.

Quote
Even RAW converters need to convert the data in order for you to preview it. The tools given on cameras are sufficient for proper exposures. Simply learn how to use them.

- From one who has just retired from the "coal face."
[a href=\"index.php?act=findpost&pid=68715\"][{POST_SNAPBACK}][/a]


Not by a long shot - the camera's feedback varies by multiple stops, relative to RAW exposure.
Title: Shooting to the right
Post by: John Sheehy on June 21, 2006, 08:47:11 am
Quote
There's no real need for an RGB RAW histogram unless the camera has a Foveon sensor. The whole point of a RAW histogram is to see the data before color interpolation and other processing has been done[a href=\"index.php?act=findpost&pid=68727\"][{POST_SNAPBACK}][/a]

If you're in a situation where you can control the light color, youcan color it so that all three channels are just short of clipping with RGB.

Of course, even getting a color-blind RAW histogram would be a big improvement over what we have now.  RGB could be optional, for those who need to know the color.
Title: Shooting to the right
Post by: Ronny Nilsen on June 21, 2006, 08:53:53 am
Quote
Here's some examples of what my proposed RAW histogram would look like:

Ideal exposure (highlights not clipped, but close):
Title: Shooting to the right
Post by: bjanes on June 21, 2006, 09:09:55 am
Quote
Naturally you can get information from a RAW but it must be converted if you want an image.
[a href=\"index.php?act=findpost&pid=68729\"][{POST_SNAPBACK}][/a]
That is true, but no image is needed for a histogram. One merely needs to place the pixels in Jonathin's bins according to their magnitude.
Quote
Since you have to convert RAW data, and that converted data is the data you are going to use, then it is more important to see the converted data and how that affects the illuminance distribution. RAW data in and of itself is pretty unless it is converted and especially without Bayer interpretation - the Bayer pattern is dominant in green so color dominance of a scene will not affect the RAW histogram in a linear fashion, but it will with a converted histogram.
[a href=\"index.php?act=findpost&pid=68729\"][{POST_SNAPBACK}][/a]
Perhaps Jonathin should divide the number of green pixels in any bin by two so as to place them on an equal footing with the red and blue pixels. Otherwise, a blown red or blue channel would cause no more than 25% of pixels to be affected. The problem of the differing multipliers for the channels has already been discussed by John Sheehy.
Quote
While the RAW data technically has no color information, it is still affected by the tri-stimulous affect of the Bayer pattern. Until that information has been interpreted, you have very strange data. The histogram of the JPG thumbnail is far more useful.
[a href=\"index.php?act=findpost&pid=68729\"][{POST_SNAPBACK}][/a]
Technically, the red, blue, and green channels in a JPG have no color information either--they are monochrome. So what is the point? If you took a raw image and displayed it on the screen so that the blue, green and red pixels of the sensor go to the corresponding pixels on the screen and viewed the screen from a distance, the colors would merge in the eye just as they do with an image on your TV. Interpolation does allow for a smoother image.
Quote
Also, highlight and shadow warnings are already in cameras. They can simply show when pixels approach the white and black points. They cannot show when they exceed them (nor can your RAW histogram). You cannot show what does not exist.
[a href=\"index.php?act=findpost&pid=68729\"][{POST_SNAPBACK}][/a]
One has to relate the histogram to the distribution of luminances in the image. If the histogram shows a large number of pixels at 4095 in a 12 bit raw image, there is a problem unless you are taking a picture of a white wall fully exposed to the right. Even with ETTR, it doesn't hurt to leave a little headroom, and one can consider pixels with a value of 4095 to be blown. It does little harm to back off exposure so that these pixels are 4094. On the otherhand, one may choose to allow specular highlights or non-essential highlights in a high dyanmic range situation to blow. Here is when a blinking highlight indicator can be helpful. Of course, the sensor may saturate below pixel value of 4095 at base ISO. At higher ISOs, overflow tends to occur in the AD converter. These complications can be addressed with appropriate techonolgy.
Quote
I think the lack of any RAW histogram in cameras today speak of its importance. This is certainly not the first time the idea has surfaced on the net or been proposed to manufacturers, but no one bites. As I said before, the tools on cameras is sufficient to get good exposures. Just like in the past, photographers need to master their craft rather than depend on technology to do it for them.
[a href=\"index.php?act=findpost&pid=68729\"][{POST_SNAPBACK}][/a]
This is a Luddite view. Do you remember Ken Olson's statement that no one would have any need for a computer in his home? The existing tools allow for decent exposures, but they allow quite a bit of headroom for the highlights. One can often expose 0.5 to 1 EV further to the right than these tools would indicate, and get better tonal values and less noise.  The current tools work fine for JPEG, but better tools are needed for raw with explosure to the right. Give the knowledable photographer better tools, and he or she will find a use for them. If the world had taken Olson's advice, there would be no Photoshop.
Title: Shooting to the right
Post by: Jonathan Wienke on June 21, 2006, 10:57:48 am
A few additional comments:

If you really need an RGB RAW histogram, you can seperate the RAW pixels by filter color and generate a separate histogram display for each filter color. As has been pointed out, the vertical scale of the green channel needs to be half of the other channels to keep everything even.

You can generate a RAW histogram with any exposure increment you desire. The interval between each successive bar is 1/(2^x), where x is the number of bars per stop. I used 1/3 stop increments because most cameras offer 1/3 stop as their smallest exposure adjustment, and a 32-bar display nicely indicates a 10-stop range, with 2 bars left over for a clipping indicator on the right and a noise/underexposed-garbage indicator on the left. The color of the bars is meant to indicate the useful data range of the camera, with yellow indicating ares where one is still OK, but caution is indicated, and red areas where reshooting is likely the best option. For most current digital cameras, a 10-stop range tracks with the abilities of the camera pretty well. It's wide enough to be more than useful, without being ridiculously wider than the camera's capabilities.
Title: Shooting to the right
Post by: Eric Myrvaagnes on June 21, 2006, 11:38:43 am
I would buy a camera with Jonathan's histogram in a minute, even if it didn't have mirror lockup (that may be another 10 - 20 years down the road for Canon at least). Thinking about how I use the current camera histogram, the only information I try to get from it is how close to clipping it is on both ends. Jonathan's system (perhaps with some of the sugested modifications) sounds perfect for that.

Eric
Title: Shooting to the right
Post by: bjanes on June 21, 2006, 11:39:43 am
Quote
I recall reports from people who had two identical bodies that metered up to a 1/2 stop different due to sloppiness in factory calibration. In the case of owning two bodies it may be worth sending them in to have them equalized, but for the typical user you just need to learn the behaviour of your's and adjust accordingly.
- DL
[a href=\"index.php?act=findpost&pid=68570\"][{POST_SNAPBACK}][/a]

As I read Bruce's post, he is talking about variations in the sensitivity of the sensor to light, not variations in metering. If the meter is well calibrated, but the sensor is not operating at the nominal ISO, then the exposure will be off. The exposure guided by an accurate hand held meter will also be off.

The simple test I mentioned with a gray card tests the system response, which is most important in an actual shooting situation.  One way to test the camera meter would involve comparing the camera reading with a that of an external meter which is known to be accurate. For accurate comparison, one would have to take into account the transmission of the lens and other factors.
Title: Shooting to the right
Post by: Jonathan Wienke on June 21, 2006, 01:14:20 pm
If you really want a color RAW histogram, you could have it look like this:

Ideal:
(http://www.visual-vacations.com/images/RAW-RGB-Histogram-Ideal.gif)

Over 2/3:
(http://www.visual-vacations.com/images/RAW-RGB-Histogram-Over.gif)

Under 2/3:
(http://www.visual-vacations.com/images/RAW-RGB-Histogram-Under.gif)

Just remember that the green channel scaling must be divided by two to match the red and blue channels. No bayer interpolation is necessary, simply sort the pixels by their filter color.
Title: Shooting to the right
Post by: Anon E. Mouse on June 22, 2006, 03:52:28 am
Well guys, I am not convinced. Why don't you show me. Take a average scene - landscape on a sunny day with a few puffy clouds either 2 hours before or after noon. Then:

1. Post the image.

2. Post the RAW histogram with no changes to the data.

3. Post the histogram of the camera thumbnail.

4. Post the histogram of the converted RAW data.

Then lets see what we get.
Title: Shooting to the right
Post by: John Sheehy on June 22, 2006, 08:45:28 am
Quote
Well guys, I am not convinced. Why don't you show me. Take a average scene - landscape on a sunny day with a few puffy clouds either 2 hours before or after noon.

You have chosen a poor example.  The biggest problems that a RAW histogram would address are with colored highlights.  For white clouds with my camera, the RAW data clips in the green channel about 1/3 stop higher than where the histogram clips, with contrast set to -2.

A red flower is a much more suitable subject.

Quote
Then:

1. Post the image.

2. Post the RAW histogram with no changes to the data.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=68848\")

What is "the image"?

Here's some red flowers, JPEG on one the right, and a linear RAW interpolation on the left:

[a href=\"http://www.pbase.com/jps_photo/image/61233204]crops[/url]

Here's a RAW RGB histogram (bottom), and an RGB histogram of the JPEG (top):

histograms (http://www.pbase.com/jps_photo/image/61277720)

My camera doesn't have a color histogram, but if it did, the red channel would be clipped, which may cause someone to under-expose a second shot, when in fact, they should be increasing exposure by about 1.5 stops.
Title: Shooting to the right
Post by: bjanes on June 22, 2006, 09:29:25 am
Quote
Well guys, I am not convinced. Why don't you show me. Take a average scene - landscape on a sunny day with a few puffy clouds either 2 hours before or after noon. Then:

1. Post the image.

2. Post the RAW histogram with no changes to the data.

3. Post the histogram of the camera thumbnail.

4. Post the histogram of the converted RAW data.

Then lets see what we get.
[a href=\"index.php?act=findpost&pid=68848\"][{POST_SNAPBACK}][/a]

I agree with John Sheehy that this scenario is not a good one to demonstrate the advantages of a raw histogram. However, it does demonstrate some disadvantages.

Here is such a scene exposed pretty much to the right. The highlights are just short of clipping in all channels.

[attachment=729:attachment]

Here is the raw conversion (DCRaw) with the histogram. In this case, I would have to agree with Bruce Fraser. Everything is to the left and the status of the highlights is very difficult to assess.

[attachment=728:attachment]
Title: Shooting to the right
Post by: Jonathan Wienke on June 22, 2006, 10:00:29 am
Quote
Here is the raw conversion (DCRaw) with the histogram. In this case, I would have to agree with Bruce Fraser. Everything is to the left and the status of the highlights is very difficult to assess.

Which is why I proposed a bar chart format with each bar representing 1/3 stop of exposure range, instead of a linear or standard gamma-adjusted display. You are comparing apples to field mice.

A histogram is nothing more than a chart displaying a distribution of values over a given range. One can be made from any data set, whether audio, video, still image, uninterpolated RAW data values, ASCII text, or stock market quotes. There's absolutely no reason whatsoever RAW data values "must" be interpolated or otherwise processed into a visually pleasing image before a histogram can be made. The whole problem we're trying to solve here is the errors introduced into the exposure evaluation process when white balance and color interpolation is performed on the data prior to running the histogram generation routine. Making a histogram from the unprocessed RAW data (especially when the histogram is segmented to exactly match the camera's exposure adjustment interval) will make judging correct exposure from the histogram much easier, not harder.
Title: Shooting to the right
Post by: John Sheehy on June 22, 2006, 02:01:12 pm
Quote
In this case, I would have to agree with Bruce Fraser. Everything is to the left and the status of the highlights is very difficult to assess.
[attachment=728:attachment]
[a href=\"index.php?act=findpost&pid=68862\"][{POST_SNAPBACK}][/a]

I really don't have any problem looking at that histogram.  If it had an overlay that marked 1/3 stops and whole stops, it would be useable by anyone
The only issue I see with the linear histogram is that there are so many values represented by the right half, that minority highlights might be indistinguishable from the bottom line.  That can be addressed by scaling, for "volume-oriented" assessment.  Or, you can use a histogram that is logarithmic or gamma-adjusted, but still RAW.

I get the impression from some responses that I am begging for a linear RAW histogram.  That has never been the jist of my argument.  I said right from the start that linear, logarithmic (which Jonathan showed an example of), or gamma-adjusted would all be more useful than what we have now.

I do not think that RAW histograms have to be linear in the x axis.

If a histogram is to be logarithmic for the x axis, I would prefer a red, green, and blue line on a dark background, with added colors where they cross.  Jonathan's 1/3 stop bars seem unnecessarily coarse to me.  1/3 stop markers I would like, but data should be finer, IMO.
Title: Shooting to the right
Post by: bjanes on June 22, 2006, 07:00:41 pm
Quote
I really don't have any problem looking at that histogram.  If it had an overlay that marked 1/3 stops and whole stops, it would be useable by anyone
The only issue I see with the linear histogram is that there are so many values represented by the right half, that minority highlights might be indistinguishable from the bottom line.  That can be addressed by scaling, for "volume-oriented" assessment.  Or, you can use a histogram that is logarithmic or gamma-adjusted, but still RAW.

I get the impression from some responses that I am begging for a linear RAW histogram.  That has never been the jist of my argument.  I said right from the start that linear, logarithmic (which Jonathan showed an example of), or gamma-adjusted would all be more useful than what we have now.

I do not think that RAW histograms have to be linear in the x axis.

If a histogram is to be logarithmic for the x axis, I would prefer a red, green, and blue line on a dark background, with added colors where they cross.  Jonathan's 1/3 stop bars seem unnecessarily coarse to me.  1/3 stop markers I would like, but data should be finer, IMO.
[a href=\"index.php?act=findpost&pid=68892\"][{POST_SNAPBACK}][/a]

John,

I just put up what I had--the PSCS2 histogram of a linear raw capture. I'm entirely receptive to some adaption of a raw histogram and think that you and Johnathin have made some very good suggestions and I am in no way trying to shoot them down.

I have a gamma 2.2 curve that I constructed by reverse engineering of some stouffer step wedge conversions in DCRaw and ACR, which appears reasonably accurate. I applied it to the previous raw image and think that the expansion of the highlight tones makes the histogram quite a bit easier to read. I would prefer to have the x-axis scaled in f-stops as Jonathin suggests, and this would be easy to implement.

The image still appears dark and the histograms indicate that I could have given at least 0.5 EV more exposure without blowing the green channel, and a magenta filter would allow even more exposure to the right. The experiment and the discussions in this thead demonstrate why the standard gamma corrected white balanced camera histogram is not up to the task of ETTR.

While it might only be a matter of sematics, the concept of a gamma 2.2 "raw" image sounds a bit like an oxymoron to me.

Raw conversion with 2.2 gamma curve:
[attachment=730:attachment]

Straight raw conversion:
[attachment=731:attachment]
Title: Shooting to the right
Post by: bjanes on June 22, 2006, 07:17:25 pm
Quote
Which is why I proposed a bar chart format with each bar representing 1/3 stop of exposure range, instead of a linear or standard gamma-adjusted display. You are comparing apples to field mice.

A histogram is nothing more than a chart displaying a distribution of values over a given range. One can be made from any data set, whether audio, video, still image, uninterpolated RAW data values, ASCII text, or stock market quotes. There's absolutely no reason whatsoever RAW data values "must" be interpolated or otherwise processed into a visually pleasing image before a histogram can be made. The whole problem we're trying to solve here is the errors introduced into the exposure evaluation process when white balance and color interpolation is performed on the data prior to running the histogram generation routine. Making a histogram from the unprocessed RAW data (especially when the histogram is segmented to exactly match the camera's exposure adjustment interval) will make judging correct exposure from the histogram much easier, not harder.
[a href=\"index.php?act=findpost&pid=68865\"][{POST_SNAPBACK}][/a]

Jonathan,

See my previous post to John Sheehy. I was just comparing what I had available and don't really what you meant by the metaphor.

You do not need to process the raw image into an image in order to make the histogram, but you do need to know about the image to interpret the histogram. Is the image low key or high key and is that why the histogram looks the way it does? Are those blown highlights expendable specular reflections and worth sacrificing for better shadow detail?

Thanks for the lession histograms 101.  

Bill
Title: Shooting to the right
Post by: John Sheehy on June 23, 2006, 01:31:38 am
Quote
I have a gamma 2.2 curve that I constructed by reverse engineering of some stouffer step wedge conversions in DCRaw and ACR, which appears reasonably accurate. I applied it to the previous raw image and think that the expansion of the highlight tones makes the histogram quite a bit easier to read. I would prefer to have the x-axis scaled in f-stops as Jonathin suggests, and this would be easy to implement.

Any RAW histogram should have subdivided f-stops indicated on it (at least for the top 3 or 4 stops).  That's a given.  The vertical marker lines would have different spacings for linear, gamma, and logarithmic (f-stop) displays.

Quote
While it might only be a matter of sematics, the concept of a gamma 2.2 "raw" image sounds a bit like an oxymoron to me.
[a href=\"index.php?act=findpost&pid=68925\"][{POST_SNAPBACK}][/a]

When does changing the axis of a chart between linear, logarithmic, or gamma-adjusted ever change the nature of the data?
Title: Shooting to the right
Post by: bjanes on June 23, 2006, 09:56:23 am
Quote
Any RAW histogram should have subdivided f-stops indicated on it (at least for the top 3 or 4 stops).  That's a given.  The vertical marker lines would have different spacings for linear, gamma, and logarithmic (f-stop) displays.
[a href=\"index.php?act=findpost&pid=68947\"][{POST_SNAPBACK}][/a]

The spacing for linear and gamma encoding can be deomonstrated by the Photoshop histograms of a Stouffer step wedge. The patches vary by 0.1 in density units, or 1/3 stop per patch. In the linear encoding, the first 3 patches (1 f/stop) occupy the right half of the histogram, the next stop the next fourth of the histogram, etc.

In the gamma encoded data, the distribution of the f/stops is more uniform, but there is still compression on the left. As you pointed out previously, this is a power function.

If one could plot the log base 2 of the exposure, the desired f/stop display would result, but I don't know of a way to do this in Photoshop.

[attachment=733:attachment]

Quote
When does changing the axis of a chart between linear, logarithmic, or gamma-adjusted ever change the nature of the data?
[a href=\"index.php?act=findpost&pid=68947\"][{POST_SNAPBACK}][/a]

The underlying data are not changed. However, if you apply a log scale to the graph, this is the same as applying a log transformation of the data and plotting on a linear scale--one has effectively applied a transformation with the use of the log scale.

By your reasoning, a linear raw image and a gamma transformed image are the same, since the underlying raw data have not changed. However, they have a quite different appearance. Again, sematics and I would not press the point.
Title: Shooting to the right
Post by: John Sheehy on June 23, 2006, 02:43:04 pm
Quote
If one could plot the log base 2 of the exposure, the desired f/stop display would result, but I don't know of a way to do this in Photoshop.

You need something like Mathcad to do this.  You can make charts with an algebraic term as the that includes the array as the data source, with a full range of mathematical operators to work with.

IRIS has a logarithmic display function.

Quote
By your reasoning, a linear raw image and a gamma transformed image are the same, since the underlying raw data have not changed. However, they have a quite different appearance. Again, sematics and I would not press the point.
[a href=\"index.php?act=findpost&pid=68971\"][{POST_SNAPBACK}][/a]

I don't think so.  A display and a histogram serve two completely different functions; a histogram is for analysis, and the scaling that best suites the analysis is the one you want to use.
Title: Shooting to the right
Post by: Jonathan Wienke on June 23, 2006, 06:45:00 pm
Quote
Jonathan,

See my previous post to John Sheehy. I was just comparing what I had available and don't really what you meant by the metaphor.

You do not need to process the raw image into an image in order to make the histogram, but you do need to know about the image to interpret the histogram. Is the image low key or high key and is that why the histogram looks the way it does? Are those blown highlights expendable specular reflections and worth sacrificing for better shadow detail?

Thanks for the lession histograms 101.  

Bill

OK, I misunderstood the point you were trying to make. The second paragraph of my post was directed at Anon E Mouse's objections to my RAW histogram proposal. He seemed to be laboring under the mistaken impression that an image had to be converted to a visually pleasing RGB format before a useful histogram could be made from it. That is obviously not the case, and I was attempting to explain why.
Title: Shooting to the right
Post by: Jonathan Wienke on June 23, 2006, 06:52:21 pm
Quote
Jonathan's 1/3 stop bars seem unnecessarily coarse to me.  1/3 stop markers I would like, but data should be finer, IMO.

I don't see much value in granulating the histogram any finer than the camera's minimum exposure adjustment. If the camera can adjust exposure in 1/10-stop increments, then a histogram that displays the data broken down in 1/10-stop segments makes sense. But if the camera only adjusts in 1/3-stop increments, anything finer than that is kind of a waste.
Title: Shooting to the right
Post by: John Sheehy on June 24, 2006, 04:09:55 pm
Quote
I don't see much value in granulating the histogram any finer than the camera's minimum exposure adjustment. If the camera can adjust exposure in 1/10-stop increments, then a histogram that displays the data broken down in 1/10-stop segments makes sense. But if the camera only adjusts in 1/3-stop increments, anything finer than that is kind of a waste.
[a href=\"index.php?act=findpost&pid=69016\"][{POST_SNAPBACK}][/a]

I don't see how more information, which takes up the same amount of room, is a waste.

Say you realize that your favorite RAW converter doesn't utilize the top 1/6 stop, and you want to draw the line somewhere else; way you want to see the finer histogram to try to figure out exactly what is close to clipping; say you want to know if that top 1/3 stop is populated more towards the top or bottom; in all these cases, 1/3 stop is too coarse.  I don't see the need to jump from our current histogram, with 200+ levels, to one with only 32.
Title: Shooting to the right
Post by: bjanes on June 26, 2006, 10:01:04 am
Quote
I don't see how more information, which takes up the same amount of room, is a waste.

Say you realize that your favorite RAW converter doesn't utilize the top 1/6 stop, and you want to draw the line somewhere else; way you want to see the finer histogram to try to figure out exactly what is close to clipping; say you want to know if that top 1/3 stop is populated more towards the top or bottom; in all these cases, 1/3 stop is too coarse.  I don't see the need to jump from our current histogram, with 200+ levels, to one with only 32.
[a href=\"index.php?act=findpost&pid=69066\"][{POST_SNAPBACK}][/a]

If you want Jonathin's histogram to include color information as per his recent demonstration, then hundreds of levels would result in bars that are so narrow that they couldn't be seen on the camera display. If you want smaller bins, then the color histograms would have to be shown separately as they are on current cameras.

Personally, I think 0.3 EV resolution is sufficient. It is true that the manual settings can only be set in 0.3 EV increments, but if one is (horror of horrors) in an automatic mode (say aperture perferred) the shutter can be ajdusted in finer increments. Finer resolotion might be useful in a studio situation where one is shooting slowly with precision, but would have little use in field use.
Title: Shooting to the right
Post by: John Sheehy on June 29, 2006, 08:49:45 am
Quote
If you want Jonathin's histogram to include color information as per his recent demonstration, then hundreds of levels would result in bars that are so narrow that they couldn't be seen on the camera display. If you want smaller bins, then the color histograms would have to be shown separately as they are on current cameras.

That's exactly what I want; 3 graphs for RGB (my preference) or one graph for greyscale RAW.  Bars would only be practical for greyscale; the colored bars as Jonathan posted them are too cluttered; single-line graphs super-impose more clearly.

Quote
Personally, I think 0.3 EV resolution is sufficient.

Aren't many things already inferior because of what people thought would be sufficient?

Why repeat history?

Quote
It is true that the manual settings can only be set in 0.3 EV increments, but if one is (horror of horrors) in an automatic mode (say aperture perferred) the shutter can be ajdusted in finer increments. Finer resolotion might be useful in a studio situation where one is shooting slowly with precision, but would have little use in field use.
[a href=\"index.php?act=findpost&pid=69159\"][{POST_SNAPBACK}][/a]

Not everything "in the field" is a moving target.

I want to see the fine humps in my histogram.  Regressing to big fat bars serves no useful purposes, and reduces information, and aliases the sampling.