Camera's histogram reliable to the RAW data

[Guillermo Luijk]

Hi, my first post here.  I have  been watching with great interest some of the technical discussions you guys have on this site.  A few points:

I think this unity wb is a great idea, as the inaccuracy of the in-camera jpg at indicating correct clipping is one of the difficulties of ETTR.  One less thing the deriders of ettr have to complain about.

But one question I have is what exactly is the highlight warning on the lcd showing?  Is it showing clipping of any one of the three channels or is it some sort of composite?  I understand that the histogram on the lcd is usually some sort of green channel weighted composite.  Is the clipping warning working on this?  If so, is it possible to be clipping a channel (particularly red perhaps?) and it not showing blinkies on the lcd?  I guess the reason I came to this conclusion is that after implementing Guillermo's method I am still getting very similar flashies on the lcd preview between the unity wb and a scene wb.  However when developing a linear non-wb tiff I see that I still have maybe a stop of so of headroom until I get clipping.  I am assuming my custom white balance image is correct (although it looks more pink than magenta (i'm on a 5D)) as it is giving multipliers of about 1.01 for each channel in dcraw.

I pointed that question in the thread, and the conclusion is that this will depend on the camera implementation. The method to cancel de Wb is flawless (the 1.0 multipliers confirm this); how it is interpreted in the camera's display will depend on your particular camera. And finding out the relation between the real RAW and the blinking camera highlights should be achieved by doing comparisions.
From the few pics I tested in my 350D, I pre-conclude that its blinking lights show any partial saturation (i.e. any channel blown makes the display blink) which is a good new. But any camera can be different.

1.01 is really very precise, you did it fine. But "after implementing Guillermo's method I am still getting very similar flashies on the lcd preview between the unity wb and a scene wb.  However when developing a linear non-wb tiff I see that I still have maybe a stop of so of headroom until I get clipping" I am a bit surprised at this (and disappointed since it can be my next camera). Can you do some more checking? or diplaying a couple of rear shots of your 5D over the same scene with and without UniWB?

Could you post the RAW file somewhere so I can check? BTW would you mind I publish it in my website? I already have 350D, Panopeeper's 40D, D300 coming and your 5D.

Regards.

[Westy]

Hi Guillermo,  I have just done a bit of checking of my results and in fact all that i have rechecked the uniWB is functioning fine and matches the linear raw quite well.  The uniWB jpgs show just slightly more clipping than the raws.  I will find the one I did where I thought I had a stop or so of headroom and recheck it as well (I can't actually remember which shot it was).  And I've just now realised that I have had my jpeg settings non-neutral.  After I set more neutral rendering parameters I suspect the uni wb jpg will match even closer the linear raw.

I've also rechecked my wb multipliers and they are 1.01,1.02,1.  You can have my custom raw, but at the moment I am limited to dial up as my asdl got zapped in a storm the other day, so it might take me a few days to sort that out.

On the subject of the 5D, I think you can't go wrong with it.  Whether the 40D has better or worse high iso noise performance is debatable, but the difference is likely to be so small as to not concern yourself.  Both will give EXCEPTIONAL high iso noise performance.  Other issues to consider for low light interiors are bigger and brighter view finder of the 5D

[Guillermo Luijk]

You could try some strong RED subject (a rose, coke can,...), and compare camera displays with Daylight WB JPEG, UniWB WB JPEG and RAW (dcraw -r 1 1 1 1).

OK, thanks for the RAW file (when available).

[Jonathan Wienke]

I've been playing around with the UniWB, and have run across some rather surprising things. With my 1D-MkII, when shooting normal "natural" objects that would probably fall within sRGB, the RGB histogram is now pretty good at predicting single-channel RAW clipping. But the histogram can still be quite wrong in some cases. I conducted a test where I photographed some red, green, and blue LED glow lights to see how accurate channel clipping might be for highly saturated colors, and was unable to get the color channel being tested to fill more than 1/4 of the rightmost histogram segment without gross overexposure (>2 stops) where the channel clipping was obvious in the image even on the camera LCD. This behavior was almost identical with both color matrix 1 (Standard sRGB) and 4 (Adobe RGB). So while the UniWB has significantly increased the conditions in which the RGB histogram accurately reflects the RAW data, there are still a few exceptions. I'm going to do more testing and then post detailed results.

[Guillermo Luijk]

Jon, so in your camera histogram and blinking highlights do not match? it seems blinking highlights are more accurate than histogram then.
Since my histogram is monochrome and that means too many blinding variables, I cannot do tests regarding this issue.

BTW would you mind to offer the UniWB for the 1D-MKII so that we can create a list of UniWB RAW files? I could settle it at the beginning of the thread.

Regards.

[John Sheehy]

I've been playing around with the UniWB, and have run across some rather surprising things. With my 1D-MkII, when shooting normal "natural" objects that would probably fall within sRGB, the RGB histogram is now pretty good at predicting single-channel RAW clipping. But the histogram can still be quite wrong in some cases. I conducted a test where I photographed some red, green, and blue LED glow lights to see how accurate channel clipping might be for highly saturated colors, and was unable to get the color channel being tested to fill more than 1/4 of the rightmost histogram segment without gross overexposure (>2 stops) where the channel clipping was obvious in the image even on the camera LCD. This behavior was almost identical with both color matrix 1 (Standard sRGB) and 4 (Adobe RGB). So while the UniWB has significantly increased the conditions in which the RGB histogram accurately reflects the RAW data, there are still a few exceptions. I'm going to do more testing and then post detailed results.
[a href=\"index.php?act=findpost&pid=166694\"][{POST_SNAPBACK}][/a]

This is exactly what I have been talking about, except with red, orange and yellow  flowers as examples.

You will never get a true RAW histogram out of any RGB conversion.  The colors are altered selectively, the saturations are altered selectively, etc, etc, regardless of the JPEG settings.  It will only reflect the RAW proportionately across the channels for grayscale subjects, and maybe close for weakly saturated ones.

[Jonathan Wienke]

You will never get a true RAW histogram out of any RGB conversion.  The colors are altered selectively, the saturations are altered selectively, etc, etc, regardless of the JPEG settings.  It will only reflect the RAW proportionately across the channels for grayscale subjects, and maybe close for weakly saturated ones.

That assesment is a bit pessimistic; UniWB not perfect, but certainly is a step in the right direction. Even with the caveats about super-saturated colors, the histogram is much closer to RAW than it is when using any other white balance.

[Guillermo Luijk]

You will never get a true RAW histogram out of any RGB conversion.  The colors are altered selectively, the saturations are altered selectively, etc, etc, regardless of the JPEG settings.

John, this question could be considered as an offtopic, but it's a long time I wonder this: if we develop a RAW file into some colour space like sRGB or AdobRGB but linearly, i.e. we still don't apply the corresponding gamma (DCRAW does this for instance), is adjusting the exposure as simple as multiplying each channel by the same factor as when we don't convert to any color space? and would this operation alter hue/saturation?

That assesment is a bit pessimistic; UniWB not perfect, but certainly is a step in the right direction. Even with the caveats about super-saturated colors, the histogram is much closer to RAW than it is when using any other white balance.

I agree. The UniWB is, at least, a good improvement.

[John Sheehy]

That assesment is a bit pessimistic; UniWB not perfect, but certainly is a step in the right direction. Even with the caveats about super-saturated colors, the histogram is much closer to RAW than it is when using any other white balance.
[a href=\"index.php?act=findpost&pid=166708\"][{POST_SNAPBACK}][/a]

Yes.  I was shooting flat magenta images a few years back, and started using it for custom WB, but what I found was that it was most accurate when the highlights were white, and guess what?  The white highlights are just as accurate with or without uniWB.  Theoretically, you might expect better ETTR of a red or blue highlight with uniWB, now that cameras have RGB histograms, but this is exactly where the color conversion messes things up.  It isn't worth the hassle to me, to look at a cyan-green preview and JPEGs, for gains which were hard to realize.

What I really want is a camera that shows the WB'ed sRGB image in the preview (at least as an option, against a linear RAW), but flashes (red, green, blue, cyan, magenta, yellow or white alternating with black) clipped highlights based on the RAW, and shows a pure RAW histogram.

[John Sheehy]

John, this question could be considered as an offtopic, but it's a long time I wonder this: if we develop a RAW file into some colour space like sRGB or AdobRGB but linearly, i.e. we still don't apply the corresponding gamma (DCRAW does this for instance), is adjusting the exposure as simple as multiplying each channel by the same factor as when we don't convert to any color space? and would this operation alter hue/saturation?
[a href=\"index.php?act=findpost&pid=166716\"][{POST_SNAPBACK}][/a]

Hue and saturation are just ways of looking at R:G:B ratios.  If the data is linear, then simply scaling them (with blackpoint at zero, of course) does not change hue or saturation, except as an unscaled curve might be applied to them at conversion.

[Panopeeper]

I conducted a test where I photographed some red, green, and blue LED glow lights to see how accurate channel clipping might be for highly saturated colors, and was unable to get the color channel being tested to fill more than 1/4 of the rightmost histogram segment without gross overexposure (>2 stops) where the channel clipping was obvious in the image even on the camera LCD

I think you stretched the concept over the limit. We can practicall turn off white balancing, we can turn off contrast and saturation, but we can't turn off the de-mosaicing.

The LED light is practically single wavelength; as such, it passes one (or two) filters to a high degree, while the other (or others) let only a relative small portion through, although it appears measurably in all three colors, see spectral response. Therefor you can clip a channel with a fraction of the values in the other channels. However, the de-mosaicing will "distribute" the pixel values between the neighbours.

You mentioned "highly saturated". Led and laser lights are highly saturated, for sure - but highly saturated what? There are many different color within for example the range of "red" in the spectrum, but there is only one saturated red in the color space.

Consequently the highly saturated color you used may appear as a composite color from the point of de-mosaicing.

[Jonathan Wienke]

What I really want is a camera that shows the WB'ed sRGB image in the preview (at least as an option, against a linear RAW), but flashes (red, green, blue, cyan, magenta, yellow or white alternating with black) clipped highlights based on the RAW, and shows a pure RAW histogram.

I agree that this would be far preferable to what we have now. But for colors that fall within sRGB, UniWB delivers a closer correlation between camera histogram and RAW than any other alternative.

[Jonathan Wienke]

I think you stretched the concept over the limit. We can practicall turn off white balancing, we can turn off contrast and saturation, but we can't turn off the de-mosaicing.

The LED light is practically single wavelength; as such, it passes one (or two) filters to a high degree, while the other (or others) let only a relative small portion through, although it appears measurably in all three colors, see spectral response.

I agree this is an extreme case; that's why I tried it--to see how well UniWB worked in extreme circumstances. I don't think that this invalidates the usefulness of UniWB in most circumstances, though.

[John Sheehy]

I think you stretched the concept over the limit. We can practicall turn off white balancing, we can turn off contrast and saturation, but we can't turn off the de-mosaicing.
[a href=\"index.php?act=findpost&pid=166745\"][{POST_SNAPBACK}][/a]

I don't see how demosaicing would be relevant.  You would get the same relatively saturated capture if R, G, and B were co-located.

[Panopeeper]

I have to disagree with you: DOF DOES depend on the cropping, i.e. on the sensor format

I stated, that DoF depends on the lens, but not on the cropping; this is a fact. It is not useful to mix up concepts.

One can bring the sensel size in the equation as well; that again has nothing to do with cropping. I had similar discussion with paper tigers, whose measure of image quality is how large an image can be printed. They will debate endlessly over side-issues, like sensel size, number of pixels, etc.

One needs to see the issues more abstract, otherwise we can not conduct a factual discussion. Another such fruitless discussion was, that some people stated that the perspective depends on focal length and cropping. Plain BS. You can state, that with a different focal length you have to go to a different distance. Right, that changes the perspective, not the focal length.

Re panos in architectural photographing: it is unquestionable, that you can achieve a very good result with a wide angle lens or tilt and shift, but do you have such wide lens?

http://www.panopeeper.com/panorama/GameRoom1.jpg

Panos is one of my pending tasks for some day. What software do you recommend me? PT?

There are many stitchers for casual panomakers. For those, who are serious about it, there is only one: PT (and its descendants, working on the very same principle). However, PT is not for human consumption, you have to have a good user interface. I prefer Panorama Tools Assembler, others prefer PTGui, and there is Hugin, but not on all platforms.

[Panopeeper]

I don't see how demosaicing would be relevant.  You would get the same relatively saturated capture if R, G, and B were co-located.

You certainly would not get the same, because the colors of a single tri-color pixel would have to undergo "only" a color space transformation, while the de-mosaicing has to work with several neighbouring pixels, where the distance between them is relevant too (and the result undergoes the color space conversion).

Nevertheless, there is no saturated color on raw level. You can not find any wavelength, which would be totally filtered by two channels. Theoretically, one wavelength per filter would be acceptable, but this irrelevant, and anyway there are no such filters.

So, you can have many different color compositions from different saturated colors (single wavelengths),  which have to be transformed in different RGBs; only one of them may be transformed in a fully saturated color.

[bjanes]

You certainly would not get the same, because the colors of a single tri-color pixel would have to undergo "only" a color space transformation, while the de-mosaicing has to work with several neighbouring pixels, where the distance between them is relevant too (and the result undergoes the color space conversion).

Nevertheless, there is no saturated color on raw level. You can not find any wavelength, which would be totally filtered by two channels. Theoretically, one wavelength per filter would be acceptable, but this irrelevant, and anyway there are no such filters.

So, you can have many different color compositions from different saturated colors (single wavelengths),  which have to be transformed in different RGBs; only one of them may be transformed in a fully saturated color.
[{POST_SNAPBACK}][/a]

You guys are getting way off base here when you talk about red lasers, LEDs, and the highly saturated or monochromatic colors produced by these devices. It is not important to capture such colors since they do not occur in nature (or only very rarely) and can not be reproduced on any current display or printed. What is important for photography are the real world surface colors, which are discussed [a href=\"http://www.colour.org/tc8-05/MetricsUpdateNov01.pdf]here.[/url]

[Jonathan Wienke]

You guys are getting way off base here when you talk about red lasers, LEDs, and the highly saturated or monochromatic colors produced by these devices. It is not important to capture such colors since they do not occur in nature (or only very rarely) and can not be reproduced on any current display or printed.

Have you ever shot at a dance club, or concert? Or a car show? Highly saturated colors outside Adobe RGB aren't that uncommon. Not natural, perhaps, but that doesn't mean you'll never find them in a photograph. Knowing what to do when you do is of more than merely academic interest.

[bjanes]

Have you ever shot at a dance club, or concert? Or a car show? Highly saturated colors outside Adobe RGB aren't that uncommon. Not natural, perhaps, but that doesn't mean you'll never find them in a photograph. Knowing what to do when you do is of more than merely academic interest.
[a href=\"index.php?act=findpost&pid=166791\"][{POST_SNAPBACK}][/a]

This has been discussed many times. If you want to capture all real world surface colors, you should render into ProPhotoRBG. Also, saturation clipping is easily seen in the ACR histogram, and alerts one to use a wider color space.

Here is a plot from Gernot Hoffmann's web site showing the real world surface colors (dotted lines) along with sRGB, aRGB, and ProPhotoRGB gamuts.

[Panopeeper]

I find it generally useful to perform abstract, in practice useless tests - in thought or in real - in order to understand the behaviour and the underlying principles of some phenomenon.

One could say it is useless to shoot brick walls and newspapers instead of looking at the nice picture of the favourite cat (usually a crop in 25%), as proof for the high quality of a lens. I am on the other side. I am shooting abstact and/or uninteresting subjects and peeping the non-demosaiced images, when I judge a lense (the test of bokeh is an exemption).