Universal white balance and ETTR

[torger]

Those that shoot still life often practice ETTR, that is expose the sensor with as much light as possible without clipping highlight detail.

Having practiced this for a while I have been troubled by that it is so easy to mistakingly overexpose that a better practice may actually be to play it safe -- rather a bit noisier shadows than completely lost highlights. Recent articles often state that ETTR practice is flawed and not applicable on modern cameras which has better dynamic range than the early digital cameras. My approach has been to make a safe shot first, and then bracket a few brighter shots and decide in post which exposure to use.

The general problem is that Canon/Nikon (and most other) camera manufacturers do not provide RAW histograms in camera, the histograms is on the camera-generated JPEG even if you shoot RAW, and is affected by white balance setting etc so it may differ a lot from how the actual sensor's RGB channels are exposed. Making ETTR decisions from the camera histograms can lead to over-exposure or under-exposure depending on lighting conditions, certainly not very precise or safe.

So I decided to test the "universal white balance" trick on a Canon camera. It does seem to work, that is ETTR decisions can be safely made down to 1/3 stop from RGB histograms in camera (since clipping levels match actual RAW data), even the live view RGB histogram works well (note you must use RGB histograms, luminance histograms are useless for this purpose). You can even safely make such bold exposure decisions like clipping one channel to reconstruct it in postprocessing which may be ok if the highlight is white (a bright cloud for example). The drawback with uniwb is that embedded JPEGs look green due to this white balance setting, which you may ignore if you want to since you will be using the RAW data anyway, but since this type of exposure optimization is only used in still life photography I usually have the time to change to a natural white balance before taking the final shot. I think it is nice to have a natural-looking embedded camera JPEG as reference.

I also noted that "ETTR" if practiced not to clip highlights (as it should) may quite often actually expose more to the left than standard metering, but of course that depends on how the camera's auto exposure is designed. That ETTR may actually lead to more conservative exposures (more to the left) on modern cameras can make it a bit confusing concept.

Anyway, after doing these initial uniwb tests it seems to me that if you are going to practice ETTR and your camera cannot show RAW RGB histograms you really should consider using uniwb.

Is anyone else using universal white balance to make manual exposure decisions? It would be nice to hear some experiences from those that have used it for a while, if it indeed is safe or if you still come home with over-exposed / under-exposed pictures.

For those new to uniwb, here's a link (for Canon cameras) http://www.guillermoluijk.com/tutorial/uniwb/index_en.htm

[bjanes]

The general problem is that Canon/Nikon (and most other) camera manufacturers do not provide RAW histograms in camera, the histograms is on the camera-generated JPEG even if you shoot RAW, and is affected by white balance setting etc so it may differ a lot from how the actual sensor's RGB channels are exposed. Making ETTR decisions from the camera histograms can lead to over-exposure or under-exposure depending on lighting conditions, certainly not very precise or safe.

In my experience with my Nikon cameras, ETTR according to the camera RGB histogram never results in clipped highlights in any channel. Most cameras underexpose to allow highlight headroom and adjust the tone curve so that the histogram and preview image appear normally exposed rather than dark. Most Nikons seem to allow 0.5 EV of headroom for highlights. The Phase One IQ 180 allows nearly two stops of headroom. This headroom results in conservative camera histograms.

With daylight exposure, the green of most cameras is most sensitive and blows first. With D50 illumination the sensitivities of the Nikon D3 are shown below in relation to sRGB primaries (data from DXO).



To equalize the channel sensitivities for a given illumination, the raw channels are multiplied by a white balance coefficient. For the D3 and D50 (near daylight illumination) the DXO testing shows that WB multipliers are 2.24, 1.0, and 1.2 respectively for red, green, and blue. If the image contains saturated red or blue, WB multipliers greater than 1.0 can cause clipping in these channels with white balance when none is present in the raw file. However, the reverse does not occur.

UniWB merely sets the WB multipliers all to unity (1.0) and is useful when one is photographing subjects with saturated reds or blues because it eliminates clipping induced by white balance. Saturation clipping can still occur if the colors are out of the sRGB or aRGB of the working spaces of Nikon cameras. Again, the RGB histograms will be conservative and the raw channels will not be clipped in the rendered image for the camera preview are not clipped.

Anyway, after doing these initial uniwb tests it seems to me that if you are going to practice ETTR and your camera cannot show RAW RGB histograms you really should consider using uniwb.

Is anyone else using universal white balance to make manual exposure decisions? It would be nice to hear some experiences from those that have used it for a while, if it indeed is safe or if you still come home with over-exposed / under-exposed pictures.

UniWB will often allow more exposure to the right for the red and blue channels, but using normal WB will result in conservative histograms and there will be no raw channel clipping. With modern low noise sensors, a degree of underexposure can often be tolerated. For most of my work, I find UniWB to be more trouble than it is worth. The main problem with ETTR is to correlate channel clipping in the camera histogram with the raw histogram shown by programs such as Rawnalize or DCRaw that look directly at the raw file. You need to know how much headroom your camera allows.

Regards,

Bill

[torger]

I tested this in indoor lighting with a Canon 7D and then I got about 1/3 stop clipping in the raw file of the green channel without any clip in the normal white balance RGB histogram (it showed that the red channel was about to clip, which it was not, and the green channel ok which was actually clipped), while uniwb presented the correct result. So this camera does not seem to be conservative as the Nikon D3. There are JPEG picture style settings to adjust though (I have default settings) that maybe will affect the result.

Current canon cameras does not exactly excel in dynamic range, so being able to extract the most out of it in a safe way is quite attractive to me. But I'll see, I'll try this practice for a while and see how it works. Since I've bracketed and test-shot a lot previously to tune an exposure, flipping white-balances actually seems like a faster way to work for me.

I've though about the color space thing, if the camera really clips the jpeg if the color does not fit or if it makes a perceptual mapping, so the histograms will still not show clipping. Something to test.

[bjanes]

I tested this in indoor lighting with a Canon 7D and then I got about 1/3 stop clipping in the raw file of the green channel without any clip in the normal white balance RGB histogram (it showed that the red channel was about to clip, which it was not, and the green channel ok which was actually clipped), while uniwb presented the correct result. So this camera does not seem to be conservative as the Nikon D3. There are JPEG picture style settings to adjust though (I have default settings) that maybe will affect the result.

How are you determining clipping in the raw channels? Are you using Rawnalize, DCRaw, or another program that looks directly at the channels? And how are you setting UniWB? Since UniWB should have a green multiplier of 1.0, I don't see how the regular and UniWB histograms for the green channel should be different, unless something else has changed. For tungsten lighting (approximately Illuminant A), the light contains more red and the red channel sensitivity will be closer to the green channel as shown in the shown DXO measurements. Still the green channel is most sensitive and should clip before the red unless the scene contains a lot of saturated red. Does your test target include saturated reds or does it have neutrals?

Regards,

Bill

[feppe]

Those that shoot still life often practice ETTR, that is expose the sensor with as much light as possible without clipping highlight detail.

Having practiced this for a while I have been troubled by that it is so easy to mistakingly overexpose that a better practice may actually be to play it safe -- rather a bit noisier shadows than completely lost highlights.

This is exactly the approach Ctein takes on TOP, and is diametrically opposed to Michael's recent article on the subject where he still appears to advocate ETTR despite its shortcomings with current camera histograms.

On topic, I found out that Olympus cameras have an ETTR setting, but it has its shortcomings - mainly that it's a spot metering mode, and Guillermo Luijk's research down the thread shows it leaves too much headroom. I have not used it since, and prefer to take Ctein's approach.

[torger]

How are you determining clipping in the raw channels? Are you using Rawnalize, DCRaw, or another program that looks directly at the channels? And how are you setting UniWB? Since UniWB should have a green multiplier of 1.0, I don't see how the regular and UniWB histograms for the green channel should be different, unless something else has changed. For tungsten lighting (approximately Illuminant A), the light contains more red and the red channel sensitivity will be closer to the green channel as shown in the shown DXO measurements. Still the green channel is most sensitive and should clip before the red unless the scene contains a lot of saturated red. Does your test target include saturated reds or does it have neutrals?

I am using rawtherapee which can display the raw histogram (a bit crude though). I'm setting uniwb using an overexposed cr2 (downloaded from the link above), a trick which works on some cameras (not all) but 7D is one of them. I'm just shooting onto some detail in my kitchen so it is not a special test target, quite neutral colors with lots of white, no saturated colors. With uniwb green channel is clearly above red, changing to tungsten wb red channel passes green in the histogram and if you go to a higher temperature wb red passes even more, despite no saturated reds, it is the color of the white furniture and walls caused by the white balance that makes this happen. If the white balance is such that the bright white furniture in this case appears slightly red then red will clip earlier. Or say if my white balance was perfect and the furniture would be slightly red it would be the same result. It is not only saturation that causes a channel to clip, brightness too. A saturated red is R 100 G 0 B 0, but a white with hint of red R 100 G 90 B 90 is not saturated but can cause clipping anyway.

However, there are many aspects here that I need to look into, I have looked at this for only 45 minutes so far... that slight overexposure I got could be a mistake, it does seem that the largest gain is to be able to ETTR more under certain lighting conditions.

[torger]

This is exactly the approach Ctein takes on TOP, and is diametrically opposed to Michael's recent article on the subject where he still appears to advocate ETTR despite its shortcomings with current camera histograms.

On topic, I found out that Olympus cameras have an ETTR setting, but it has its shortcomings - mainly that it's a spot metering mode, and Guillermo Luijk's research down the thread shows it leaves too much headroom. I have not used it since, and prefer to take Ctein's approach.

Ctein's approach simply seems to be "pull an guesstimated large amount too the left just to make sure you don't clip since you can't trust camera display or histograms" and I don't really like that   , that would be wasting 1 - 2 stops in every picture, and as a Canon user I cannot afford that .

If this white balance trick works to show clipping safely (which it does seem, although it might be a little bit conservative due to color space reduction) then I have the tool I need. Bjanes suggest that it may provide only mariginal advantage over standard RGB histograms since they are conservative, I shall look more into that, but if so then RGB histograms with any white balance will work alright. Extremely small highlights that don't really become visible in histograms and still should have detail are really a special case (just like the case when you need to clip a large part of the picture) and then there's the conservative guesstimate fallback, but only then. As I said in the first post, practicing ETTR with histograms really often leads to more conservative exposure than using auto-exposure, so it often becomes an expose to the left in practice.

[bjanes]

This is exactly the approach Ctein takes on TOP, and is diametrically opposed to Michael's recent article on the subject where he still appears to advocate ETTR despite its shortcomings with current camera histograms.

Ctein makes an interesting point, and I should revise my statement that highlights are never blown when the camera histogram shows no clipping--there may be clipped areas that are too small to show up in the camera histogram. To paraphrase Bruce Fraser, proper ETTR involves not blowing highlights you wish to preserve. Ctein's image in the article was not properly exposed to the right if he wanted to retain those highlights . Jeff Schewe's response is good, but in his image of the Niagra Falls, the degree of overexposure is not nearly so great as the ACR histogram would indicate. Examination of that raw file with Rawnalize demonstrates much less clipping.

Rather than systematically underexposing all one's images in order to prevent possibly blown highlights, the more prudent photographer would anticipate those situations where the camera histogram might not detect small important clipped highlights. For night scenes, bracketing gives the best chance of obtaining an optimal exposure.

Regards,

Bill

[feppe]

Rather than systematically underexposing all one's images in order to prevent possibly blown highlights, the more prudent photographer would anticipate those situations where the camera histogram might not detect small important clipped highlights. For night scenes, bracketing gives the best chance of obtaining an optimal exposure.

I do quite a bit of night photography with city lights in the frame, and bracketing is indeed the only solution. Sometimes 5 seconds is enough, sometimes 60 seconds is not. Also, digital blending (or HDR or exposure fusion if you're so inclined) is often necessary to contain the DR of the scene, in which cases ETTR is not very helpful.

I find that I can usually get along with just 2 exposures, one for highlights, one for overall scene, and blend in post. The problem is figuring out which 2 brackets are the correct ones while in the field is pretty difficult due to same issues with histogram (in)accuracy and lack of true RAW histogram.

[torger]

I've done some more testing with a color checker and a light source similar to daylight.

Observations: Bjanes was right, no matter which white balance the RGB histograms are conservative so it will not clip raw channels. The advantage of uniwb is thus that you can expose less conservative and still be sure that you don't overexpose. In daylight scenes I estimate the gain to be about 2/3 stops, that is not very large, but if you don't have much DR it may be valuable 2/3 stops. With more extreme lighting the gain can be 1 - 2 stops.

For saturated colors the camera's JPEG color space will keep the RGB histograms conservative regardless of uniwb or not, so AdobeRGB will make you a bit less conservative than sRGB, but for scenes with very saturated colors you may still underexpose with a stop or so.

Histograms need to be quite readable to be useful, so you'll need a newer camera with 3" screen to do this. Still you'll miss really small highlights, so as discussed above night scenes can be tricky.

[NikoJorj]

This is exactly the approach Ctein takes on TOP, and is diametrically opposed to Michael's recent article on the subject where he still appears to advocate ETTR despite its shortcomings with current camera histograms.

Why should these appear as opposite views? I'd thought that since advent of digital, everybody knows that clipping is to be avoided, and that it's taking for granted in Michael's article. The main difference is a tradeoff between noise and the risk of clipping.

On topic, I found out that Olympus cameras have an ETTR setting, but it has its shortcomings - mainly that it's a spot metering mode, and Guillermo Luijk's research down the thread shows it leaves too much headroom. I have not used it since, and prefer to take Ctein's approach.

I discovered it too (have an EPL1 since 2 weks), and even if it's not optimized for raw, I feel for now that it's the best guesstimate at hand when you have to shoot quickly.

Of course, it would be much more simple if manufacturers would give us the proper tools, but as long as pigs don't fly , is there a way to set a different exposition bias for spot metering (+3EV) and matrix metering (say 0EV) on Canon cameras that have spot metering?
Olympus provides a way to fine tune each metering method (on my EPL1 it's custom menu/J-Utility/Exposure shift), and it's quite helpful in the present case.

[torger]

Of course, it would be much more simple if manufacturers would give us the proper tools, but as long as pigs don't fly , is there a way to set a different exposition bias for spot metering (+3EV) and matrix metering (say 0EV) on Canon cameras that have spot metering?

I think I've heard the new 1D X have possibilities to tune auto exposure function with preset biases, but I don't think this 7D has. I also note that live view metering really sucks, it is supposed to be matrix metering but it is heavily affected where you have your zoom box. Normal matrix metering misses small highlights. So for still life, manual exposure guided by histograms is the only sane way to do it on this camera. Just hope that they eventually implement real histograms and real clipping indications based on RAW data in coming products, but they haven't done it yet and it is an obvious feature since many years so I don't expect it to happen soon.

[RFPhotography]

Ctein is missing a couple key points with his article.  BJanes brings up one of them.

Using ETTR, the exposure is pushed to the right such that no clipping occurs.  None.  Not 'unwanted clipping'.  Not 'non-specular highlight clipping'.  No clipping.

Also, for ETTR to be of any value the scene/subject contrast has to fit within the range of the sensor.  The scene CTein was using as an illustration fails that aspect of ETTR. 

[hjulenissen]

Ctein is missing a couple key points with his article.  BJanes brings up one of them.

Using ETTR, the exposure is pushed to the right such that no clipping occurs.  None.  Not 'unwanted clipping'.  Not 'non-specular highlight clipping'.  No clipping.

Also, for ETTR to be of any value the scene/subject contrast has to fit within the range of the sensor.  The scene CTein was using as an illustration fails that aspect of ETTR. 

That is not how I understand the popular consensus of how "ETTR" should be interpreted, and it does not seem like a sensible approach.

1. Figure out what parts of the true scene luminance is to be accurately captured
2. Use ETTR to make sure that the brightest _important_ parts of the scene are as close to the sensor saturation point in each primary, but not beyond it

Doing 1 and 2 may well give you clipped highlights, but only if you think that it is ok. It will give you the most "accurate" capture of the prioritized luminance range that your equipment is capable of.

-h

[RFPhotography]

Just because an interpretation is 'popular' doesn't make it right. 

[hjulenissen]

Just because an interpretation is 'popular' doesn't make it right. 

No, but if a definition is both popular and sensible, I think that both make my case stronger. Of course, you may choose to provide arguments that it is not sensible, or that some expert thinks otherwise.

-h

[RFPhotography]

Well clearly you don't consider me an expert.   

http://schewephoto.com/ETTR/index.html

Read and be educated.

[hjulenissen]

Well clearly you don't consider me an expert.  

On the internet, everyone is his/her own expert, so refering to oneself as an authority in discussions tends to be less than productive :-)

http://schewephoto.com/ETTR/index.html

Read and be educated.

I think that this sentence is more or less in line with what I said:
"if you have highlights whose texture and detail is important"

I will give you that he supports what you have been saying here:
" This is not intended to be used in every photographic situation, only those situations where the scene's contrast range is lower than you camera's sensor dynamic range."


So basically, someone agrees with you. I still think that it sounds counterproductive, and neither you nor Schewe (?) have provided arguments for why ETTR should _never_ be used together clipping highlights.

My argument is very simple. If a single point-source of light is dramatically more bright than the average brightness of the scene and covering e.g. exactly one pixel in my camera, I may not care that it is clipped. I may not care that it is accurately rendered in the final image. If it has a color cast, I can relatively easily fix it later on. What IS important to me is that the remaining 17.999 megapixels turn into a great image. I believe that the insight gained by ETTR can help me ensure that the interesting part of the image is slightly better.

-h

[Schewe]

So basically, someone agrees with you. I still think that it sounds counterproductive, and neither you nor Schewe (?) have provided arguments for why ETTR should _never_ be used together clipping highlights.

Huh?

'why ETTR should _never_ be used together clipping highlights'

Not sure what you mean...

If the scene contrast is less than the dynamic range of the sensor, then you are leaving IQ on the table if you don't do ETTR...at what point do you consider 'clipping highlights' to be an issue?

If the scene is less than 8, 10 or 12 stops (depending on your sensor and your determination of base level noise) where do you expose a raw image capture?

Let's be clear, under exposing sucks, right?

So, if the scene is less than the sensor, where do YOU suggest we expose? Up to but under clipping?

Just want to be sure we're talking about the same thing...ETTR, when the scene is LESS than the sensor DR, makes sense in terms of IQ.

Right?

[hjulenissen]

Huh?

'why ETTR should _never_ be used together clipping highlights'

Not sure what you mean...

I was referring to this:

Using ETTR, the exposure is pushed to the right such that no clipping occurs.  None.  Not 'unwanted clipping'.  Not 'non-specular highlight clipping'.  No clipping.
...
for ETTR to be of any value the scene/subject contrast has to fit within the range of the sensor.

If the scene contrast is less than the dynamic range of the sensor, then you are leaving IQ on the table if you don't do ETTR...at what point do you consider 'clipping highlights' to be an issue?

I am proposing that any given pixel will look better the closer to the sensor saturation point it is brought without actually clipping(given that artistic goals about scene movement, DOF etc actually allows some freedom in exposure). If possible, we would want all pixels to be just on the virge of saturation. Sadly, that is only possible if you take snaps of gray, uniform walls.

For real scenes, some pixels may be close to saturation, and others far from it. We might even have some that are outright clipped and some that are undistinguishable from the noise floor. How should we ideally expose - I think that is what ETTR tries to solve? Traditional techniques tried to home in on some sort of "midpoint" of the scene brightness - suitable for a sensor technology with sort of symmetric degrees of error for both very bright and very dark parts of the scene - soft clipping of highlights and soft disappearance into the noisefloor

My point is that even for scenes of very high dynamic range, it makes sense to optimize the exposure of the pixels of interest. We want them as hot as possible without clipping them. There might be pixels of less interest, or pixels that we are willing to sacrifice given that our sensors have finite DR. I am basically saying "don't care about the pixels you don't care about".

I see no fundamental difference between scenes of high DR and scenes of low DR when it comes to exposure. Figure out what parts matter to you (art) figure out how to capture those parts as accurate as possible (science). Perhaps bracketing is needed if you decide that a large dynamic range is your artistic goal. But if a bright, small lightsource is deemed irrelevant, then you might choose to just clip it and do the exposure as if it was not there - using e.g. ETTR.

-h