Luminous Landscape Forum

Equipment & Techniques => Digital Cameras & Shooting Techniques => Topic started by: torger on October 26, 2011, 07:30:46 am

Title: Universal white balance and ETTR
Post by: torger on October 26, 2011, 07:30:46 am
Those that shoot still life often practice ETTR, that is expose the sensor with as much light as possible without clipping highlight detail.

Having practiced this for a while I have been troubled by that it is so easy to mistakingly overexpose that a better practice may actually be to play it safe -- rather a bit noisier shadows than completely lost highlights. Recent articles often state that ETTR practice is flawed and not applicable on modern cameras which has better dynamic range than the early digital cameras. My approach has been to make a safe shot first, and then bracket a few brighter shots and decide in post which exposure to use.

The general problem is that Canon/Nikon (and most other) camera manufacturers do not provide RAW histograms in camera, the histograms is on the camera-generated JPEG even if you shoot RAW, and is affected by white balance setting etc so it may differ a lot from how the actual sensor's RGB channels are exposed. Making ETTR decisions from the camera histograms can lead to over-exposure or under-exposure depending on lighting conditions, certainly not very precise or safe.

So I decided to test the "universal white balance" trick on a Canon camera. It does seem to work, that is ETTR decisions can be safely made down to 1/3 stop from RGB histograms in camera (since clipping levels match actual RAW data), even the live view RGB histogram works well (note you must use RGB histograms, luminance histograms are useless for this purpose). You can even safely make such bold exposure decisions like clipping one channel to reconstruct it in postprocessing which may be ok if the highlight is white (a bright cloud for example). The drawback with uniwb is that embedded JPEGs look green due to this white balance setting, which you may ignore if you want to since you will be using the RAW data anyway, but since this type of exposure optimization is only used in still life photography I usually have the time to change to a natural white balance before taking the final shot. I think it is nice to have a natural-looking embedded camera JPEG as reference.

I also noted that "ETTR" if practiced not to clip highlights (as it should) may quite often actually expose more to the left than standard metering, but of course that depends on how the camera's auto exposure is designed. That ETTR may actually lead to more conservative exposures (more to the left) on modern cameras can make it a bit confusing concept.

Anyway, after doing these initial uniwb tests it seems to me that if you are going to practice ETTR and your camera cannot show RAW RGB histograms you really should consider using uniwb.

Is anyone else using universal white balance to make manual exposure decisions? It would be nice to hear some experiences from those that have used it for a while, if it indeed is safe or if you still come home with over-exposed / under-exposed pictures.

For those new to uniwb, here's a link (for Canon cameras) http://www.guillermoluijk.com/tutorial/uniwb/index_en.htm
Title: Re: Universal white balance and ETTR
Post by: bjanes on October 26, 2011, 09:54:04 am
The general problem is that Canon/Nikon (and most other) camera manufacturers do not provide RAW histograms in camera, the histograms is on the camera-generated JPEG even if you shoot RAW, and is affected by white balance setting etc so it may differ a lot from how the actual sensor's RGB channels are exposed. Making ETTR decisions from the camera histograms can lead to over-exposure or under-exposure depending on lighting conditions, certainly not very precise or safe.

In my experience with my Nikon cameras, ETTR according to the camera RGB histogram never results in clipped highlights in any channel. Most cameras underexpose to allow highlight headroom and adjust the tone curve so that the histogram and preview image appear normally exposed rather than dark. Most Nikons seem to allow 0.5 EV of headroom for highlights. The Phase One IQ 180 allows nearly two stops of headroom. This headroom results in conservative camera histograms.

With daylight exposure, the green of most cameras is most sensitive and blows first. With D50 illumination the sensitivities of the Nikon D3 are shown below in relation to sRGB primaries (data from DXO).

(http://bjanes.smugmug.com/Photography/ETTR2/i-J4Gs7JZ/0/O/D3ColorSens.png)

To equalize the channel sensitivities for a given illumination, the raw channels are multiplied by a white balance coefficient. For the D3 and D50 (near daylight illumination) the DXO testing shows that WB multipliers are 2.24, 1.0, and 1.2 respectively for red, green, and blue. If the image contains saturated red or blue, WB multipliers greater than 1.0 can cause clipping in these channels with white balance when none is present in the raw file. However, the reverse does not occur.

UniWB merely sets the WB multipliers all to unity (1.0) and is useful when one is photographing subjects with saturated reds or blues because it eliminates clipping induced by white balance. Saturation clipping can still occur if the colors are out of the sRGB or aRGB of the working spaces of Nikon cameras. Again, the RGB histograms will be conservative and the raw channels will not be clipped in the rendered image for the camera preview are not clipped.

Anyway, after doing these initial uniwb tests it seems to me that if you are going to practice ETTR and your camera cannot show RAW RGB histograms you really should consider using uniwb.

Is anyone else using universal white balance to make manual exposure decisions? It would be nice to hear some experiences from those that have used it for a while, if it indeed is safe or if you still come home with over-exposed / under-exposed pictures.

UniWB will often allow more exposure to the right for the red and blue channels, but using normal WB will result in conservative histograms and there will be no raw channel clipping. With modern low noise sensors, a degree of underexposure can often be tolerated. For most of my work, I find UniWB to be more trouble than it is worth. The main problem with ETTR is to correlate channel clipping in the camera histogram with the raw histogram shown by programs such as Rawnalize or DCRaw that look directly at the raw file. You need to know how much headroom your camera allows.

Regards,

Bill
Title: Re: Universal white balance and ETTR
Post by: torger on October 26, 2011, 10:14:37 am
I tested this in indoor lighting with a Canon 7D and then I got about 1/3 stop clipping in the raw file of the green channel without any clip in the normal white balance RGB histogram (it showed that the red channel was about to clip, which it was not, and the green channel ok which was actually clipped), while uniwb presented the correct result. So this camera does not seem to be conservative as the Nikon D3. There are JPEG picture style settings to adjust though (I have default settings) that maybe will affect the result.

Current canon cameras does not exactly excel in dynamic range, so being able to extract the most out of it in a safe way is quite attractive to me. But I'll see, I'll try this practice for a while and see how it works. Since I've bracketed and test-shot a lot previously to tune an exposure, flipping white-balances actually seems like a faster way to work for me.

I've though about the color space thing, if the camera really clips the jpeg if the color does not fit or if it makes a perceptual mapping, so the histograms will still not show clipping. Something to test.
Title: Re: Universal white balance and ETTR
Post by: bjanes on October 26, 2011, 12:17:34 pm
I tested this in indoor lighting with a Canon 7D and then I got about 1/3 stop clipping in the raw file of the green channel without any clip in the normal white balance RGB histogram (it showed that the red channel was about to clip, which it was not, and the green channel ok which was actually clipped), while uniwb presented the correct result. So this camera does not seem to be conservative as the Nikon D3. There are JPEG picture style settings to adjust though (I have default settings) that maybe will affect the result.

How are you determining clipping in the raw channels? Are you using Rawnalize, DCRaw, or another program that looks directly at the channels? And how are you setting UniWB? Since UniWB should have a green multiplier of 1.0, I don't see how the regular and UniWB histograms for the green channel should be different, unless something else has changed. For tungsten lighting (approximately Illuminant A), the light contains more red and the red channel sensitivity will be closer to the green channel as shown in the shown DXO measurements. Still the green channel is most sensitive and should clip before the red unless the scene contains a lot of saturated red. Does your test target include saturated reds or does it have neutrals?

Regards,

Bill
Title: Re: Universal white balance and ETTR
Post by: feppe on October 26, 2011, 01:35:02 pm
Those that shoot still life often practice ETTR, that is expose the sensor with as much light as possible without clipping highlight detail.

Having practiced this for a while I have been troubled by that it is so easy to mistakingly overexpose that a better practice may actually be to play it safe -- rather a bit noisier shadows than completely lost highlights.

This is exactly the approach (http://theonlinephotographer.typepad.com/the_online_photographer/2011/10/expose-to-the-right-is-a-bunch-of-bull.html) Ctein takes on TOP, and is diametrically opposed to Michael's recent article (http://www.luminous-landscape.com/tutorials/optimizing_exposure.shtml) on the subject where he still appears to advocate ETTR despite its shortcomings with current camera histograms.

On topic, I found out that Olympus cameras have an ETTR setting (http://www.luminous-landscape.com/forum/index.php?topic=46071.0), but it has its shortcomings - mainly that it's a spot metering mode, and Guillermo Luijk's research down the thread shows it leaves too much headroom. I have not used it since, and prefer to take Ctein's approach.
Title: Re: Universal white balance and ETTR
Post by: torger on October 26, 2011, 01:37:20 pm
How are you determining clipping in the raw channels? Are you using Rawnalize, DCRaw, or another program that looks directly at the channels? And how are you setting UniWB? Since UniWB should have a green multiplier of 1.0, I don't see how the regular and UniWB histograms for the green channel should be different, unless something else has changed. For tungsten lighting (approximately Illuminant A), the light contains more red and the red channel sensitivity will be closer to the green channel as shown in the shown DXO measurements. Still the green channel is most sensitive and should clip before the red unless the scene contains a lot of saturated red. Does your test target include saturated reds or does it have neutrals?

I am using rawtherapee which can display the raw histogram (a bit crude though). I'm setting uniwb using an overexposed cr2 (downloaded from the link above), a trick which works on some cameras (not all) but 7D is one of them. I'm just shooting onto some detail in my kitchen so it is not a special test target, quite neutral colors with lots of white, no saturated colors. With uniwb green channel is clearly above red, changing to tungsten wb red channel passes green in the histogram and if you go to a higher temperature wb red passes even more, despite no saturated reds, it is the color of the white furniture and walls caused by the white balance that makes this happen. If the white balance is such that the bright white furniture in this case appears slightly red then red will clip earlier. Or say if my white balance was perfect and the furniture would be slightly red it would be the same result. It is not only saturation that causes a channel to clip, brightness too. A saturated red is R 100 G 0 B 0, but a white with hint of red R 100 G 90 B 90 is not saturated but can cause clipping anyway.

However, there are many aspects here that I need to look into, I have looked at this for only 45 minutes so far... that slight overexposure I got could be a mistake, it does seem that the largest gain is to be able to ETTR more under certain lighting conditions.
Title: Re: Universal white balance and ETTR
Post by: torger on October 26, 2011, 01:46:44 pm
This is exactly the approach (http://theonlinephotographer.typepad.com/the_online_photographer/2011/10/expose-to-the-right-is-a-bunch-of-bull.html) Ctein takes on TOP, and is diametrically opposed to Michael's recent article (http://www.luminous-landscape.com/tutorials/optimizing_exposure.shtml) on the subject where he still appears to advocate ETTR despite its shortcomings with current camera histograms.

On topic, I found out that Olympus cameras have an ETTR setting (http://www.luminous-landscape.com/forum/index.php?topic=46071.0), but it has its shortcomings - mainly that it's a spot metering mode, and Guillermo Luijk's research down the thread shows it leaves too much headroom. I have not used it since, and prefer to take Ctein's approach.

Ctein's approach simply seems to be "pull an guesstimated large amount too the left just to make sure you don't clip since you can't trust camera display or histograms" and I don't really like that  ;) , that would be wasting 1 - 2 stops in every picture, and as a Canon user I cannot afford that :).

If this white balance trick works to show clipping safely (which it does seem, although it might be a little bit conservative due to color space reduction) then I have the tool I need. Bjanes suggest that it may provide only mariginal advantage over standard RGB histograms since they are conservative, I shall look more into that, but if so then RGB histograms with any white balance will work alright. Extremely small highlights that don't really become visible in histograms and still should have detail are really a special case (just like the case when you need to clip a large part of the picture) and then there's the conservative guesstimate fallback, but only then. As I said in the first post, practicing ETTR with histograms really often leads to more conservative exposure than using auto-exposure, so it often becomes an expose to the left in practice.
Title: Re: Universal white balance and ETTR
Post by: bjanes on October 26, 2011, 02:21:37 pm
This is exactly the approach (http://theonlinephotographer.typepad.com/the_online_photographer/2011/10/expose-to-the-right-is-a-bunch-of-bull.html) Ctein takes on TOP, and is diametrically opposed to Michael's recent article (http://www.luminous-landscape.com/tutorials/optimizing_exposure.shtml) on the subject where he still appears to advocate ETTR despite its shortcomings with current camera histograms.

Ctein makes an interesting point, and I should revise my statement that highlights are never blown when the camera histogram shows no clipping--there may be clipped areas that are too small to show up in the camera histogram. To paraphrase Bruce Fraser, proper ETTR involves not blowing highlights you wish to preserve. Ctein's image in the article was not properly exposed to the right if he wanted to retain those highlights . Jeff Schewe's response (http://schewephoto.com/ETTR/index.html) is good, but in his image of the Niagra Falls, the degree of overexposure is not nearly so great as the ACR histogram would indicate. Examination of that raw file with Rawnalize demonstrates much less clipping.

Rather than systematically underexposing all one's images in order to prevent possibly blown highlights, the more prudent photographer would anticipate those situations where the camera histogram might not detect small important clipped highlights. For night scenes, bracketing gives the best chance of obtaining an optimal exposure.

Regards,

Bill
Title: Re: Universal white balance and ETTR
Post by: feppe on October 26, 2011, 02:27:01 pm
Rather than systematically underexposing all one's images in order to prevent possibly blown highlights, the more prudent photographer would anticipate those situations where the camera histogram might not detect small important clipped highlights. For night scenes, bracketing gives the best chance of obtaining an optimal exposure.

I do quite a bit of night photography with city lights in the frame, and bracketing is indeed the only solution. Sometimes 5 seconds is enough, sometimes 60 seconds is not. Also, digital blending (or HDR or exposure fusion if you're so inclined) is often necessary to contain the DR of the scene, in which cases ETTR is not very helpful.

I find that I can usually get along with just 2 exposures, one for highlights, one for overall scene, and blend in post. The problem is figuring out which 2 brackets are the correct ones while in the field is pretty difficult due to same issues with histogram (in)accuracy and lack of true RAW histogram.
Title: Re: Universal white balance and ETTR
Post by: torger on October 26, 2011, 03:34:20 pm
I've done some more testing with a color checker and a light source similar to daylight.

Observations: Bjanes was right, no matter which white balance the RGB histograms are conservative so it will not clip raw channels. The advantage of uniwb is thus that you can expose less conservative and still be sure that you don't overexpose. In daylight scenes I estimate the gain to be about 2/3 stops, that is not very large, but if you don't have much DR it may be valuable 2/3 stops. With more extreme lighting the gain can be 1 - 2 stops.

For saturated colors the camera's JPEG color space will keep the RGB histograms conservative regardless of uniwb or not, so AdobeRGB will make you a bit less conservative than sRGB, but for scenes with very saturated colors you may still underexpose with a stop or so.

Histograms need to be quite readable to be useful, so you'll need a newer camera with 3" screen to do this. Still you'll miss really small highlights, so as discussed above night scenes can be tricky.
Title: Re: Universal white balance and ETTR
Post by: NikoJorj on October 26, 2011, 04:43:36 pm
This is exactly the approach (http://theonlinephotographer.typepad.com/the_online_photographer/2011/10/expose-to-the-right-is-a-bunch-of-bull.html) Ctein takes on TOP, and is diametrically opposed to Michael's recent article (http://www.luminous-landscape.com/tutorials/optimizing_exposure.shtml) on the subject where he still appears to advocate ETTR despite its shortcomings with current camera histograms.
Why should these appear as opposite views? I'd thought that since advent of digital, everybody knows that clipping is to be avoided, and that it's taking for granted in Michael's article. The main difference is a tradeoff between noise and the risk of clipping.

Quote
On topic, I found out that Olympus cameras have an ETTR setting (http://www.luminous-landscape.com/forum/index.php?topic=46071.0), but it has its shortcomings - mainly that it's a spot metering mode, and Guillermo Luijk's research down the thread shows it leaves too much headroom. I have not used it since, and prefer to take Ctein's approach.
I discovered it too (have an EPL1 since 2 weks), and even if it's not optimized for raw, I feel for now that it's the best guesstimate at hand when you have to shoot quickly.

Of course, it would be much more simple if manufacturers would give us the proper tools (http://www.luminous-landscape.com/forum/index.php?topic=33267), but as long as pigs don't fly ::) , is there a way to set a different exposition bias for spot metering (+3EV) and matrix metering (say 0EV) on Canon cameras that have spot metering?
Olympus provides a way to fine tune each metering method (on my EPL1 it's custom menu/J-Utility/Exposure shift), and it's quite helpful in the present case.
Title: Re: Universal white balance and ETTR
Post by: torger on October 27, 2011, 01:52:58 am
Of course, it would be much more simple if manufacturers would give us the proper tools (http://www.luminous-landscape.com/forum/index.php?topic=33267), but as long as pigs don't fly ::) , is there a way to set a different exposition bias for spot metering (+3EV) and matrix metering (say 0EV) on Canon cameras that have spot metering?

I think I've heard the new 1D X have possibilities to tune auto exposure function with preset biases, but I don't think this 7D has. I also note that live view metering really sucks, it is supposed to be matrix metering but it is heavily affected where you have your zoom box. Normal matrix metering misses small highlights. So for still life, manual exposure guided by histograms is the only sane way to do it on this camera. Just hope that they eventually implement real histograms and real clipping indications based on RAW data in coming products, but they haven't done it yet and it is an obvious feature since many years so I don't expect it to happen soon.
Title: Re: Universal white balance and ETTR
Post by: RFPhotography on October 27, 2011, 11:50:36 am
Ctein is missing a couple key points with his article.  BJanes brings up one of them.

Using ETTR, the exposure is pushed to the right such that no clipping occurs.  None.  Not 'unwanted clipping'.  Not 'non-specular highlight clipping'.  No clipping.

Also, for ETTR to be of any value the scene/subject contrast has to fit within the range of the sensor.  The scene CTein was using as an illustration fails that aspect of ETTR. 
Title: Re: Universal white balance and ETTR
Post by: hjulenissen on October 27, 2011, 02:19:43 pm
Ctein is missing a couple key points with his article.  BJanes brings up one of them.

Using ETTR, the exposure is pushed to the right such that no clipping occurs.  None.  Not 'unwanted clipping'.  Not 'non-specular highlight clipping'.  No clipping.

Also, for ETTR to be of any value the scene/subject contrast has to fit within the range of the sensor.  The scene CTein was using as an illustration fails that aspect of ETTR. 
That is not how I understand the popular consensus of how "ETTR" should be interpreted, and it does not seem like a sensible approach.

1. Figure out what parts of the true scene luminance is to be accurately captured
2. Use ETTR to make sure that the brightest _important_ parts of the scene are as close to the sensor saturation point in each primary, but not beyond it

Doing 1 and 2 may well give you clipped highlights, but only if you think that it is ok. It will give you the most "accurate" capture of the prioritized luminance range that your equipment is capable of.

-h
Title: Re: Universal white balance and ETTR
Post by: RFPhotography on October 27, 2011, 03:30:39 pm
Just because an interpretation is 'popular' doesn't make it right. 
Title: Re: Universal white balance and ETTR
Post by: hjulenissen on October 27, 2011, 04:29:07 pm
Just because an interpretation is 'popular' doesn't make it right. 
No, but if a definition is both popular and sensible, I think that both make my case stronger. Of course, you may choose to provide arguments that it is not sensible, or that some expert thinks otherwise.

-h
Title: Re: Universal white balance and ETTR
Post by: RFPhotography on October 27, 2011, 04:58:26 pm
Well clearly you don't consider me an expert.   ;D

http://schewephoto.com/ETTR/index.html

Read and be educated.
Title: Re: Universal white balance and ETTR
Post by: hjulenissen on October 28, 2011, 01:44:22 am
Well clearly you don't consider me an expert.   ;D
On the internet, everyone is his/her own expert, so refering to oneself as an authority in discussions tends to be less than productive :-)
Quote
http://schewephoto.com/ETTR/index.html

Read and be educated.
I think that this sentence is more or less in line with what I said:
"if you have highlights whose texture and detail is important"

I will give you that he supports what you have been saying here:
" This is not intended to be used in every photographic situation, only those situations where the scene's contrast range is lower than you camera's sensor dynamic range."


So basically, someone agrees with you. I still think that it sounds counterproductive, and neither you nor Schewe (?) have provided arguments for why ETTR should _never_ be used together clipping highlights.

My argument is very simple. If a single point-source of light is dramatically more bright than the average brightness of the scene and covering e.g. exactly one pixel in my camera, I may not care that it is clipped. I may not care that it is accurately rendered in the final image. If it has a color cast, I can relatively easily fix it later on. What IS important to me is that the remaining 17.999 megapixels turn into a great image. I believe that the insight gained by ETTR can help me ensure that the interesting part of the image is slightly better.

-h
Title: Re: Universal white balance and ETTR
Post by: Schewe on October 28, 2011, 02:24:42 am
So basically, someone agrees with you. I still think that it sounds counterproductive, and neither you nor Schewe (?) have provided arguments for why ETTR should _never_ be used together clipping highlights.

Huh?

'why ETTR should _never_ be used together clipping highlights'

Not sure what you mean...

If the scene contrast is less than the dynamic range of the sensor, then you are leaving IQ on the table if you don't do ETTR...at what point do you consider 'clipping highlights' to be an issue?

If the scene is less than 8, 10 or 12 stops (depending on your sensor and your determination of base level noise) where do you expose a raw image capture?

Let's be clear, under exposing sucks, right?

So, if the scene is less than the sensor, where do YOU suggest we expose? Up to but under clipping?

Just want to be sure we're talking about the same thing...ETTR, when the scene is LESS than the sensor DR, makes sense in terms of IQ.

Right?
Title: Re: Universal white balance and ETTR
Post by: hjulenissen on October 28, 2011, 02:48:33 am
Huh?

'why ETTR should _never_ be used together clipping highlights'

Not sure what you mean...
I was referring to this:
Using ETTR, the exposure is pushed to the right such that no clipping occurs.  None.  Not 'unwanted clipping'.  Not 'non-specular highlight clipping'.  No clipping.
...
for ETTR to be of any value the scene/subject contrast has to fit within the range of the sensor.

If the scene contrast is less than the dynamic range of the sensor, then you are leaving IQ on the table if you don't do ETTR...at what point do you consider 'clipping highlights' to be an issue?
I am proposing that any given pixel will look better the closer to the sensor saturation point it is brought without actually clipping(given that artistic goals about scene movement, DOF etc actually allows some freedom in exposure). If possible, we would want all pixels to be just on the virge of saturation. Sadly, that is only possible if you take snaps of gray, uniform walls.

For real scenes, some pixels may be close to saturation, and others far from it. We might even have some that are outright clipped and some that are undistinguishable from the noise floor. How should we ideally expose - I think that is what ETTR tries to solve? Traditional techniques tried to home in on some sort of "midpoint" of the scene brightness - suitable for a sensor technology with sort of symmetric degrees of error for both very bright and very dark parts of the scene - soft clipping of highlights and soft disappearance into the noisefloor

My point is that even for scenes of very high dynamic range, it makes sense to optimize the exposure of the pixels of interest. We want them as hot as possible without clipping them. There might be pixels of less interest, or pixels that we are willing to sacrifice given that our sensors have finite DR. I am basically saying "don't care about the pixels you don't care about".

I see no fundamental difference between scenes of high DR and scenes of low DR when it comes to exposure. Figure out what parts matter to you (art) figure out how to capture those parts as accurate as possible (science). Perhaps bracketing is needed if you decide that a large dynamic range is your artistic goal. But if a bright, small lightsource is deemed irrelevant, then you might choose to just clip it and do the exposure as if it was not there - using e.g. ETTR.

-h
Title: Re: Universal white balance and ETTR
Post by: torger on October 28, 2011, 03:41:31 am
ETTR is not exactly a scientifically defined term, so I guess its meaning can differ a bit depending on who explains it.

I can realize that it may be confusing especially for us landscape photographers since a large number of outdoor scenes have some impossible highlight, lamp, sun, star, bright water reflection etc, and then it is better to talk about suitable exposure to get the right look of highlights, a little bit of "glow" around a lamp may be suitable for example. I rarely find it valuable to capture highlights that are too bright for the eye to see when standing at the scene. The moon is an interesting example, for that I usually find it suitable to overexpose it a little, still keep some structure on it but get the glow around it, it often gives the most natural look.

Another problem is that histograms in cameras are so poorly detailed that you won't really detect in them if small highlights are overexposed. Often clipping "blinkies" are only on luminance and will thus often not blink if one channel is clipped etc. It would be very easy for the manufacturers to provide the tools needed to be able to exactly evaluate the exposure, but by some unknown reason they don't think photographers are interested in having those tools. Uniwb seems to improve this a little bit, but I understand those that think it is too little to be worth it.

I think the best way to explain how to expose digitally is that you should concentrate on the highlights, try to get them as close as saturation as possible in order to capture as many photons from the scene as possible for best signal-to-noise ratio in the whole picture, and if not possible to fit them without overexposure then decide what a suitable amount of overexposure is, but keep in mind what is lost is lost so do not overdo it, and that poor camera exposure evaluation tools can fool you, so play it safe and bracket if there is a difficult situation. Sounds pretty much as expose-to-the-right to me.
Title: Re: Universal white balance and ETTR
Post by: RFPhotography on October 28, 2011, 07:37:47 am
To be perfectly clear, h I didn't refer to myself as an expert. 

If you actually think about it, the two go hand in hand.  If a scene has specular highlights then it's not going to fit within the range of the sensor because those highlights are going to be several stops brighter than 'regular' light.  It's often even difficult to control specular highlights with HDR or other blending methods because it's sometimes difficult or impractical to bracket that many stops. 

And if you don't know who Jeff is, then I'd suggest doing a bit of research because he knows a hell of a lot more than either of us likely ever will.
Title: Re: Universal white balance and ETTR
Post by: NikoJorj on October 28, 2011, 07:42:36 am
I think the best way to explain how to expose digitally is that you should concentrate on the highlights, try to get them as close as saturation as possible in order to capture as many photons from the scene as possible for best signal-to-noise ratio in the whole picture, and if not possible to fit them without overexposure then decide what a suitable amount of overexposure is, but keep in mind what is lost is lost so do not overdo it, and that poor camera exposure evaluation tools can fool you, so play it safe and bracket if there is a difficult situation. Sounds pretty much as expose-to-the-right to me.
Aaaamen, ite missa est, and all that sort of things as far as I am concerned. ;)

It's often even difficult to control specular highlights with HDR or other blending methods because it's sometimes difficult or impractical to bracket that many stops. 
And moreover, those specular haighlight themselves are often best rendered as pure white in a print, so clipping them is not that a problem. The trickier part is to render the "glow" around it, it's where having enough non-clipped data can make a huge difference.
In some cases as the sun veiled by thin clouds, it's not that easy to render and another exposure for the glow itself might be what is needed...
Title: Re: Universal white balance and ETTR
Post by: hjulenissen on October 28, 2011, 08:41:09 am
If you actually think about it, the two go hand in hand.  If a scene has specular highlights then it's not going to fit within the range of the sensor because those highlights are going to be several stops brighter than 'regular' light.  It's often even difficult to control specular highlights with HDR or other blending methods because it's sometimes difficult or impractical to bracket that many stops.  
I dont see how any of that contradicts my initial statement. You seem to presuppose that all highlights must always be captured accurately. My point is that sometimes it makes sense to clip them. Sure the image wont fit within the range of the sensor. So what, if the image looks better as a result?
And if you don't know who Jeff is, then I'd suggest doing a bit of research because he knows a hell of a lot more than either of us likely ever will.
I have seen good posts from Schewe. I try to evaluate statements based on their content, not so much on the author. On the bright side, that means that I am open for good arguments from you even though I don't know you.

-h
Title: Re: Universal white balance and ETTR
Post by: RFPhotography on October 28, 2011, 08:56:04 am
No, I don't presuppose that all highlights must be captured properly.  I understand that sometimes some highlights may be clipped.  What I'm saying though is that if highlights are clipped then ETTR doesn't apply.  'Better' is a subjective term.  What's better to you may not be to me and vice versa.
Title: Re: Universal white balance and ETTR
Post by: hjulenissen on October 28, 2011, 12:41:25 pm
No, I don't presuppose that all highlights must be captured properly.  I understand that sometimes some highlights may be clipped.  What I'm saying though is that if highlights are clipped then ETTR doesn't apply.  'Better' is a subjective term.  What's better to you may not be to me and vice versa.
Then we just have do agree that we disagree. I believe that you think that ETTR is about increasing exposure until the brightest pixels nearly saturate. I believe that ETTR is about increasing exposure until the brightest pixels that one care about nearly saturate. In all fairness, that difference is fairly minor in the big picture.

-h
Title: Re: Universal white balance and ETTR
Post by: RFPhotography on October 28, 2011, 01:30:13 pm
In all fairness, that difference is fairly minor in the big picture.

-h

True.
Title: Re: Universal white balance and ETTR
Post by: Guillermo Luijk on October 28, 2011, 08:47:55 pm
For those new to uniwb, here's a link (for Canon cameras) http://www.guillermoluijk.com/tutorial/uniwb/index_en.htm

The link is not exclusively for Canon cameras, it's for any camera providing custom WB on a neutral gray card.

Regarding the ETTR discusion, I think it's all about semantics. Anyway IMO saying that ETTR is only of application when absolutely no highlights are clipped, sounds like trying to put a silly restriction on a still 100% valid technique.

ETTR is a concept, not a definition. In a case with specular highlights, one could ETTR the rest of the scene. In a scene of a bride and groom in the shadow with a bright sky in the frame, one could ETTR the sky (so there would be no clipped areas in the capture), but one could prefer to ETTR the bride's dress instead (the highlights of interest), in order to have less noise in the groom's suit (in that case the sky would get clipped, no problem with that regarding the ETTR concept):

(http://www.guillermoluijk.com/article/ettr3/novios.jpg)

Regards
Title: Re: Universal white balance and ETTR
Post by: RFPhotography on October 28, 2011, 09:42:09 pm
(in that case the sky would get clipped, no problem with that regarding the ETTR concept):



That's a matter of opinion.
Title: Re: Universal white balance and ETTR
Post by: Guillermo Luijk on October 28, 2011, 09:54:03 pm
That's a matter of opinion.

It is not an opinion, it is a fact that the bride's dress was ETTR.
Title: Re: Universal white balance and ETTR
Post by: RFPhotography on October 28, 2011, 11:26:09 pm
You missed the point entirely.
Title: Re: Universal white balance and ETTR
Post by: Guillermo Luijk on October 29, 2011, 06:21:51 am
You missed the point entirely.

...and that is an opinion.
Title: Re: Universal white balance and ETTR
Post by: madmanchan on November 03, 2011, 03:51:51 pm
This is why I think improved camera histograms don't address the issue at hand.  I think it would be vastly preferable to have a visualization overlay on the image itself to show which pixels are clipped in the raw data.

As an example, it is common in image processing software to see a visualization of which pixels have clipped (e.g., red highlight warning in ACR).  As you drag Exposure slider to the right, eventually more and more "red warning" pixels show up, right?  Well, would be very nice if you could do this on the camera (with a knob) at capture time ...
Title: Re: Universal white balance and ETTR
Post by: NikoJorj on November 03, 2011, 04:06:29 pm
This is why I think improved camera histograms don't address the issue at hand.  I think it would be vastly preferable to have a visualization overlay on the image itself to show which pixels are clipped in the raw data.
Rethinking to it (with the almost-ETTR feature of my new EPL1), I'd say you do need both to be efficient in the field : either the photo is underexposed, and then the histogram tells you by how much it is, or it is overexposed and the HL blinker tells you where it is blown.
But if one has to choose (alas, one must with EPL1, no blinkers AND histogram in the same display) I agree the HL blinker is more useful. I still miss the histogram when nothing blinks on the HL side.
Title: Re: Universal white balance and ETTR
Post by: Guillermo Luijk on November 03, 2011, 05:48:43 pm
This is why I think improved camera histograms don't address the issue at hand.  I think it would be vastly preferable to have a visualization overlay on the image itself to show which pixels are clipped in the raw data.

If I had to choose, I would also prefer visual highlight clipping warning, but not so vastly as you suggest. At least not if there is not also an underexposure warning (certain number of stops below saturation). Otherwise you can be sure not to clip the highlights, but there will remain uncertainty about noise that can be expected when lifting the shadows.

In this sense, a dynamic histogram in EV divisions can even be more accurate and helpful than clip warnings. If this histogram let's you know visually the range of acceptable noise, it can be even used to do ETTL (Expose to the left = expose as low as possible right before having SNR below a threshold in the shadows) in critical shutter and/or DOF applications, where we want to expose as less as possible.

With this kind of histogram:

(http://www.guillermoluijk.com/misc/histogram.gif)

The RAW shooter just needs to make sure the histogram falls into the range between clipping and noise. Any exposure inside this range will be good enough regarding noise and highlight information.

Regards
Title: Re: Universal white balance and ETTR
Post by: ErikKaffehr on November 04, 2011, 12:41:46 am
Hi,

I'd would prefer blinking highlights and also shadows markings. I like the histogram, but it does not show what is clipped. The dynamic histogram suggested by Guillermo is good, but I think that we could simply have a clipping warning.

I really would like to see more live view based viewfinder solutions, and in that context I feel that we should maximize information but minimize viewfinder intrusion.

Best regards
Erik



If I had to choose, I would also prefer highlight clipping, but not so vastly as you suggest. At least not if there is not also an underexposure warning (certain number of stops below saturation). Otherwise you can be sure not to clip the highlights, but there will remain uncertainty about noise that can be expected when lifting the shadows.

In this sense, a dynamic histogram in EV divisions can even be more accurate and helpful than clip warnings. If this histogram let's you know visually the range of acceptable noise, it can be even user to do ETTL (Expose to the left = expose as low as possible right before having SNR below a threshold in the shadows) in critical shutter and/or DOF applications, where we want to expose as less as possible.

With this kind of histogram:

(http://www.guillermoluijk.com/misc/histogram.gif)

The RAW shooter just needs to make sure the histogram falls into the range between clipping and noise. Any exposure inside this range will be good enough regarding noise and highlight information.

Regards

Title: Re: Universal white balance and ETTR
Post by: RFPhotography on November 04, 2011, 11:21:41 am
Just to be clear, you all are talking about a live view image on the back LCD before the image is made, yes?  And it would have to be calculated off some sort of raw (unprocessed light hitting the sensor) data, yes?  Because I think pretty much all cameras have blinking highlight indicators available after capture calculated off JPEG data and the 'knob' in that case is the aperture value, shutter value or exp. comp. and retaking the shot (not always possible).

Or..... one could simply do a reasonable amount of testing with their camera in different conditions so that they know what headroom is available and not have to rely on the histogram or a clipping overlay, yes? 
Title: Re: Universal white balance and ETTR
Post by: fdisilvestro on November 04, 2011, 11:54:58 am
Just to be clear, you all are talking about a live view image on the back LCD before the image is made, yes?  And it would have to be calculated off some sort of raw (unprocessed light hitting the sensor) data, yes?  Because I think pretty much all cameras have blinking highlight indicators available after capture calculated off JPEG data and the 'knob' in that case is the aperture value, shutter value or exp. comp. and retaking the shot (not always possible).

Yes, current cameras have that for the rendered JPEG, we would like the information based on the raw values
Title: Re: Universal white balance and ETTR
Post by: madmanchan on November 04, 2011, 12:18:34 pm
Exactly:  want the clip indicator (overlay, visualization, etc.) to be calculated based on raw image pixel values, before white balance & other color-related processing is applied.
Title: Re: Universal white balance and ETTR
Post by: madmanchan on November 04, 2011, 12:22:36 pm
Quote
Otherwise you can be sure not to clip the highlights, but there will remain uncertainty about noise that can be expected when lifting the shadows.

Guillermo, my thinking is that as long as the data is not clipped, more light is better right?  So just increase the exposure dial till the clip warning overlay appears (over something that is important to you, e.g., can allow some small metallic areas to blow out if you want), and that's the limit.  If this creates an unacceptably long exposure time, then just reduce the exposure time to the acceptable limit.
Title: Re: Universal white balance and ETTR
Post by: RFPhotography on November 04, 2011, 12:40:01 pm
Sorry about the italics in that previous post.  Not sure why the entire string ended up italicised.  Only intended to do one word.

OK, that's what I figured.  But for it to be of best use, it has to be pre-capture for the reason noted previously.  Something akin to a peaking indicator on a video camera.  Eric, before WB?  But WB can impact colours/clipping so wouldn't it be better to have the preview include the WB?
Title: Re: Universal white balance and ETTR
Post by: fdisilvestro on November 04, 2011, 12:48:47 pm
If your Raw channels are not clipped then WB should not result in clipped channels, provided you do the right math in post processing

Some basic options:
- Use multipliers lower than 1 (this approach has issues when there are clipped raw values)
- Add bits (i.e. a 14 bit raw channel multiplied by the WB factor resulting in a 16 bit value)

Title: Re: Universal white balance and ETTR
Post by: NikoJorj on November 04, 2011, 12:55:08 pm
But WB can impact colours/clipping so wouldn't it be better to have the preview include the WB?
No : as long as the raw data is not clipped, recovery is dead easy (or almost so) in post. For me in LR, no math involved but some sliders tweaking (recovery+, exposure-/brightness+ eg).
Title: Re: Universal white balance and ETTR
Post by: fdisilvestro on November 04, 2011, 12:59:33 pm
Well, I meant the Raw processor to do the right math. LR certainly does the right math.
Title: Re: Universal white balance and ETTR
Post by: RFPhotography on November 04, 2011, 01:30:15 pm
If open an image in LR that has a lot of yellows/oranges/reds and push the WB to the right, I can induce clipping in the image.  

Attached are two images.  One taken with a WB of 10000 (read as 9100 in ACR), the other taken with a WB of 2500 (read as 2600 in ACR).  The histograms are different and the clipping indicators are different.  If WB doesn't affect a RAW exposure, what explains the difference between the two.  

Just so it can't be claimed that I've overexposed the images (ETTR'd, actually) I purposely took two additional shots at -1 exp comp..  Those screen shots are also included and, again, are very different based on the WB.
Title: Re: Universal white balance and ETTR
Post by: fdisilvestro on November 04, 2011, 01:51:47 pm
The difference is in the rendered image. If you opened both images in a tool like Rawnalize you will see no difference between both.

If the raw channels are not clipped, but clipping occurs as you adjust WB, you could then adjust back with the exposure slider (or recovery) and you'll be working out from real raw data.

In a simplified way, WB multiplies the Red and Blue channels by numerical factors depending on the color temperature and tint. Exposure is a linear multiplication of a same number for all the channels

So, suppose that a specific WB multiplies the Red channel by 2 and the Blue channel by 1.5. Here you might end with clipped values in the red and blue channels because of this multiplication.

Then, adjusting the exposure by -1 f stop will multiply all channels by 0.5, now you get back to non-clipped values.
Title: Re: Universal white balance and ETTR
Post by: fdisilvestro on November 04, 2011, 02:04:43 pm
Attached are two images.  One taken with a WB of 10000 (read as 9100 in ACR), the other taken with a WB of 2500 (read as 2600 in ACR).  The histograms are different and the clipping indicators are different.  If WB doesn't affect a RAW exposure, what explains the difference between the two.  

Bob,

I see that you have on both images the WB "as shot". Just do this test: On the image taken with the WB of 10000 (shown as 9100), use a custom WB of 2600; Tint +2 (as shown in the other image). In the second one, taken with the WB of 2500 use a custom WB of 9100; tint +12. You'll see that the results swap perfectly from one image to the other. This is because the RAW values are the same and what you see in ACR is the result of the rendered image according to the slider settings.
Title: Re: Universal white balance and ETTR
Post by: RFPhotography on November 04, 2011, 02:41:48 pm
See, that's what I always thought too, Francisco.  There was another thread recently that indicated otherwise.  Given that I'm a photographer and not a scientist I decided I was wrong and that the scientists were right.  

Making the adjustment you suggested gets them close but not exact.  That difference could be the slightly different framing of the two shots though.  

I just found the last version of Rawnalyze and looked at the histos for the two images.  Very close, but slight differences.  Again, likely due to the slightly different framing of the two shots.
Title: Re: Universal white balance and ETTR
Post by: fdisilvestro on November 04, 2011, 03:06:32 pm
There are a few cameras, mostly early models, that altered the raw values depending on the WB. I think the Nikon D1 was one of these cases. I don't know of new cameras that do that.

Regarding getting an exact match between exposures, there are some factors that can affect this:
- exact aperture repeatability (there might be minor differences between exposures due to mechanical limitations, some brands are more affected than others)
- illumination flickering

About Rawnalyze: The histogram have a linear scale, so they are difficult to compare if there is a slight exposure difference between images. It would be easier if the scale was logarithmic (it would be analogous to f/stops) as in Guillermo Luijk's Histogrammar (http://www.guillermoluijk.com/software/histogrammar/index.htm). Unfortunately, Rawnalize will not be developed any further since its creator passed away and its code wasn't public.

Title: Re: Universal white balance and ETTR
Post by: RFPhotography on November 04, 2011, 04:09:39 pm
Not really that difficult to analyse.  The small differences are pretty easily figured out.
Title: Re: Universal white balance and ETTR
Post by: fdisilvestro on November 04, 2011, 04:39:20 pm
Not really that difficult to analyse.  The small differences are pretty easily figured out.

You're right, it is not really that difficult. What happens is that in a linear scale the Histogram will "expand" or "contract" as you change exposure. In a logarithmic scale it would shift left or right (like in Guillermo's great animation), which I consider more intuitive. Two raw images where the only difference is exposure will show the same Histogram shape at different locations.
Title: Re: Universal white balance and ETTR
Post by: RFPhotography on November 04, 2011, 06:38:12 pm
You're right, it is not really that difficult. What happens is that in a linear scale the Histogram will "expand" or "contract" as you change exposure. In a logarithmic scale it would shift left or right (like in Guillermo's great animation), which I consider more intuitive.
Two raw images where the only difference is exposure will show the same Histogram shape at different locations.


I downloaded GL's tool and it's less intuitive to use than Rawnalyze.  It also seems to not directly read raw data unless the processed image is run through some process with dcraw.  I forget exactly what the pop up message was but it doesn't accept raw files natively, apparently.

Anyway, it's Friday night and, while this is fun and all, there are more fun things to do on a Friday.   ;D
Title: Re: Universal white balance and ETTR
Post by: Guillermo Luijk on November 08, 2011, 05:02:41 pm
in a linear scale the Histogram will "expand" or "contract" as you change exposure. In a logarithmic scale it would shift left or right (like in Guillermo's great animation), which I consider more intuitive.

This is a sequence of histograms from 8 captures from a white wall (it looks green because it has no WB applied, the G channel got max exposure as usual) taken at 1 stop intervals. The histogram doesn't change in shape, it just shifts.

Only once some channel (R and G) gets saturated, the neighbour pixels (B) are affected and their histogram slightly expands (see most exposed shot). In the least exposed shots, the lack of levels in the RAW files produces peaks in the histogram (this histograms correspond to demosaiced data, but without any WB applied nor colour profile conversion, so they can be considered as RAW histograms).

(http://www.guillermoluijk.com/misc/sequence.gif)


Regards
Title: Re: Universal white balance and ETTR
Post by: Graystar on December 02, 2011, 10:21:50 am
If the raw channels are not clipped, but clipping occurs as you adjust WB, you could then adjust back with the exposure slider (or recovery) and you'll be working out from real raw data.

In a simplified way, WB multiplies the Red and Blue channels by numerical factors depending on the color temperature and tint. Exposure is a linear multiplication of a same number for all the channels

So, suppose that a specific WB multiplies the Red channel by 2 and the Blue channel by 1.5. Here you might end with clipped values in the red and blue channels because of this multiplication.

Then, adjusting the exposure by -1 f stop will multiply all channels by 0.5, now you get back to non-clipped values.

I guess it depends on what tool you're using, but I've found that Rawnalyze and Raw Therapee don't work that way.  With both Rawnalyze and RT, if you reduce the exposure to -1 all the previously "blown" pixel become a lower value but they're all still the same...they're just not at 255 anymore.  In RT, if I use highlight recovery, the same thing happens...except that the rest of the image remains unchanged.

The only thing that worked for me in RT is the Highlight Reconstruction tool.  It seems to get rid of the obvious field of "blown but recovered" pixels.

If LR does something different, it would be interesting to hear what process it follows.

In any case...I see UNIWB as being quite a bit of work for too small a gain.  I'd rather not have to use a highlight recontruction tool on all of my images, and I don't feel like buying LR.  But of course, that just my personal feeling...others will feel differently.
Title: Re: Universal white balance and ETTR
Post by: ejmartin on December 03, 2011, 12:49:29 am
I guess it depends on what tool you're using, but I've found that Rawnalyze and Raw Therapee don't work that way.  With both Rawnalyze and RT, if you reduce the exposure to -1 all the previously "blown" pixel become a lower value but they're all still the same...they're just not at 255 anymore.  In RT, if I use highlight recovery, the same thing happens...except that the rest of the image remains unchanged.

The only thing that worked for me in RT is the Highlight Reconstruction tool.  It seems to get rid of the obvious field of "blown but recovered" pixels.


In RT, go to the raw tab and open Preprocessing>Raw white/black point.  Adjust the 'White point: linear correction factor slider' to a value less than one.  There are two ways that channels can be blown -- (1) in the raw data; and (2) due to white balance pushing a color channel past the nominal maximum (eg 255 for 8-bit depth per channel).  If all blown channel pixels are due to (2) then it makes sense to reduce the exposure to bring back the channels and not clip them; this is what the above tool in RT does -- adjusts what the channel clipping point is, and brings down all channels uniformly. 

The problem comes when you do this, and some channels are clipped -- then one will get nasty color tints in those highlights (typically pinkish, since the red WB factor is usually largest).  Now one might say, clip the ones where channels are clipped but recover the ones where no channel is clipped; but that can lead to some nasty transitions.  So, in RT the default is to clip to the white point anything over 255, regardless of whether it is from WB or actual clipping, so as to avoid nasty transitions and color tints.  But that can be bypassed with the above tool, or by turning on highlight reconstruction which tries to build back the highlights that are clipped in the raw data.