overexposure and color shifting

[spib]

I know overexposure and clipping highlights is to be handled with care in digital photography, but still I would like to ask the question if overexposing (beyond the right edge of the histogram) causes color shifting?

I know of the effect of blooming (colored edges) and have also read that it does not happen with CMOS. But is there color shifting that is caused by other technical reasons and if so what are that reasons? In a book I read that underexposure does not have any effect on color balance since digital photography is a linear process. What about overexposure?
(For instance if I make multiple exposures to combine them later.)

In the discussion about 'Expanding dynamic range, suppressing one colour channel' I found the following example:

I've brought up before (in the 'interesting article' relating to ETTR), the issue of 'true blue' skies shifting towards cyan during overexposure. I'm going to post an example of this effect and then try to predict what might happen if I had used a magenta filter. I'm really thinking while I'm writing because I don't have any hard evidence.

...

In other words, green has become more prominent as a result of the blue channel clipping. (Pure cyan is a mixture of blue and green in equal parts).

Does this mean, if one (or all) color channel is clipping there will be shifted colors? - If I look at the histogram of the mentioned picture I don't see any clipping in the blue channel. Can a channel clip without clipping in the histogram?

lots of questions - thanks for your help

[Andrew Larkin]

If one channel clips while the others remain intact, a colour shift will result.  I cannot see how this could not happen.

I have had this happen when photographing laser effects - the red of the laser turns yellow then white as the intensity of each channel clips.

Andrew

[DiaAzul]

In a book I read that underexposure does not have any effect on color balance since digital photography is a linear process. What about overexposure?
(For instance if I make multiple exposures to combine them later.)

[a href=\"index.php?act=findpost&pid=66401\"][{POST_SNAPBACK}][/a]

My observations on this are:

The sensors approximate to a linear device over their operating range. However, at the limits of exposure (over exposure) the sensor increasingly moves away from being a linear device up until the point that the sensor clips. The issue here is whether the camera manufacturer constrains the measured values (conversion to digital) to the part of the sensor response curve which approximates to linear response, or whether they allow measurements into the top end of the curve where the characteristics of the device moves signficantly away from a linear response.

Second observation is that the conversion from the raw sensor data to the final image (in whatever colour space or file format) has a major impact on the rendition of colour across the tonal range of the image. If the mappings from raw sensor RGB values to final image RGB values is not accurate then you will see colour shifts in the image. A good conversion will accommodate any non linearities of the sensor such that you perceive that the camera has a linear response. A poor quality convertor will assume that the sensor is linear and has imutable characteristics with colour temperature/ exposure value/ etc... and may result in colour shifts - this happened with ACR3.0 earlier and Canon SLR cameras such that in overblown highlights there was/ is a significant magenta shift in the displayed colours.

For the most part, I believe that most raw convertors are now of sufficient complexity and quality that colour shifting is less of a problem and that to a certain extent they are able to correct for clipped channels - though experimentation of how well they achieve this is required.

[Ray]

Does this mean, if one (or all) color channel is clipping there will be shifted colors? - If I look at the histogram of the mentioned picture I don't see any clipping in the blue channel. Can a channel clip without clipping in the histogram?

[a href=\"index.php?act=findpost&pid=66401\"][{POST_SNAPBACK}][/a]

Spib,
I don't think I got it quite right with regard to the following statement ...."In other words, green has become more prominent as a result of the blue channel clipping.".... It seems true that green has become more prominent in relation to blue (and red), causing a cyan shift in parts of the sky, but the reason might not relate to the clipping of the blue channel. There's no doubt that the blue channel has clipped, together with the other 2 channels, because the white clouds are blown, but as you rightly point out, there seems to be no specific clipping of the blue channel in the blue parts of the sky.

I did a linear conversion of an overexposed and correctly exposed RAW image and compared the histograms of the same area of sky where the cyan shift takes place, to get a better idea of what's happening. There seems to be no doubt that a shift to green has taken place in parts of the blue sky in the overexposed image below.

[attachment=597:attachment]

The histograms relate to crops of uncorrected linear images without embedded profile. Of course, now they have an embedded sRGB profile for the web.

Below are the 2 full images converted into the ProPhoto space in ACR with some 'levels' correction to exaggerate the cyan shift which I expected to be less noticeable in an sRGB jpeg for the web.

[attachment=598:attachment]

[bjanes]

My observations on this are:

The sensors approximate to a linear device over their operating range. However, at the limits of exposure (over exposure) the sensor increasingly moves away from being a linear device up until the point that the sensor clips. The issue here is whether the camera manufacturer constrains the measured values (conversion to digital) to the part of the sensor response curve which approximates to linear response, or whether they allow measurements into the top end of the curve where the characteristics of the device moves signficantly away from a linear response.

Second observation is that the conversion from the raw sensor data to the final image (in whatever colour space or file format) has a major impact on the rendition of colour across the tonal range of the image. If the mappings from raw sensor RGB values to final image RGB values is not accurate then you will see colour shifts in the image. A good conversion will accommodate any non linearities of the sensor such that you perceive that the camera has a linear response. A poor quality convertor will assume that the sensor is linear and has imutable characteristics with colour temperature/ exposure value/ etc... and may result in colour shifts - this happened with ACR3.0 earlier and Canon SLR cameras such that in overblown highlights there was/ is a significant magenta shift in the displayed colours.

For the most part, I believe that most raw convertors are now of sufficient complexity and quality that colour shifting is less of a problem and that to a certain extent they are able to correct for clipped channels - though experimentation of how well they achieve this is required.
[a href=\"index.php?act=findpost&pid=66427\"][{POST_SNAPBACK}][/a]

I think that DiaAzul has summarized the situation well. Here are some tests with the Nikon D200 using uncompressed NEFs. The Macbeth color checker is overexposed by about two stops and converted normally with ACR. No color shift in the blown rightmost two neutral squares in Row 4 is present. The preview is shown:

[attachment=604:attachment]


With Exposure Compensation in ACR of -2EV the situation is mostly salavged with no color shifts in the neutral squares as shown:

[attachment=605:attachment]

[Ray]

The Macbeth color checker is overexposed by about two stops and converted normally with ACR. No color shift in the blown rightmost two neutral squares in Row 4 is present.

[a href=\"index.php?act=findpost&pid=66458\"][{POST_SNAPBACK}][/a]

Bjanes,
I can't see any overexposure of the color chart. You seem to have given it the correct exposure from an ETTR perspective. The color shift I'm referring to occurs with real overexposure but in parts of the image that do not appear to be clipped or overexposed.

Below is my overexposed image showing a -2EC correction. The brightest parts are still blown. That's what I mean by overexposure. Do you mean something else?

[attachment=606:attachment]

[bjanes]

Bjanes,
I can't see any overexposure of the color chart. You seem to have given it the correct exposure from an ETTR perspective. The color shift I'm referring to occurs with real overexposure but in parts of the image that do not appear to be clipped or overexposed.

Below is my overexposed image showing a -2EC correction. The brightest parts are still blown. That's what I mean by overexposure. Do you mean something else?

[a href=\"index.php?act=findpost&pid=66464\"][{POST_SNAPBACK}][/a]

Ray,

If you look at the first preview in my post, you will see that R4C2 is blown: it reads 255 in all channels. This is not proper ETTR, but an example of successful highlight recovery in ACR. Your example is more extreme.

[Ray]

Ray,

If you look at the first preview in my post, you will see that R4C2 is blown: it reads 255 in all channels. This is not proper ETTR, but an example of successful highlight recovery in ACR. Your example is more extreme.
[a href=\"index.php?act=findpost&pid=66467\"][{POST_SNAPBACK}][/a]

Bjane,
Well, we should define our terms more carefully before we continue, shouldn't we?

I've always been under the impression that a full ETTR exposure usually involves a negative EC adjustment in a good RAW converter such as ACR. If the histogram shows highlights with a value of 255 in all channels with zero EC adjustment, the image is not necessarily overexposed, but it might be, as in my image. It clearly isn't in the case of your GM chart and it doesn't appear to be in the sunrise images I recently posted in another current thread, "Today's DSLRs Should Have Another Exposure Mode".

Check out those images and tell me what you think.

[bjanes]

Bjane,
Well, we should define are terms more carefully before we continue, shouldn't we?

I've always been under the impression that a full ETTR exposure usually involves a negative EC adjustment in a good RAW converter such as ACR. If the histogram shows highlights with a value of 255 in all channels with zero EC adjustment, the image is not necessarily overexposed, but it might be, as in my image. It clearly isn't in the case of your GM chart and it doesn't appear to be in the sunrise images I recently posted in another current thread, "Today's DSLRs Should Have Another Exposure Mode".

Check out those images and tell me what you think.
[{POST_SNAPBACK}][/a]

Ray,

In my example R4C2 should read less than 255 (in fact about 202 in sRGB). Since R4C1 should read higher than R4C2,  R4C1 is blown, over exposed in my opinion. Your opinion may differ. However, expose to the right means just that, not beyond the right. If you expect to recover 2 stops of highlight in Adobe Camera Raw, you are not being very realistic. Bruce Fraser says 1/4 to one stop in his ACR book. Elsewhere, Bruce has defined overexposure as blowing highlights that you did not intend to blow. In a high dynamic range situation it may be necessary to sacrifice highlights if they are less important than quarter tones or midtones, but normally one does not want to blow highlights.

Here is Michael's original description of ETTR:

[a href=\"http://www.luminous-landscape.com/tutorials/expose-right.shtml]http://www.luminous-landscape.com/tutorial...ose-right.shtml[/url]

"The simple lesson to be learned from this is to bias your exposures so that the histogram is snugged up to the right, but not to the point that the highlights are blown. This can usually be seen by the flashing alert on most camera review screens. Just back off so that the flashing stops."

If exposing to the right places the tones lighter than you want, then negative exposure correction in ACR is called for, but with proper ETTR one does not blow highlights except as discussed above. ACR is adept at highlight recovery, but do not delude yourself into thinking that you have not lost data when you blow highlights.

[bjanes]

Spib,
I don't think I got it quite right with regard to the following statement ...."In other words, green has become more prominent as a result of the blue channel clipping.".... It seems true that green has become more prominent in relation to blue (and red), causing a cyan shift in parts of the sky, but the reason might not relate to the clipping of the blue channel. There's no doubt that the blue channel has clipped, together with the other 2 channels, because the white clouds are blown, but as you rightly point out, there seems to be no specific clipping of the blue channel in the blue parts of the sky.

I did a linear conversion of an overexposed and correctly exposed RAW image and compared the histograms of the same area of sky where the cyan shift takes place, to get a better idea of what's happening. There seems to be no doubt that a shift to green has taken place in parts of the blue sky in the overexposed image below.

[attachment=597:attachment]

The histograms relate to crops of uncorrected linear images without embedded profile. Of course, now they have an embedded sRGB profile for the web.

[a href=\"index.php?act=findpost&pid=66448\"][{POST_SNAPBACK}][/a]

[bjanes]

[attachment=607:attachment]

Spib,
I don't think I got it quite right with regard to the following statement ...."In other words, green has become more prominent as a result of the blue channel clipping.".... It seems true that green has become more prominent in relation to blue (and red), causing a cyan shift in parts of the sky, but the reason might not relate to the clipping of the blue channel. There's no doubt that the blue channel has clipped, together with the other 2 channels, because the white clouds are blown, but as you rightly point out, there seems to be no specific clipping of the blue channel in the blue parts of the sky.

I did a linear conversion of an overexposed and correctly exposed RAW image and compared the histograms of the same area of sky where the cyan shift takes place, to get a better idea of what's happening. There seems to be no doubt that a shift to green has taken place in parts of the blue sky in the overexposed image below.

[attachment=597:attachment]

The histograms relate to crops of uncorrected linear images without embedded profile. Of course, now they have an embedded sRGB profile for the web.

[a href=\"index.php?act=findpost&pid=66448\"][{POST_SNAPBACK}][/a]

Ray,

What do you mean by linear? Gamma = 1?

Rather than a shift in color, I think most of the observed differences in the two exposures are due to differences in luminosity. To demonstrate this, I used the eyedropper readout with LAB. The a and b values of the normally exposed and overexposed images are virtually the same, while the L value is different.

I then converted the overexposed image into LAB and used a levels command to adjust the luminonsity to the same value as in the normally exposed image. As is evident, the colors are now similar. In the LAB readout, one should ignore the RGB reading.

[attachment=607:attachment]

[Ray]

However, expose to the right means just that, not beyond the right. If you expect to recover 2 stops of highlight in Adobe Camera Raw, you are not being very realistic.

Bjanes,
'Not beyond the right' of what? The histogram on the camera's LCD screen? The histogram of the default settings in ACR?

You seem to be missing my point. I've already admitted that the reason for an apparent color shift is not necessaryly directly due to a clipping of the blue channel in the area where the color shift takes place as I first thought, but is possibly a side effect of the way the converter attempts to recover highlights. It's an observation I've made and demonstrated and it's an effect that comes into play as a result of overexposure, and by overexposure I mean exposing beyond the point where highlights can be recovered.

There's no point in telling me I'm unrealistic in expecting to recover over 2 stops of highlight detail. I know that. I've converted thousands of images in my time. Some converters do a better job than others. The image demonstrating this color shift in this thread was deliberately overexposed. I have other images which were unintentionally overexposed.

Exposing to the right often involves a choice of deliberately overexposing an unimportant part of the composition for the sake of better tonality elsewhere in the image, and the part that is deliberately overexposed is often the sky.

"The simple lesson to be learned from this is to bias your exposures so that the histogram is snugged up to the right, but not to the point that the highlights are blown. This can usually be seen by the flashing alert on most camera review screens. Just back off so that the flashing stops."

Michael also mentions in his article that an image correctly exposed to the right will often appear too light (as your GM color chart is). In fact, I would say that any low dynamic range scene (such as you GM color chart) will appear too light with ETTR.

The practice of using the camera's highlight warning as a guide to overexposure is a conservative one. If there's no flashing alert on the LCD then it's unlikely you will blow any highlights other than  highlights of a specral nature. But as we both know, the histogram is from a jpeg conversion and its appearance can vary with camera contrast settings and possibly between different models of camera and different designs of metering systems.

[bjanes]

Bjanes,
'Not beyond the right' of what? The histogram on the camera's LCD screen? The histogram of the default settings in ACR?

[a href=\"index.php?act=findpost&pid=66509\"][{POST_SNAPBACK}][/a]

That is where testing comes in. The raw histogram is the key. Any important highlight should be no higher than 4095 in the raw file, or else it will be clipped. Thus far, I have seen no evidence that you have looked at any raw histograms. You need to know how your camera histogram and blinking highlights relate to the raw values.

You seem to be missing my point. I've already admitted that the reason for an apparent color shift is not necessaryly directly due to a clipping of the blue channel in the area where the color shift takes place as I first thought, but is possibly a side effect of the way the converter attempts to recover highlights. It's an observation I've made and demonstrated and it's an effect that comes into play as a result of overexposure, and by overexposure I mean exposing beyond the point where highlights can be recovered.

[a href=\"index.php?act=findpost&pid=66509\"][{POST_SNAPBACK}][/a]

What color shift? If I am correct, all you are seeing is a change in luminance, which you have misinterpreted as a color shift. Naturally, increased exposure time increases the luminance.  How do you expalin that the a and b values in LAB are the same?

Exposing to the right often involves a choice of deliberately overexposing an unimportant part of the composition for the sake of better tonality elsewhere in the image, and the part that is deliberately overexposed is often the sky.
Michael also mentions in his article that an image correctly exposed to the right will often appear too light (as your GM color chart is). In fact, I would say that any low dynamic range scene (such as you GM color chart) will appear too light with ETTR.

[a href=\"index.php?act=findpost&pid=66509\"][{POST_SNAPBACK}][/a]

With a high dynamic range subject, one might have to accept highlight clipping, but with the GM chart, a short scale subject, there is no reason to clip the highlights, since the camera can easily record the entire DR of the subject. In this case, highlight clipping is overexposure. I would agree that a low dynamic range subject exposed to the right may well look too light and negative exposure compensation may be needed, but highlight recovery should not be needed. R4C1 of the GM checker has an OD of 0.05, and with a gamma 2.2 space, normal exposure places it at pixel value 242, leaving a bit of headroom for specular highlights. If you place the square at 255, all of the tonal values will be too light, but a bit of negative exposure compensation in ACR will  correct things and will probably recover the specular highlights. But why take that risk?

[Ray]

What color shift? If I am correct, all you are seeing is a change in luminance, which you have misinterpreted as a color shift. Naturally, increased exposure time increases the luminance.  How do you expalin that the a and b values in LAB are the same?
[a href=\"index.php?act=findpost&pid=66519\"][{POST_SNAPBACK}][/a]

Bill,
I've done a number of linear conversion, in BreezeBrowser, of this overexposed image with the cyan shift and run the eyedropper over the problem areas to check the values.  It now appears that ACR is attempting to recreate the clipped blue channel but is not quite succeeding.

Below are two crops of the problem area. The linear conversion shows equal values in all 3 channels in the sky area between the clouds and the mountain and immediately above the clouds.

Interestingly, the linear conversion shows no more detail in the clouds than ACR at -4 EC.

[attachment=610:attachment]

[Ray]

That is where testing comes in. The raw histogram is the key. Any important highlight should be no higher than 4095 in the raw file, or else it will be clipped. Thus far, I have seen no evidence that you have looked at any raw histograms. You need to know how your camera histogram and blinking highlights relate to the raw values.
[a href=\"index.php?act=findpost&pid=66519\"][{POST_SNAPBACK}][/a]

Blinking highlights might well relate to the RAW values, but not directly as far as I can tell. The appearance of the camera's histogram is affected by the contrast settings on the camera's menu and the blinking highlight warning is also affected by those internal settings.

The difference between a minimum and maximum contrast setting amounts to about 1 and a 3rd stops as regards estimating an exposure to the right from the degree of highlight warning, on the 5D at least. In other words, if I back off from an exposure that causes highlight blinking to the point where there is no highlight flashing, the accuracy of my exposure with regard to ETTR will depend on the contrast setting of the picture mode the camera is set on, and the variation is greater than a whole stop.

That's more significant than the magenta filter trick   .

[John Sheehy]

That's more significant than the magenta filter trick   .
[a href=\"index.php?act=findpost&pid=66628\"][{POST_SNAPBACK}][/a]

They're not mutually exclusive!

I came up with the idea of the magenta filter after I was already exposing the RAW data all the way to the right.

[bjanes]

Blinking highlights might well relate to the RAW values, but not directly as far as I can tell. The appearance of the camera's histogram is affected by the contrast settings on the camera's menu and the blinking highlight warning is also affected by those internal settings.

The difference between a minimum and maximum contrast setting amounts to about 1 and a 3rd stops as regards estimating an exposure to the right from the degree of highlight warning, on the 5D at least. In other words, if I back off from an exposure that causes highlight blinking to the point where there is no highlight flashing, the accuracy of my exposure with regard to ETTR will depend on the contrast setting of the picture mode the camera is set on, and the variation is greater than a whole stop.

That's more significant than the magenta filter trick   .
[a href=\"index.php?act=findpost&pid=66628\"][{POST_SNAPBACK}][/a]

If you want to map the histogram and blinking highlights to the contents of the raw file, a tone curve is a much more precise tool than the contrast setting.

Exposure to the right and the magenta filters are useful tools to use the full well capacity of the sensor and equalize the channels respectively, but often the ISO setting is more important than ETTR or use of a magenta filter for daylight shooting.

At base ISO, let's assume that it is 100, the highlights should be at around 4000 in the 12 bit linear file and the sensor should be full well or nearly so. If you use ISO 400, the sensor is only at 25% of full well, and the gain is altered so that the data numbers are where they should be. In terms of ISO 100 exposure, that is two stops under--more significant than ETTR at ISO 400 and use of a magenta filter.

Of course, as John has pointed out, these options are not mutually exclusive.