Luminous Landscape Forum
Raw & Post Processing, Printing => Digital Image Processing => Topic started by: AFairley on December 09, 2010, 12:36:33 pm
-
I am aware of "shooting to the right" and the thinking behind it. But I did run into a situation where the main subject matter was dark to dark brown tones, and my exposure was way to the right (but not clipped). Naturally, in ACR the image looked quite washed out as shot. No matter what I tried in ACR (the current version) to the RAW file I just could not get same look I believe I would have gotten from a normally exposed RAW. It was like the tones were just dead when I brought them down to where they should be; fiddling with all kinds of combinations of tone sliders, constrast, vibrance, saturation, etc. just could not get me to where I knew the shot would look if I had exposed "normally" (i.e., a la modified zone system, metering off the target and adjusting exposure to place it where I wanted on the tone scale). I'm no ACR guru, but I've gotten pretty good at knowing how to get to the look I want with adustments in ACR. Not this time.
So I wonder if you can expose too far to the right for a given subject even though you're not cllpping. Thoughts?
-
Interesting observation and I had a similar "hunch" that got further solidified when reading this thread on getDPI (http://forum.getdpi.com/forum/showthread.php?t=7885&highlight=hardloaf).
Even though it's not the main subject being discussed what I learned is that the problem with high DR sensors (especially increased highlight headroom) ETTR can lead to putting your whole picture in a narrower gamut at the bright side of the histogram, and then by bringing the EV down in the raw converter you can't get all you would have gotten at a lower exposure.
Since I value colour in a picture much more than the absence of shadow noise I've stopped using ETTR, and am more exposing such that my pictures are "correctly" exposed when using the default raw conversion and am much happier with the colours I'm getting from this tactic.
-
So I wonder if you can expose too far to the right for a given subject even though you're not cllpping. Thoughts?
Unquote
Yes I think you have a good point. This is where subjectivity comes into play. I have in the past used the Uni WB, but I don't now. However it helps me judge my exposure. The Uni WB in my experience was that I could get about an extra 2/3 to 1 stop more exposure. In ACR I found that though the highlights weren't technically blown out lowering the exposure slider didn't give me a pleasing result. Using a Nikon camera if I get slight blinking on the camera highlight warning then I am happy with the results though I know there is nearly a stop more. This is using raw and I know about the jpeg rendering of the image to give me a histogram reading. Experience will guide you and I think you are on the right track. :)
-
ETTR can lead to putting your whole picture in a narrower gamut at the bright side of the histogram
I would have thought that this problem of reduced gamut at max luminance really does mean that you're clipping a channel on that high-luminance zone.
In the link you mention, the phrase (message #4 from hardloaf) :
This means that brightest stop of the camera range has most of colors gone forever and they cannot be restored with negative exposure compensation.
makes me think of the old mistake of not taking into account the 3 color channels for ETTR (discussion just goes on about exposing grays), or much more probably the old chore of not having a raw histogram to judge it.
That said, it's just an hypothesis and I'd like to hear more on the subject.
So I wonder if you can expose too far to the right for a given subject even though you're not cllpping. Thoughts?
That might indicate some non-linearity in the upper part of the histogram...
I've heard some claims about it with some cameras (including IIRC the 10D), but couldn't get evidence of something like that with my good'olRebel (almost identical sensor as the 10D) : with a set of upwards bracketed exposures, set to "match total exposures" within LR, colors are spot on up to the point where they clip (LR clipping warning, spike in the histogram).
Though, there is a gray zone where I see the clear effects of clipping, and LR displays the clipping warning in the histogram, but no on-screen clipping overlay is shown. But it's really something like a third stop or so, much less than the error given by evaluating exposure with a jpeg-based histogram.
Of course, for saturated colors the clipping arises before high luminances, hence the traditional problem to render a sunrise/sunset on print : it's difficult to get both the proper saturation and the proper brightness.
-
Interesting observation and I had a similar "hunch" that got further solidified when reading this thread on getDPI (http://forum.getdpi.com/forum/showthread.php?t=7885&highlight=hardloaf).
Even though it's not the main subject being discussed what I learned is that the problem with high DR sensors (especially increased highlight headroom) ETTR can lead to putting your whole picture in a narrower gamut at the bright side of the histogram, and then by bringing the EV down in the raw converter you can't get all you would have gotten at a lower exposure.
Hi Pieter,
Don't believe everything you read ;) .
At capture time, there is no color space (with its particular gamut hull) defined. The mostly linear sensitivity of our sensor arrays just records photons converted to electrons in R/G/B filtered sampling positons. Then a Raw converter needs to convert that data into RGB colors at each sample position (=demosaicing), and a number of calculations is performed before mapping those RGB coordinates to some RGB coordinate system. One of the operations, after demosaicing, is a linear scaling of the RGB channels to perform White Balancing and Exposure correction. Both should be done in linear gamma space.
Only after all these operations is the data set mapped to a color space (a coordinate system), that is why we effectively get a wider gamut by e.g. choosing Adobe RGB instead of sRGB, and ProPhoto RGB can squeeze a bit more out of our files (although most coordinates in the PP RGB are left unused in a real image).
So it depends on the Raw converter if it scales the ETTR exposure down to the proper/required level before mapping it to a colorspace. Only then, with a colorspace assigned, will the change of exposure potentially impact the saturation due to the remapping model in the given colorspace coordinate system.
Therefore the answer to the OPs question can only be given for a specific Raw converter, because we have no guarantee that the processing is done for optimal quality rather than speed. In fact, quality probably requires to do most of the processing in floating point math, which is slower than integer math. I saw some Raw conversion samples by Iliah Borg, indicating that (while slower) floating point math during Raw conversion indeed produces superior results compared to the current LibRaw/DCRaw conversions.
The best proof is therefore to shoot a scene (or a ColorChecker card) at different exposure levels, making sure that there is no single channel clipping (!), and pulling the exposure at Raw conversion to a common brightness level. Then a comparison of color differences will reveal if there is a problem with the Raw processing engine or not.
Cheers,
Bart
-
I am aware of "shooting to the right" and the thinking behind it. But I did run into a situation where the main subject matter was dark to dark brown tones, and my exposure was way to the right (but not clipped). Naturally, in ACR the image looked quite washed out as shot. No matter what I tried in ACR (the current version) to the RAW file I just could not get same look I believe I would have gotten from a normally exposed RAW. It was like the tones were just dead when I brought them down to where they should be; fiddling with all kinds of combinations of tone sliders, constrast, vibrance, saturation, etc. just could not get me to where I knew the shot would look if I had exposed "normally" (i.e., a la modified zone system, metering off the target and adjusting exposure to place it where I wanted on the tone scale). I'm no ACR guru, but I've gotten pretty good at knowing how to get to the look I want with adustments in ACR. Not this time.
So I wonder if you can expose too far to the right for a given subject even though you're not cllpping. Thoughts?
Yes, you can - with pretty much any modern raw processor you get color shifts, exactly as you found. If you want all the gory detail, I went through the technical reasons why in a blog post here (http://chromasoft.blogspot.com/2009/09/why-expose-to-right-is-just-plain-wrong.html) some time ago.
Sandy
-
Yes, you can - with pretty much any modern raw processor you get color shifts, exactly as you found.
The hue shift you expose in your post is interesting, but... would you expect to detect it in a real-world image? I couldn't see any of these shifts with my test.
Btw, are you also sure that no channel clipped in the green color you show?
And yes, ETTR is of no use past base ISO and is to be applied to that specific situation (in which I personally am 95% of time). ;)
To rephrase ETTR, it is a simple way to adapt the exposure thinking from film (exposing middle grays at the middle between the shoulder and the toe of the emulsion) to digital (no shoulder but a progressive toe and saturation).
So it's not very different to simply taking care on what is clipped (generally quite bad), and what is drowned in the shadows (generally not as bad as clipped but to be avoided).
For low-contrast subjects I fully agree that ETTR does not bring much (as long as you don't clip anything important).
-
The hue shift you expose in your post is interesting, but... would you expect to detect it in a real-world image? I couldn't see any of these shifts with my test.
Btw, are you also sure that no channel clipped in the green color you show?
Oh yes. Although usually you only see the issue if you aggressively post process. E.g., If you use a lot of "recovery" in Lightroom while using the newer camera profiles. There are a bunch of threads in the various Adobe forums on this, and the issue led to dcpTool getting its ability to make profile "invariate" or "untwisted", etc
And no, the green channel wasn't blown.
Sandy
-
Recovery in LR and ACR need some work... If you clip one of the three channels and Recovery comes into the scene, I’ve seen color shifts (interestingly enough, Raw Developer does a much cleaner job). So yes, use ETTR but do not blow out any channel data that needs to be reconstructed. That can lead to ugly color results:
http://digitaldog.net/files/RD.jpg
-
If you use a lot of "recovery" in Lightroom [...]
Oh, I'd think hue shifts with Recovery are another, different issue... I've seen even much more than what Andrew shows here.
-
Simply put, in my experience in deciding on exposures, as long as I have good separation in my shadows, I get better results with a correct exposure as seen, rather than moving my whole histogram far to the right when I don't need to. I shoot with a 65 plus phase and a Leica m9. Eleanor
Interesting observation and I had a similar "hunch" that got further solidified when reading this thread on getDPI (http://forum.getdpi.com/forum/showthread.php?t=7885&highlight=hardloaf).
Even though it's not the main subject being discussed what I learned is that the problem with high DR sensors (especially increased highlight headroom) ETTR can lead to putting your whole picture in a narrower gamut at the bright side of the histogram, and then by bringing the EV down in the raw converter you can't get all you would have gotten at a lower exposure.
Since I value colour in a picture much more than the absence of shadow noise I've stopped using ETTR, and am more exposing such that my pictures are "correctly" exposed when using the default raw conversion and am much happier with the colours I'm getting from this tactic.
-
As Bart explained, the limitations are in the current Raw converters. They are simply not expecting or designed for ETTR. From a signal processing point of view, there is absolutely no reason why an ETTR image will have a compromised color rendition (as long as there is no clipping or saturation in any of the channels)
I guess that the next generation of Raw converters will treat exposure in a different way, since cameras are getting better (where noise is determined by shot noise), so ISO could become metadata. For this you need to be able to adjust exposure more than the +/- 4 EV allowed by some current RAW converters, and it should be done before demosaicing (the same way white balance is treated today, just a scalar multiplication of the values)
-
Simply put, in my experience in deciding on exposures, as long as I have good separation in my shadows, I get better results with a correct exposure as seen, rather than moving my whole histogram far to the right when I don't need to.
+1 ;)
-
Recovery in LR and ACR need some work... If you clip one of the three channels and Recovery comes into the scene, I’ve seen color shifts (interestingly enough, Raw Developer does a much cleaner job). So yes, use ETTR but do not blow out any channel data that needs to be reconstructed. That can lead to ugly color results:
http://digitaldog.net/files/RD.jpg
Of course, recovery and exposure in ACR are entirely separate entities. Recovery is for recovering clipped highlight tones or toning down the highlights. Its effects are constrained to the highlight tones. Exposure is global and is the tool that should be used for ETTR images that are not clipped. As SandyMC and others have noted it does have some limitations in terms of hue-shifts and other problems.
Regards,
Bill
-
Isn't the very concept behind ETTR flawed? It tries to mash a non-linear construct (the doubling or halving of light and effective 'stops') into a linear system. While more nominal data (electrons) will be captured in highlights than shadows, it's not the exponential increase as posited by many; including the article on the main LL site. Is it? In a linear system (digital sensors) if it takes 1000 electrons to move from one brightness level to another in the shadows then it takes 1000 electrons to move from one brightness level to another in the highlights. The idea that exponentially more data is contained in highlights than shadows runs counter to the idea of the linear nature of digital sensors. Doesn't it?
-
The concept behind ETTR is not flawed and it has nothing to do with the linear response of the sensor. The idea is that shoot noise is proportional to the square root of total photon count. So the higher the photon count, the higher the Signal to noise ratio.
The idea is to "fill" the sensor with as many photons as possible to achieve the highest SNR possible.
The problem with the linear response of the sensor vs the nonlinear nature of perception of light is that ETTR poses a high risk of blowing the highlights.
-
Thank you for your replies. I do want to emphasize that none of the color channels were clipped when I opened the RAW file in ACR 6.2, so I did not need to do any highlight recovery, though I obviously wanted to bring the overexposed high tones down.
-
Doesn't Sandy's article basically refute that point, Francisco as far as the noise levels?
And if the concept behind ETTR isn't flawed (and I'm not convinced given evidence of it I've seen) then perhaps it's the communication of the concept that's flawed.
-
In a linear system (digital sensors) if it takes 1000 electrons to move from one brightness level to another in the shadows then it takes 1000 electrons to move from one brightness level to another in the highlights.
The sensor is linear, wheras our perception of brightness (and the exposure count by stops based on it) is logarithmic. So the brightness levels you quote are not equal to our eye and so not equal in the rendered print.
-
Doesn't Sandy's article basically refute that point, Francisco as far as the noise levels?
And if the concept behind ETTR isn't flawed (and I'm not convinced given evidence of it I've seen) then perhaps it's the communication of the concept that's flawed.
Well, I may interpret it the wrong way (since English is not my first language), but I don't see contradictions regarding noise. The point in Sandy's article accepts there is a reduction of Noise at the expense of degraded color rendition and reduced DR. I claim that the problem with degraded color is due to the way the current Raw converters work and not because of the data itself.
Another point in Sandy's article is that ETTR only deals with sensor noise but creates other problems in the rest of the chain. I completely agree here, with the same comment as before: Because how the rest of the components / softwares are designed today.
ETTR only makes sense at base ISO - Yes. I think this will eventually be settled in the future when ISO will become metadata.
ETTR is not necessary today with modern cameras - I agree up to a point. For example, under incandescent light (2000K) if you expose normally, you will get a rather noisy and underexposed blue channel. ETTR or even using a color compensating filter will improve that.
I do agree that the communication of the concept may be flawed.
-
I'll try to give a numerical example of why ETTR is not a flawed concept,
For simplicity, let's suppose that the possible values for the RGB channels go from 0 to 100. This are the linear values as output from the A/D converter.
The raw data for any given image will be values from 0 to 100 in any channel.
Suppose that as a result of a given exposure we have the following ranges:
R= from 1 to 35
G= from 2 to 48
B= from 1 to 25
If I input those values to a Raw converter, I will get a result, which I'll call result A
Now, let's just multiply all values by 2, we get:
R= from 2 to 70
G= from 4 to 96
B= from 2 to 50
After input this values to the same Raw converter, I will get result B
Now, if we consider both sets of data coming from a black box, we could say that the difference between both is 1EV or 1 stop
How to get output A from the second set of data? Two choices:
1) Divide by 2 all values (which will result in the exact first set of data)
2) Reduce exposure by 1EV in the Raw converter
In 1) you will get the exact result A, no question about it.
In 2) you may or may not get the same result than A, since it depends on how and when exposure correction takes place in the Raw converter
Now, going back to the black box, if we didn't know that the second set of data is just the first set multiplied by 2, we could suppose:
a) The difference is due to a 1EV exposure (time or aperture)
b) The difference is due to different ISO (double)
c) The difference is due to ETTR exposure (note that now the range of G = 4 to 96, covering almost all possible values)
Up to this point, Is there anything on the data that will compromise color rendition? NO, nothing.
If we knew the exact proportion of the ETTR exposure related to the "correct" exposure and divide all raw values by that factor, then we will have as a result the same raw values as if we used the "correct" exposure, but with less noise.
So let me change the conditions of the example: The second set of data is a the result of ETTR exposure which correspond to twice the correct exposure, then just divide the second set by 2 and you get the first one.
-
The linearity of a sensor is an assumption in ETTR. Please see the following post on this topic:
http://forums.dpreview.com/forums/read.asp?forum=1018&message=37133825
and Iliah Borg's response that includes a link that shows some non-linearity of a sensor:
http://forums.dpreview.com/forums/read.asp?forum=1018&message=37136263
Joofa
-
a link that shows some non-linearity of a sensor:
Let's quote it :
http://www.maxmax.com/nikon_d700_study.htm
The linearity is not perfect in these curves... And it would be very interesting to know from what that comes.
NotNote (sorry for the typo :-[), of course that the big non-linearity in the red channel comes from clipping.
However, the behaviour just before saturation is strongly non-linear : one point made.
-
But there is one situation where ETTR can help - when you're already at the lowest ISO setting you camera offers.
Sorry but I have to say “duh”. We all know raising ISO produces more noise and often we need that trade off. And by using the base ISO and implementing ETTR, Sandy shows and agrees the result is less noise because at this ISO, using proper exposure for raw results in less noise in the shadows. If that’s something you desire, you should use it.
We analog shooters know from experience we had to test or film and processing to determine the true (optimal) ISO. We also knew that if we pushed our film (because sometimes we had too), the result was kind of the equivalent today of raising the ISO, more “noise”. Sometimes it just had to be done (push processing beat an under exposed chrome by a mile and we lived with the trade off).
While the G10 has a ISO 100 setting it doesn't have a ISO 50 setting. (It does a ISO 80 setting, but I'm ignoring that as it's too close to 100 to make much of a difference.) So what ETTR is doing here is allowing us to synthesize a lower ISO setting, and hence a better noise performance, than the camera actually has. The disadvantage of course is that the camera's dynamic range is reduced by one stop, but if you have a low contrast image, that might be a price worth paying.
What makes this affect the dynamic range?
And yes, the entire idea is better noise performance (and we could say, the optimal, true (?) ISO).
-
Thanks for those links Joofa, very interesting information.
Absolutely right, if the response is not linear, ETTR just don't make any sense
-
I think this brief article (http://schewephoto.com/ETTR/index.html) explains it well. The recovery of detail in the Niagara Falls shot is noteworthy.
-
Sorry but I have to say “duh”. We all know raising ISO produces more noise and often we need that trade off. And by using the base ISO and implementing ETTR, Sandy shows and agrees the result is less noise because at this ISO, using proper exposure for raw results in less noise in the shadows. If that’s something you desire, you should use it.
Andrew, yes, agree, it is "duh" - to you, me, and probably (maybe?) many of the readers of this particular forum. But it sure isn't at all clear to a huge number of ETTR proponents.....
Sandy
-
Yes, Francisco, that's what Sandy's article shows but it also shows that there's really no difference in noise by using a normal exposure at a lower ISO setting. Leaving us with the idea, as he outlined, that the only situation where ETTR may be useful is when you're already at base ISO and can't go any lower.
Joofa, yes and I corrected myself by saying the communication of ETTR in so far as there being more data in highlights in non-nominal terms was flawed. Sorry for the confusion.
Anti-blooming circuitry on 'modern' sensors will cause a slight non-linearity in the response at the top of the curve (line).
-
Non-linearity is a very strong point against ETTR, so strong as to relegate it to a theoretical exercise
it also shows that there's really no difference in noise by using a normal exposure at a lower ISO setting
The difference in noise would be more apparent in the deep shadows or in one color channel if the light source is very far from white (like a low power incandescent light, where a normal exposure will result in a very underexposed blue channel)
-
Sorry but I have to say “duh”. We all know raising ISO produces more noise and often we need that trade off. And by using the base ISO and implementing ETTR, Sandy shows and agrees the result is less noise because at this ISO, using proper exposure for raw results in less noise in the shadows. If that’s something you desire, you should use it.
I have to quibble with the wording here. Raising the ISO does not 'produce more noise' (ie lower the SNR); rather, lowering the exposure lowers the SNR ('produces more noise'). Only in an indirect sense does raising the ISO result in more noise -- that clipping in the raw data occurs at a lower absolute exposure, so if the ISO is raised one may have to lower the absolute exposure to prevent clipping.
It is a fact that in most modern CMOS sensored cameras, raising the ISO actually lowers the noise relative to absolute exposure -- any noise introduced after the ISO amplification is smaller relative to signal when the amplification is raised. This can be quite significant in Canon DSLRs and the Nikon D3/D3s/D700 at low ISO, and is the major reason why there is a benefit to raising the ISO in low light in raw shooting.
-
I think the point Andrew was making was that nobody (that I know of) recommends using ETTR at higher ISOs. The whole point of raising ISO is to get a faster shutter speed, so why would you then go an increase your exposure time just to ETTR?
-
I think the point Andrew was making was that nobody (that I know of) recommends using ETTR at higher ISOs. The whole point of raising ISO is to get a faster shutter speed, so why would you then go an increase your exposure time just to ETTR?
Because you need a certain exposure time?
-
Looking at the OP and assuming that the object of the exercise is to get an image that resembles the subject matter, the author says that his subject was mainly dark browns. Now if I am photographing a black cat in coal bunker, personally I would under-expose by one to two stops. In my humble experience with digital (and film), this would give me the final result desired. Using ETTR would just not be logical - surely? I can understand the concept for more typical scenes where very light tones are present however.
Jim
-
Now if I am photographing a black cat in coal bunker, personally I would under-expose by one to two stops. In my humble experience with digital (and film), this would give me the final result desired. Using ETTR would just not be logical - surely?
Jim
I guess you are refering to a reflective metering of the scene. If you use a graycard or incident metering, then you would not underexpose. This would give you the desired result if you want a final image straight out of the camera. ETTR capture will require postprocessing to get the desired results.
Suppose you don't underexpose from a reflective metering, the result will be a light gray cat on a light gray background instead of the dark grays you are expecting. To be able to obtain the desired result, you will have to compensate exposure in the Raw converter by -2 EV.
An ETTR approach in your example could mean overexposing the reflected measure by 1 or 2 stops (as long as you don't blow any channel) and then compensate exposure by -3 or -4 EV in the Raw processing.
As it was mentioned before, this would work only if the response of the sensor + A/D converter were perfectly linear, so in practice this cannot be guaranteed.
If you think about film, something analog to ETTR was actually performed regularly between negative film users. It was common to overexpose color negative film by 1/3 or 2/3 EV and then compensate in the darkroom. This was due to the characteristics of color negative film: difficult to blow highlights and quality degrades quickly in the shadows.
This of course could not be applied to positive film. you had to use correct exposure to get the desired results.
-
Francisco
Yes, of course I was referring to a reflected reading, which is how I now work most of the time, using exposure compensation and experience to get a decent result. In the past with film I did use an incident meter for almost everything and I can see that if I was to use the same method now it might well make sense to play safe and ETTR to some extent. I guess with the sort of shooting I now do it is quicker (for me) to rely on the DSLR meter and just use compensation as needed.
Jim
-
I think the point Andrew was making was that nobody (that I know of) recommends using ETTR at higher ISOs. The whole point of raising ISO is to get a faster shutter speed, so why would you then go an increase your exposure time just to ETTR?
No, as far as the raw data quality is concerned, the point of raising ISO is to get a lower read noise (better SNR) for a given shutter speed and aperture, provided one is not clipping any highlights one doesn't want clipped. For those cameras whose DxO DR curve is not a straight line but flattens at low ISO, if the exposure fits within the higher ISO DR then you are better off using it.
One way to think of it is as follows -- here is the SNR plot of my 1D3 for various ISO, but instead of always placing raw saturation at the same point, let's align them according to absolute exposure:
(http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/DRwindow1d3.png)
When the ISO is doubled, one more stop or EV of input is pushed past the saturation point of the ADC and is lost; thus each subsequent one stop increase of ISO removes one stop of highlight exposure. At the low end is the noise "floor", really the point at which one chooses to quit using the sensor data because the signal/noise ratio becomes too poor. This floor varies with ISO, and thus how much room on the noise floor is available depends on ISO. From the figure we see that, in normalized exposure terms, the floor is lowered with increasing ISO.
So if one has chosen a particular exposure, set by one's needs for DOF and subject motion, one then wants to select an ISO that fits the scene's brightness range so that not too much is clipped on the right, and as much SNR as possible is available on the left. The camera's meter suggests an exposure for a particular ISO, but if one can get away with raising the ISO, esp at low ISO, then more shadow SNR is available and if the scene brightness range is narrow enough that the ISO chosen is such that the meter is indicating too high an exposure, that can be considered ETTR.
Of course, better capture SNR always involves increasing the exposure as much as possible to get more photons, and lowering the ISO if needed to preserve highlights. Finally, there are cameras that don't have this flattening of the DR at low ISO (in particular, those using Sony EXMOR sensors) for which the benefit to raising the in-camera ISO is much, much less; for them, there is little or no point to ETTR, or for that matter raising the ISO at all much over base ISO.
-
if sensors were nonlinear, hdr would be one application where it could easily be detected?
-h
-
So it depends on the Raw converter if it scales the ETTR exposure down to the proper/required level before mapping it to a colorspace. Only then, with a colorspace assigned, will the change of exposure potentially impact the saturation due to the remapping model in the given colorspace coordinate system.
Bart, I see no reason for this as long as the colourspace mapping is linearly done (as in DCRAW and I guess most RAW developers) with a typical 3x3 matrix operation. So there is no reason to expect any saturation/hue change because of exposure adjustments done afterwards.
The effects that can lead to this are different from the colourspace conversion: white balance if not applied linearly (in many RAW developers WB is again a linear operation), and specially any contrast/bright curve applied that delinearized the image, or any other non-linear processing done to the image.
But if you just perform a basic RAW development: linear white balance, demosaicing, and output colourspace linear conversion, there should be no difference in adjusting exposure before of after this RAW development. After all is all about scaling the RGB values by the same linear multiplying factor (the non-standard gamma curve in sRGB would need specific adjustments to accomplish this operation properly though. In standard gamma spaces just a linear factor should model a correct exposure correction).
Regarding the main question of this thread, I found my 350D linear in all three channels beyond what the eye can distinguish as long as no RAW channel gets clipped. Once some channel clips, it can affect linearity on the remaining channels, because of the sensor readout electronics I guess.
(http://www.guillermoluijk.com/article/anavsdig/curvaloglog350d.gif)
However in a properly ETTR'ed RAW file there should be no clipping in the RAW file, so this shouldn't be a problem.
These two images have the same tones, just different noise because they were differently exposed:
(http://www.guillermoluijk.com/article/ettr3/ruido.jpg)
(http://www.guillermoluijk.com/article/ettr3/ruido2.jpg)
Regards
-
At capture time, there is no color space (with its particular gamut hull) defined. ...Only after all these operations is the data set mapped to a color space (a coordinate system),
A little OT to the OP but there is a notion of color space at the capture time, and there is a defined relationship between the standard color spaces (which you refer to those having a 'corordinate system'). Please see the image below to see the geometry of the setup:
(http://djjoofa.com/data/images/metameric_consistency.jpg)
For more information, please see:
http://forums.dpreview.com/forums/read.asp?forum=1018&message=36956797 (http://forums.dpreview.com/forums/read.asp?forum=1018&message=36956797)
Joofa
-
Bart, I see no reason for this as long as the colourspace mapping is linearly done (as in DCRAW and I guess most RAW developers) with a typical 3x3 matrix operation. So there is no reason to expect any saturation/hue change because of exposure adjustments done afterwards.
Yes, completely sound theory.
But the point is, the mapping is not linearly done anymore - three years ago, conversion was linear for most raw converters. It's not anymore - LR and ACR have been non-linear ever since the second generation camera profile with 3D HueSatMap and Look tables that enabled "hue twists" came in. C1 don't publicly document their pipeline, but almost certainly they're the same.
Sandy
-
Yes, completely sound theory.
But the point is, the mapping is not linearly done anymore - three years ago, conversion was linear for most raw converters. It's not anymore - LR and ACR have been non-linear ever since the second generation camera profile with 3D HueSatMap and Look tables that enabled "hue twists" came in. C1 don't publicly document their pipeline, but almost certainly they're the same.
Sandy
If camera response is linear, what is the point of nonlinear color correction in raw converters?
-h
-
If camera response is linear, what is the point of nonlinear color correction in raw converters?
I'd think it's there to mimic a perceptual bias of our vision...
BTW, some did also find such a (color) non-linearity in the processing of images taken with Highlight priority on with Canon DPP (see this discussion in French (http://www.chassimages.com/forum/index.php?topic=100806.0)).
-
The camera sensor and ADC converter are almost linear (small non linearties are unavoidables).
But your vision is logarithmic, that's why gamma correction exist !
And yes in the low lights and high lights, there are tint shift, and gamut variation.
So, converting RAW data to a jpg file is not an easy task !!
Niko, article in French is very interresting.
Thierry
-
The camera sensor and ADC converter are almost linear (small non linearties are unavoidables).
But your vision is logarithmic, that's why gamma correction exist !
That is a common misconception, but accurate reproduction of a scene requires a 1:1 relationship between input and output. The application of gamma allows recording and processing in a perceptually uniform way and requires fewer bits than linear encoding. However, the gamma encoding is reversed for viewing so that the output is linear. Gamma correction is somewhat of a misnomer--one should use the term gamma encoding.
This illustration from the Gamma FAQ (http://www.poynton.com/notes/colour_and_gamma/GammaFAQ.html#smoothly_shade) makes the point, but it is geared to CRTs which automatically perform the inverse gamma function.
-
Sensors are (mostly) linear, but human vision and subjective preferences are not.
-
That is a common misconception, but accurate reproduction of a scene requires a 1:1 relationship between input and output. The application of gamma allows recording and processing in a perceptually uniform way and requires fewer bits than linear encoding. However, the gamma encoding is reversed for viewing so that the output is linear. Gamma correction is somewhat of a misnomer--one should use the term gamma encoding.
I do believe that the end-to-end gamma is usually different from 1:1. Supposedly this compression simply looks good on low-dynamic range displays.
-h
-
The camera sensor and ADC converter are almost linear (small non linearties are unavoidables).
But your vision is logarithmic, that's why gamma correction exist !
And yes in the low lights and high lights, there are tint shift, and gamut variation.
So, converting RAW data to a jpg file is not an easy task !!
Gamma exists partly due to the native response of CRT displays, and partly because it maps noise/distortion in a perceptually uniform manner.
And I dont see the relevance to my question. I was wondering why one would do nonlinear complex stuff in the color-correction part where some native camera response is transformed into some reference color representation (e.g. XYZ). Gamma, jpeg, etc is done later.
-h
-
Sensors are (mostly) linear, but human vision and subjective preferences are not.
And nearly all our output devices (display and print).
-
Gamma exists partly due to the native response of CRT displays, and partly because it maps noise/distortion in a perceptually uniform manner.
There is absolutely no need of gamma per se applied to digital images. The correct mapping of the RGB values so that they display right can be done in software, in the output device (monitor plus video card), or both working together.
Gamma in digital images is only necessary to avoid posterization in the shadows if they are encoded with integer values (for instance 16-bit TIFF files and specially 8-bit JPEG files). But an image editor working with floating point numbers doesn't need gamma at all; images can be linear and deep shadows would be represented as richly as the highlights. It's a matter of encoding efficiency.
These two images display correctly thanks to the software (Photoshop is interpreting the 2.2 gamma of the image on the left, and the 1.0 gamma of the image on the right), which is sending to the output video devices the appropiate values for correct rendering:
(http://img820.imageshack.us/img820/6038/gamma.jpg)
However the RGB values are totally different, just look at the histograms.
Gamma exists because of the original output devices (basicall CRT's) non linear behaviour, and has no relation to the way human vision works. To correctly mimic real life, any imaging system must be linear end to end. If gamma exists at some point, is to compensate an inverse non-linearity somewhere else into the system, and this has no relation to the way we perceive light.
Today, in digital imaging, gamma is only needed because integer formats are still the most widely used.
Regards
-
These two images display correctly thanks to the software (Photoshop is interpreting the 2.2 gamma of the image on the left, and the 1.0 gamma of the image on the right), which is sending to the output video devices the appropiate values for correct rendering
Yup, all you need is a profile to define that condition and it previews fine. I use a similar example in classes, loading an untagged gamma 1.0 image (which looks awful), put some Photoshop samples on the image and show how Assign Profile with the correct profile makes the image look fine without altering the numbers a lick. Most people who say “linear images look dark” don’t realize that without the proper embedded profile, its assumed to be gamma corrected and looks just awful.
Of course without ICC aware apps, this kind of falls apart.
Poynton sums up the debate about this in his FAQ (http://www.poynton.com/notes/colour_and_gamma/GammaFAQ.html#gamma):
21. Should I do image processing operations on linear or nonlinear image data?
If you wish to simulate the physical world, linear-light coding is necessary. For example, if you want to produce a numerical simulation of a lens performing a Fourier transform, you should use linear coding. If you want to compare your model with the transformed image captured from a real lens by a video camera, you will have to "remove" the nonlinear gamma correction that was imposed by the camera, to convert the image data back into its linear-light representation.
On the other hand, if your computation involves human perception, a nonlinear representation may be required. For example, if you perform a discrete cosine transform on image data as the first step in image compression, as in JPEG, then you ought to use nonlinear coding that exhibits perceptual uniformity, because you wish to minimize the perceptibility of the errors that will be introduced during quantization.
The image processing literature rarely discriminates between linear and nonlinear coding. In the JPEG and MPEG standards there is no mention of transfer function, but nonlinear (video-like) coding is implicit: unacceptable results are obtained when JPEG or MPEG are applied to linear-light data. In computer graphic standards such as PHIGS and CGM there is no mention of transfer function, but linear-light coding is implicit. These discrepancies make it very difficult to exchange image data between systems.
When you ask a video engineer if his system is linear, he will say Of course! - referring to linear voltage. If you ask an optical engineer if her system is linear, she will say Of course! - referring to linear luminance. But when a nonlinear transform lies between the two systems, as in video, a linear transformation performed in one domain is not linear in the other.
-
Sensors are (mostly) linear, but human vision and subjective preferences are not.
Indeed, but I think the question is where should the software become nonlinear. I would think that exposure compensation adjustments should take place in the linear domain, for instance with the D7000 there should be no difference (at least for non-clipped raw data) between choosing ISO 800 in the camera and converting with 0 EV exposure compensation, and ISO 200 pushed two stops in the converter, or for that matter ISO 1600 pulled one stop (even though the latter makes little sense from the point of view of noise). Of course after the initial EC to get the histogram in the right rendering range, all sorts of nonlinear transformations may help achieve a better rendering, but shouldn't this initial exposure map take place in the linear domain?
-
There is absolutely no need of gamma per se applied to digital images. ... Today, in digital imaging, gamma is only needed because integer formats are still the most widely used.
Digital imaging is not a small term applied to only acquisition stage. Gamma, essentially a non-linearity of a certain type, is used for several reasons in digital images, including compression.
Indeed, but I think the question is where should the software become nonlinear ... shouldn't this initial exposure map take place in the linear domain?
Yes, if the acquisition device is known to be linear.
Joofa
-
Digital imaging is not a small term applied to only acquisition stage. Gamma, essentially a non-linearity of a certain type, is used for several reasons in digital images, including compression.
Compression of the RGB data is not necessary when a floating point format is used. Can you speak about those reasons for which gamma is used?
-
Yup, all you need is a profile to define that condition and it previews fine.
I don't have the expertise to add much to this discussion, but I have an example of using a profile to handle a linear image. My Minolta 5400 scanner and software permit the output of a 16 bit linear file. When viewed without assigning the proper Minolta profile, the image indeed looks very dark. Apply the proper profile, and the image looks normal.
-
Of course after the initial EC to get the histogram in the right rendering range, all sorts of nonlinear transformations may help achieve a better rendering, but shouldn't this initial exposure map take place in the linear domain?
Hi Emil,
Indeed, there are several operations that are best done in linear gamma space. Blackpoint correction, exposure compensation, and white balancing come to mind, to name a few obvious ones. Then, once we enter the realm of human perception (color appearance models) though, there may be more efficient transforms needed. Efficiency is the reason for transforms, but they should not introduce color shifts unless that models human vision better.
Cheers,
Bart
-
Compression of the RGB data is not necessary when a floating point format is used. Can you speak about those reasons for which gamma is used?
Compression is dictated by an output device parameters such as bitrate, bucket capacity, etc. You are mixing up input stage (acquisition) with an output stage technique (compression). However, I just wanted to let you know that the term "digital imaging" is not to be used loosely for a certain domain-specific computation while giving an impression that it is true for all of its domain.
Now a little OT: the floating point's resolution is not always greater than an integer! For the same number of bits, the usual floating point will have more resolution in lower numbers, but rest assured that after some crossover point the integer would have more resolution. For e.g., I think for the IEEE 32-bit floating point the crossover points occur a little before 23 bits. In usual calculations that situation should not occur. Or in other words, make sure to keep your calculations below 2^23 if using IEEE 32-bit floats.
Now coming back to your question. Gamma is used at many different places: (1) Image data visualization, say in Lab space; uses a gamma of 1/3. (2) Compression; linear data is not good for compression such as jpeg/wavelet, etc. A gamma of an appropriate variety is applied many times. (3) SNR range "stabilization" in certain systems; that is more audio, but just to let you know that it is there. (4) Etc., Etc.
Joofa
-
If camera response is linear, what is the point of nonlinear color correction in raw converters?
Like Eric Chan said - "Sensors are (mostly) linear, but human vision and subjective preferences are not."
Non-linear processing is intended to give results that are perceptually "better"
Sandy
-
Like Eric Chan said - "Sensors are (mostly) linear, but human vision and subjective preferences are not."
Non-linear processing is intended to give results that are perceptually "better"
Sandy
I think that you are all mixing different things. Producing an XYZ-space "image" is not so much perception, but mathematics, I believe. dcraw does no strange stuff in producing XYZ. Now, after getting rid of camera quirks and having an image this is somewhat "absolutely referred" (within sensor limits, and on a normalized perceptual basis), it is time to make it "look good". That is when all of the proprietary stuff (that e.g. Adobe would apply to all cameras, since it has a generic representation in XYZ-space). Am I not right?
-h
-
There is absolutely no need of gamma per se applied to digital images. The correct mapping of the RGB values so that they display right can be done in software, in the output device (monitor plus video card), or both working together.
...
Gamma exists because of the original output devices (basicall CRT's) non linear behaviour, and has no relation to the way human vision works. To correctly mimic real life, any imaging system must be linear end to end. If gamma exists at some point, is to compensate an inverse non-linearity somewhere else into the system, and this has no relation to the way we perceive light.
Gamma is not only used for still images. If you read Poyntons books, I think that you will find that he supports my claim that Gamma exists for primarily two reasons:
- The first reason is to compensate for CRT native behaviour (and it was cheaper to do this once in the camera instead of in every customers tv set back in the days of analog signal processing).
- The second reason is to make it easier for lossy compression or any other dsp thath wants to work on "perceptually linear" data. I think that in dsp terms, this could be called a homomorphic transform (http://www.dspguide.com/ch22/7.htm).
Non-linear end-to-end functions exist and are described by Poynton:
From Poynton:
Television is usually viewed in a dim environment. If an images’s correct physical intensity is reproduced in a dim surround, a subjective effect called simultaneous contrast causes the reproduced image to appear lacking in contrast, as demonstrated above. The effect can be overcome by applying an end-to-end power function whose exponent is about 1.1 or 1.2. Rather than having each receiver provide this correction, the assumed 2.5-power at the CRT is under-corrected at the camera by using an exponent of about 1⁄ 2.2 instead of 1⁄ 2.5. The assumption of a dim viewing environment is built into video coding
-
I think that you are all mixing different things. Producing an XYZ-space "image" is not so much perception, but mathematics, I believe. dcraw does no strange stuff in producing XYZ. Now, after getting rid of camera quirks and having an image this is somewhat "absolutely referred" (within sensor limits, and on a normalized perceptual basis), it is time to make it "look good". That is when all of the proprietary stuff (that e.g. Adobe would apply to all cameras, since it has a generic representation in XYZ-space). Am I not right?
-h
No, actually I'm not mixing things. Adobe (and others) are ;D
If you use a camera profile with hue twists in it, then you have non-linear processing. To be more precise, before we get hung up on what "non-linear processing" is, you have hue varying with luminance. In the case of Adobe's profiles, that happens to occur in HSL space, but the space in which it occurs is irrelevant. What is relevant is where in the pipeline the non-linearity occurs. The point for ETTR is whether the non-linearity occurs before of after exposure correction. If the non-linearity is before, you have variations in hue in hue as a result of ETTR. And for many, although not all, Adobe profiles, the hue twists are configured to occur before the exposure settings take effect.
Where there is confusion is around what I would call "accidental" versus "deliberate" non-linearity. Accidental non-linearity is as a result of sensor imperfections or whatever, and can (should) be corrected for in the image processing pipeline somewhere. So generally the accidental non-linearities are not a major problem for ETTR, because they're usually quite small, and corrected for anyway. However, hue twists (and other similar mechanisms) are deliberate non-linearities introduced to enhance perceived image quality. They are generally large, and by definition not corrected for, and so are a problem for ETTR.
BTW, DCRaw doesn't have any hue twists, or anything similar to them - it does a straight linear conversion. Old school!! But DCRaw isn't the problem.
Sandy
-
I see.
So camera non-linearity (besides channel clipping) should be minor, and raw-processor non-linearity to correct it should only make things more linear. Channel clipping can be partially hidden by highlight recovery, but that is not something that should be happening with "ETTR".
Since commercial raw converters may not use XYZ or any other known intermediate representation (that we can access) we can only observe the input and the end-result. Someone did and concluded that it was non-linear/signal-dependent.
A hue-twist that is a smooth function of input level may be "large" and "visible", but I believe it to be still correctable (using a potentially large amount of effort). Clipping is not generally corectable. Did someone try to compare the color differences using ETTR and using a more conservative exposure? Even if Lightroom & friends really is color non-linear, that does not mean that a moderate difference in exposure will look 1)significantly different or 2)worse color-wise.
-
I hope it’s not inappropriate to ask this question in this thread, but there is a lot of knowledge here about linear.
Since my Minolta 5400 scanner outputs 16 bit linear tiff files, I've done so for positive film scans, then assigned the Minolta linear profile (which applies gamma encoding to produce a normal looking image), converted to my working color space (now ProPhoto) and then edited my photos in PS/ACR.
I’ve wondered, would there be any benefit if I could make my edits in ACR in linear mode? I’ve assumed that this is merely an academic question since ACR doesn’t support a “RAW” mode for my scanner and when ACR sees an untagged tiff file it assumes it is gamma encoded sRGB. However, if the answer to my first question is yes, then is there a way to process my linear files in ACR?
-
I hope it’s not inappropriate to ask this question in this thread, but there is a lot of knowledge here about linear.
Since my Minolta 5400 scanner outputs 16 bit linear tiff files, I've done so for positive film scans, then assigned the Minolta linear profile (which applies gamma encoding to produce a normal looking image), converted to my working color space (now ProPhoto) and then edited my photos in PS/ACR.
I’ve wondered, would there be any benefit if I could make my edits in ACR in linear mode? I’ve assumed that this is merely an academic question since ACR doesn’t support a “RAW” mode for my scanner and when ACR sees an untagged tiff file it assumes it is gamma encoded sRGB. However, if the answer to my first question is yes, then is there a way to process my linear files in ACR?
ACR internally operates in linear mode anyway, so no, there would be no benefit, even it were possible.
But more broadly, the various ACR controls operate on the assumption that the profile attached to a file correctly represents the data (linear data = linear profile); to try to, for example, use a different profile to what the data actually is will just result in very odd control behavior.
Sandy
-
Gamma is not only used for still images. If you read Poyntons books, I think that you will find that he supports my claim that Gamma exists for primarily two reasons:
- The first reason is to compensate for CRT native behaviour (and it was cheaper to do this once in the camera instead of in every customers tv set back in the days of analog signal processing).
- The second reason is to make it easier for lossy compression or any other dsp thath wants to work on "perceptually linear" data. I think that in dsp terms, this could be called a homomorphic transform (http://www.dspguide.com/ch22/7.htm).
The first reason is irrelevant today, since it can be corrected in the output device without any need to affect the image editing values.
For the second reason, is again irrelevant in the context we are interested here: digital photography edition. So following joofa's comment I will restrict my claim: "Gamma, in digital photography edition, is per se unnecessary. A floating point linear editor is totally possible, and gamma would play no role in it".
Regards
-
The first reason is irrelevant today, since it can be corrected in the output device without any need to affect the image editing values.
You would still need gamma to be able to traverse DVI connections limited to 8 bits of precision without banding. Doing Gamma "right" is quite difficult, and current LCD displays use all kinds of dirty tricks to pretend that they have a gamma-like response.
For the second reason, is again irrelevant in the context we are interested here: digital photography edition. So following joofa's comment I will restrict my claim: "Gamma, in digital photography edition, is per se unnecessary. A floating point linear editor is totally possible, and gamma would play no role in it".
Sure. But what editors are using floating point today? I have read that Adobe tried using linear light values in Photoshop, but there were so many customers complaining after being brought up on gamma-space values that they brought it back in?
-h
-
You would still need gamma to be able to traverse DVI connections limited to 8 bits of precision without banding. Doing Gamma "right" is quite difficult, and current LCD displays use all kinds of dirty tricks to pretend that they have a gamma-like response.
Sure, but irrelevant from the point of view of image edition. I am talking all time about the need vs no need of gamma in the image RGB values, which is the field in which the photographer edits his image.
Sure. But what editors are using floating point today?
No idea, but again this is irrelevant. I am talking about the need vs no need of gamma in the image RGB values, not about any particular (sub-optimal) implementation. In fact in my previous example of the dogs, the linear 1.0 gamma image is wrongly displayed (banding appears, although the image is not posterized in its RGB numbers) when not zoomed at 100%. Photoshop is not well suited to deal with linear images, even if it allows you to use linear colour profiles.
Regards
-
I've read this thread from beginning to end and I'm amazed that those who claim there might be a problem with ETTR exposures due to a non-linearity of sensor output as exposure approaches full-well capacity, and/or a problem with the capacity of RAW converters to handle this non-linearity, have not provided any 'real-world', photographic images demonstrating the problem.
Gullermo is the only one to provide images which demonstrate that there really is no problem here.
I frequently auto-bracket 3 exposures in situations where I don't have the time to use procedures to get a precise exposure, either because I'm trying to capture the moment which might pass quickly, or because I'm in transit, as on a guided tour, and have little time in general to get my shots.
I therefore have lots of examples of identical scenes that differ in exposure by two stops (I usually auto-bracket +/- one stop).
I can honestly claim that I have never experienced a problem with hue and color shifts as a result of choosing the ETTR exposure out of the 3 instead of one that has either 1 or 2 stops less exposure.
What I do find is that sometimes the longest exposure of the 3 might be very marginally overexposed in the sense that one channel is clipped, and as a consequence I might prefer to use one of the underexposed shots. I might decide to use the shot that is underexposed by 2/3rds of an EV in preference to the one that is overexposed by 1/3rd of a stop.
Even if it's too difficult to merge the 3 shots to HDR because of movement in the scene or movement of the hand-held camera, having 3 different exposures increases one's options in processing. For example, sometimes it might be easy to use the sky in the least exposure and the foreground in the greatest exposure, if the horizon is not to complicated.
DXOMark in one of their 'insights' articles have very clearly explained the relationship between noise and exposure. For example, underexposing a shot by 1 EV has a greater effect on the shadows than it does on the midtones.
As a consequence of a 1 stop underexposure, the midtones and upper midtones will have approximately a 3dB lower SNR. However, the shadows will have a 6dB lower SNR.
Of course, this is an oversimplification. In practice there's no such clear-cut delineation. There will be a sliding scale. The upper midtones will lose about 2dB of SNR, the midtones about 3dB, the lower midtones about 4dB, the moderate shadows about 5dB and the deep shadows about 6dB. The very deep shadows will lose even more than 6dB of SNR.
How significant is a 3dB loss in SNR? Well, that depends upon print or monitor enlargement. At 100% on the monitor, a 3DB difference in SNR is clearly noticeable, and a 6dB difference smacks you in the face. But on a postcard size print it wouldn't.
Generally, the ETTR principle is only of benefit to those who wish to maximise the potential image quality of their camera, either by printing large now, or at some time in the future.
The practical photographer, as opposed to the armchair theoretician, needs to see the practical relevance and significance of these subtle issues which are discussed in this thread.
There are certain individuals on this forum, whom I shall not name because my intention is not to embarrass, who seem very technically competent and who appear to have a deep understanding of the digital technology of cameras, but who never post any real-world images to demonstrate what they are talking about.
I often wonder why. Is it because such individuals are very, very shy, or is it because (more likely) that many people on such forums as this appear to judge the worth of an argument or technical point on the artistic merits of the photos that demonstrate the point.
Some of you may have noticed that not all of the photos that I use to demonstrate a technical point are photographic masterpieces. The reason is, when I search for a photo to demonstrate a technical point, I'm not searching for a photo that I'm particularly proud of, which I think is great, that I think has signifiant artistic merit, that I want to 'show off' as a fine example of the best in my portfolio.
I search for an image which best demonstrates the point I'm making. If it also happens to be an interesting or 'artistic' image in its own right regardless of any technical issues, then that's also fine, but that's not the priority.
Okay! Now I've got that off my chest, I'm going to present a couple of images that differ in exposure by 2 stops. They've been chosen because the back-lighting situation has created some deep shadows, and because the auto-bracketed exposures were taken at ISO 800 (perhaps needlessly) because I didn't have time to change it. However, a couple of seconds later the birds changed their posture. I also like this particular shot. I like the silhouette effect. It's a keeper, and I won't be raising the shadows.
The fact that these shots were at ISO 800 demonstrates the greater need for an ETTR exposure at high ISO. Dynamic range at ISO 800 is always reduced considerably (compared with base ISO), whatever the camera. Therefore a 6dB reduction of SNR in the shadows is of major concern at ISO 800.
These shots have had zero sharpening and no processing, apart from the lightening of the shadows to demonstrate differences in shadow noise. I simply hit the 'auto' button in ACR, then converted. The auto button in ACR gave the longest exposure a -0.2 EV adjustment plus a bit of highlight recovery, and the shortest exposure received a +1.5 EV adjustment plus a bit of additional brightness and contrast. One could argue that the longest exposure is not quite a full ETTR. It could perhaps have received an addition 1/3rd of a stop exposure. But let's not quibble.
No sharpening or noise reduction has been applied to either image.
The first image is of the full scene comparing exposures differing by 2 stops. Which is which?
The second is a 200% crop showing the increased noise in the sky, in the underexposed shot.
The third is a 67% crop showing the dramatically increased shadow noise in the 2 stop's underexposed image.
What more is there to say on the subject?
-
I've read this thread from beginning to end and I'm amazed that those who claim there might be a problem with ETTR exposures due to a non-linearity of sensor output as exposure approaches full-well capacity, and/or a problem with the capacity of RAW converters to handle this non-linearity, have not provided any 'real-world', photographic images demonstrating the problem.
.....
What more is there to say on the subject?
To answer your question let me try to provide a real world example taken with my Sony A850 and a Minolta 50/1.7 lens in the recently renovated Antwerp Central Station.
Auto exposure bracket + 2EV, 0 and -2 EV, all within 1 second. Lightroom 3.3 and Auto adjustment, with afterwards a slight play with exposure and brightness to get the overall tone somewhat more similar (all the other setting were the same and all 3 shots were AWB and came back identical 5000/10 "as shot")
-2EV: (http://i227.photobucket.com/albums/dd43/pegelli/PEG_A850_02053_20101017.jpg)
0EV: (http://i227.photobucket.com/albums/dd43/pegelli/PEG_A850_02052_20101017.jpg)
+2EV: (http://i227.photobucket.com/albums/dd43/pegelli/PEG_A850_02054_20101017.jpg)
My conclusion is there is a slight hue shift between all three pictures, especially apparent if you look at the background old stone building. A shorter exposure warms up the grey stone consistently in this bracketed series.
Don't know which one is "better" but the hue shift can be easily seen in this case.
I do support Ray's conclusion (can provide crops if needed) that the noise in the - 2EV picture boosted with + EV in Lightroom is significantly more than the "measured" exposure @ 0EV and the +2EV example.
Btw, I never HDR'd this picture, too much movement in the people om the escalator :-[
-
My conclusion is there is a slight hue shift between all three pictures, especially apparent if you look at the background old stone building.
It would be useful if the term hue shift were better defined. Some hue’s in color space shift while others do not? Everything is shifting slightly? Is this the effect of ETTR or due to slight differences in rendering needed to bring the ETTR and “normal” exposure to match tone? All raw converters?
In the last example below, the +2EV and the others show differences in color (the orange and green “flags” on the left wall) but they also show exposure/tone differences as illustrated in the sky peaking through the glass in the upper right area of the image. There’s more here than a difference in “hue” to my eye.
-
Is the +2EV raw file available for inspection? It is quite possible that the places identified as having a "hue shift" are simply instances where one or more channels is blown (either in the raw, or due to white balance amplification) and the raw converter can't recover accurate color information.
-
I've read this thread from beginning to end and I'm amazed that those who claim there might be a problem with ETTR exposures due to a non-linearity of sensor output as exposure approaches full-well capacity, and/or a problem with the capacity of RAW converters to handle this non-linearity, have not provided any 'real-world', photographic images demonstrating the problem.
<Bunch of text cut>
The first image is of the full scene comparing exposures differing by 2 stops. Which is which?
The second is a 200% crop showing the increased noise in the sky, in the underexposed shot.
The third is a 67% crop showing the dramatically increased shadow noise in the 2 stop's underexposed image.
What more is there to say on the subject?
Oh dear. Now the real photographers have arrived, all us mere "armchair theoreticians" should bow out ;D ;D ;D ;D
Before I bow out, just so we're all on the same page, I think that everybody on all sides of the ETTR debate would agree with almost all of what you said. Specifically:
- Yes, absolutely, if you underexpose you will get more noise
- Yes, if you are for some reason limited in ISO selection, ETTR is a good idea - if you read the blog post I referenced right at the beginning of this, you'll see that I specifically identify already being at base ISO as the one situation where ETTR makes sense.
But the question is not, to use your ISO 800 example, whether if you take two exposures both at ISO800 one with 2 stops overexposure, one with normal exposure, that you will get lower noise in the overexposed version. Everybody on this thread (so far as I know anyway) agrees that at a single ISO, ETTR makes sense.
The question that's actually being addressed, and I'm afraid your example isn't relevant to, is what if you just changed the ISO setting to 200? Those of us on the "ETTR is over hyped" side of the fence will say that you get the same noise levels, with none of the problems as you would with the "ISO800 plus 2 stops of ETTR".
<Sandy politely bows out of conversation>
Sandy
Post script to Andrew: If you haven't seen them, I did some articles on hue shifts here: http://chromasoft.blogspot.com/2009/02/adobe-hue-twist.html
The term "hue twist" is actually Eric Chan's. Although I think he now regrets coining the term(!) As regards what raw converter, etc. Certainly most "new generation" Adobe profiles, but not all. Capture One also seen to use hue twists, judging by some of the hue variations I've seen, although not as much as Adobe.
-
Lightroom 3.3 and Auto adjustment, with afterwards a slight play with exposure and brightness to get the overall tone somewhat more similar
Thanks for having let the metadata in the file ; one can see the changes are caused by vastly different LR settings (there are some local editing with both brush and gradient in the 0EV shot only, some recovery for the +2EV and 0EV shots causing probably much of the hue shift, the exposure setting are not matched and the brightness/blacks/contrast tweaked...). That still leads to a valid conclusion : the application of some LR settings may adjust the hue and tone of your images. ;)
However, it might be more interesting (once you've verified nothing is blown in the +2 shot, as said Emil) to get the 3 shots reset at default and then only "match total exposures" between them?
-
@ digitaldog, sorry can't define it better. I was looking for colour/hue shifts in the midtones where there seems to be no clipping. don't know if it's universal and based on the comments from NikoJorj I tinkered too much with the files to be representative anyway. There are unrecoverable blown highlights in the dome, but I was keeping those out of my comparison.
@ ejmartin, no problem. I'll try to put them in a dropbox site or something. Need to sort it out to see how to do that but when I've done that I'll post the links here. Might well be you're right, I have no knowledge nor software to test it so any help you can give is more than welcome.
@ NikoJorj, thanks, and I think you're right (or almost, because all three files have similar (not the same) local adjustments and gradients), but you're absolutely right I tinkered too much with the files in an inconsistent manner to draw any conclusions.
I've found 3 more files (from my A700) so here's 6 center crops (A700: 1370x914, then reduced to 800 for web posting||A850: 2834x1884, reduced to 800 for web posting) where the only differences are the EV slider to make them look roughly the same (so +2EV from default for the -2EV shot, default for the 0EV shot and -2EV for the +2EV shot). Colour shifts are much less now, and those still present are probably due to unrecoverable blown channels. Exif should again be fully included in the file for you to take a look at.
A700 series:
(http://i227.photobucket.com/albums/dd43/pegelli/PEG_A700_12206_20101017.jpg)
(http://i227.photobucket.com/albums/dd43/pegelli/PEG_A700_12207_20101017.jpg)
(http://i227.photobucket.com/albums/dd43/pegelli/PEG_A700_12208_20101017.jpg)
A850 series (the same frames as posted before, only now a center crop)
(http://i227.photobucket.com/albums/dd43/pegelli/PEG_A850_02052_20101017-1.jpg)
(http://i227.photobucket.com/albums/dd43/pegelli/PEG_A850_02053_20101017-1.jpg)
(http://i227.photobucket.com/albums/dd43/pegelli/PEG_A850_02054_20101017-1.jpg)
Advice and comments more than welcome, I need to learn a lot in this field so thanks for all your reactions so far.
-
Post script to Andrew: If you haven't seen them, I did some articles on hue shifts here: http://chromasoft.blogspot.com/2009/02/adobe-hue-twist.html
The term "hue twist" is actually Eric Chan's. Although I think he now regrets coining the term(!) As regards what raw converter, etc. Certainly most "new generation" Adobe profiles, but not all. Capture One also seen to use hue twists, judging by some of the hue variations I've seen, although not as much as Adobe.
But in the above context, this isn’t necessarily an ETTR related issue?
-
Oh dear. Now the real photographers have arrived, all us mere "armchair theoreticians" should bow out ;D ;D ;D ;D
Before I bow out, just so we're all on the same page, I think that everybody on all sides of the ETTR debate would agree with almost all of what you said. Specifically:
- Yes, absolutely, if you underexpose you will get more noise
- Yes, if you are for some reason limited in ISO selection, ETTR is a good idea - if you read the blog post I referenced right at the beginning of this, you'll see that I specifically identify already being at base ISO as the one situation where ETTR makes sense.
But the question is not, to use your ISO 800 example, whether if you take two exposures both at ISO800 one with 2 stops overexposure, one with normal exposure, that you will get lower noise in the overexposed version. Everybody on this thread (so far as I know anyway) agrees that at a single ISO, ETTR makes sense.
The question that's actually being addressed, and I'm afraid your example isn't relevant to, is what if you just changed the ISO setting to 200? Those of us on the "ETTR is over hyped" side of the fence will say that you get the same noise levels, with none of the problems as you would with the "ISO800 plus 2 stops of ETTR".
<Sandy politely bows out of conversation>
Sandy
Post script to Andrew: If you haven't seen them, I did some articles on hue shifts here: http://chromasoft.blogspot.com/2009/02/adobe-hue-twist.html
The term "hue twist" is actually Eric Chan's. Although I think he now regrets coining the term(!) As regards what raw converter, etc. Certainly most "new generation" Adobe profiles, but not all. Capture One also seen to use hue twists, judging by some of the hue variations I've seen, although not as much as Adobe.
Since you intend to 'bow out', perhaps there's no point in responding. However, for the benefit of others, I shall.
My first reaction to your post was one of confusion. Then I looked at your blog in an earlier post and all is clear. You have failed to distinguish between the two basic methods that cameras of different types handle ISO increases. One type, usually CCD such as many P&S cameras and most MFDBs, are essentially 'one-ISO' cameras.
With such cameras no image quality benefit is derived by using a correct ETTR at a higher ISO than base. One might as well underexpose at base ISO and get the increased assurance that one will at least not overexpose, even though the LCD review might appear unhelpfully dark.
The other type of camera, usually most DSLRs, and particularly Canon DSLRs with which I have the most experience, provides a distinct IQ advantage when raising the ISO to create an ETTR exposure at the increased ISO setting, as opposed to underexposing at base ISO.
There's no doubt about this. I've tested it, although it has to be said the most dramatic improvement will be observed when the ISO increase is great. For example, an ETTR at ISO 1600 exhibits very significantly less noise, from shadows to upper midtones, than the same exposure at base ISO, which would of course be 4 stops underexposed.
As far as I understand, the reason for this improved IQ when using the same exposure at higher ISOs, is due to a boosting or amplification of the analog signal prior to A/D conversion and of course prior to all other signal processing further up the chain before the data is eventually written to the memory card.
Such amplification in itself cannot reduce noise. In fact, whatever noise is present in the sensor at the precise time of amplification will also be amplified. The reduced SNR in the final output is not due to a reduction in absolute noise but a reduction in relative noise, that is, the noise introduced after amplification is smaller as a proportion of the larger signal.
In other words, with DSLRs the higher ISO setting is merely an instruction to the camera to amplify the analog signal by a specific amount, at the earliest possible stage, even before A/D conversion. Whereas, with the other type of sensor, mostly CCDs I believe (and that would include the Canon G10 you used), the ISO setting would appear to be an instruction to the camera's processor to boost the signal at the end of the processing chain. There's a big difference.
However, with the introduction of Sony's latest sensor used in the Pentax K5 and Nikon D7000, we have a new development whereby it seems that only one analog boost takes place, and that's at base ISO. This provides no additional advantage at very high ISO, but does result in lower noise at base ISO and lower noise even up to ISO 400 where, remarkably, DR in this cropped format is even higher than that of the 5DMk2 at ISO 400, whether at pixel level or at normalised print sizes.
In summary, unless one is using one of the 'effectively' one-ISO cameras, such as the CCD type, and now the D7000 and Pentax K5, it's always advisable to raise ISO till one gets an ETTR exposure, rather than underexpose at base ISO.
And of course, for maximum image quality across the entire tonal range from deep shadows to highlights, an ETTR at base ISO provides the best image quality the camera can deliver, assuming shutter speed and f/stop are appropriate for the conditions and the composition.
Hue shifts or twists are not a problem for me. If I were to see any, I'd fix them. Most of my images are given individual treatment in Photoshop. I'm changing hues, color saturation, shadows, midtones etc all the time. Let's not create imaginary problems. ;D
-
To answer your question let me try to provide a real world example taken with my Sony A850 and a Minolta 50/1.7 lens in the recently renovated Antwerp Central Station.
Auto exposure bracket + 2EV, 0 and -2 EV, all within 1 second. Lightroom 3.3 and Auto adjustment, with afterwards a slight play with exposure and brightness to get the overall tone somewhat more similar (all the other setting were the same and all 3 shots were AWB and came back identical 5000/10 "as shot")
My conclusion is there is a slight hue shift between all three pictures, especially apparent if you look at the background old stone building. A shorter exposure warms up the grey stone consistently in this bracketed series.
Don't know which one is "better" but the hue shift can be easily seen in this case.
I do support Ray's conclusion (can provide crops if needed) that the noise in the - 2EV picture boosted with + EV in Lightroom is significantly more than the "measured" exposure @ 0EV and the +2EV example.
Btw, I never HDR'd this picture, too much movement in the people om the escalator :-[
Pegelli,
Thanks for taking the trouble to at least try to demonstrate the problem.
My impression is that all these images are overexposed. Not even the shortest exposure at 1/1600th is an ETTR, judging by the histogram.
'Expose to the right' does not mean that the histogram is pushed against the right side like a cliff face, although taken literally the expression could mean that.
What I consider to be an ETTR is a histogram that barely touches the right vertically and has no obvious spikes in the vicinity of that right vertical after a reasonable, negative EC adjustment in ACR, such as -0.67 EV.
It's possible to get a histogram of a significantly overexposed shot looking as though it's representative of an ETTR, by throwing everything at it, in ACR. For example, by applying a -2 EV adjustment (or greater), moving the 'recovery' slider to its maximum of 100, shifting the 'brightness' slider to zero or less, and moving the highlight slider of the 'Tone Curve' to its minimum of -100.
It's very easy to inadvertently blow out one of the channels without it appearing obvious, as a result of Adobe's clever reconstruction of blown highlights.
If one is comparing different exposures which are all overexposed to varying degrees, then it's quite possible there will be hue shifts in the upper midtones.
This is how the histogram of your shortest exposure appears in PS.
-
@ digitaldog, I think what sandymc is saying is that you get colour shifts when for instance you compare a 0 EV shot with another one that got +1 EV in the camera followed by -1 EV in the raw converter. That's the "root problem" she is describing. Since this is what you do with ETTR it's only sideways an ETTR problem.
@ Ray, sorry for posting such a confusing example. I know there are blown regions in the frame. However I was more looking to colour changes in the "mid tones" (the red beam left in the frame and the facade in the background. Since I was reading the problem more as described to digitaldog above, and am not looking at colour shifts in blown regions it still demonstrated the effect in these mid tone regions (I thought). However NikoJorj poited out I had "trown" too many other different setting at the picture (causing the shifts), hence posting my second series where the only difference is the EV slider in my raw converter and voila, the colour changes are virtually gone which supports the position you're taking in this discussion.
One thing though, the white spike you demonstrate in the histogram is there, but since I used white lettering to mark the shots it's bigger than of the picture without the lettering. Also this picture got quite a bit of +EV in the converter, at the default import settings the histogram of the shortest exposure is just touching the right side, and I get only something like 10 pixels highlight warning when pressing "alt" + recovery slider.
Btw, here's another experiment I did.
Used the EV slider to "normalise"everything to the -2EV shot in camera.
Also put the black slider at 0 for all 3, rest at default settings
(http://i227.photobucket.com/albums/dd43/pegelli/PEG_A850_02052_20101017-2.jpg)
(http://i227.photobucket.com/albums/dd43/pegelli/PEG_A850_02053_20101017-2.jpg)
(http://i227.photobucket.com/albums/dd43/pegelli/PEG_A850_02054_20101017-2.jpg)
Conclusion: plenty of blown channels in the + 2EV shot to cause all kind of weird shifts 8)
Btw, changed the colour of the lettering typed in, so they don't show up as inadvertent "blown highlights"
-
Conclusion: plenty of blown channels in the + 2EV shot to cause all kind of weird shifts 8)
Ditto ; this one is definitely not exposed to the right.
It's really a matter of "it's fun to play with matches but it hurts to burn your fingers".
For the 2 other ones, I don't see any hue change but do see a minor exposure change (inaccurate shutter, which is quite probable for speeds way beyond X-sync, or maybe slightly stuck aperture?).
-
Conclusion: plenty of blown channels in the + 2EV shot to cause all kind of weird shifts
Btw, changed the colour of the lettering typed in, so they don't show up as inadvertent "blown highlights"
Here are my own tests using the Nikon D3, a Colorchecker, and Imatest for an ETTR expsoure with no blown channels and a Minus 1.4 EV exposure with exposure correction in ACR. I used ACR 6.3 with the AdobeStandard profile and linear settings in ACR (sliders on main tab all set to zero). The images appear somewhat flat, since a linear tone curve was employed. The color shifts are minimal. The Colorchecker images used for analysis were in 16 bit ProPhotoRGB, but are converted to 8 bit JPEG for web display. Manual white balance in ACR on the second brightest neutral square was employed. The raw files were analyzed with Rawnalize.
Rawnalize view of ETTR exposure:
(http://bjanes.smugmug.com/Photography/D3-ACR-ETTR/008RA/1134454595_pMUwm-O.png)
Rawnalize view of Underexposure:
(http://bjanes.smugmug.com/Photography/D3-ACR-ETTR/012RA/1134454638_oXZsF-O.png)
JPEG for ETTR exposure:
(http://bjanes.smugmug.com/Photography/D3-ACR-ETTR/0008linexpMinus05/1134455119_K6BvS-O.jpg)
JPEG for corrected underexposure:
(http://bjanes.smugmug.com/Photography/D3-ACR-ETTR/0012linexpPlus175/1134455213_DbyUB-O.jpg)
Imatest analysis of ETTR exposure:
(http://bjanes.smugmug.com/Photography/D3-ACR-ETTR/0008linexpMinus05colorerror/1134455912_sAY3c-O.png)
Imatest analysis of the corrected underexposure:
(http://bjanes.smugmug.com/Photography/D3-ACR-ETTR/0012linexpPlus175colorerror/1134455955_Bm5t8-O.png)
-
@ digitaldog, I think what sandymc is saying is that you get colour shifts when for instance you compare a 0 EV shot with another one that got +1 EV in the camera followed by -1 EV in the raw converter. That's the "root problem" she is describing. Since this is what you do with ETTR it's only sideways an ETTR problem.
But that doesn’t put the blame if that’s the correct term, solely on the use of ETTR as I read this. How do we know that when using correct ETTR techniques, we don’t have to apply differing rendering settings in one or all raw converters to produce a result that has no such “hue shifts”? Just normalizing the exposure slider (A Michael term but one that makes sense) is all that is necessary? For all raw converters?
-
Ditto ; this one is definitely not exposed to the right.
It's really a matter of "it's fun to play with matches but it hurts to burn your fingers".
For the 2 other ones, I don't see any hue change but do see a minor exposure change (inaccurate shutter, which is quite probable for speeds way beyond X-sync, or maybe slightly stuck aperture?).
Agree, it's clearly exposed "over the right", but don't think I got burned. It was never intended to be an ETTR exposure, it was simply an example I had from a HDR bracketing sequence.
Also see the exposure change, however - 0,1 EV on the -2 EV shot fixes that almost completely, but didn't want to do that for the "purity"of the comparison I did between the three frames. As you say, minor inaccuracies of the shutter as well as in the mechanical aperture (this was taken with a 15 year old Minolta AF lens) probably account for that. I've seen changes as big as 0.2-0.3 EV in continuous shooting series with this and other lenses as well. Since I shoot raw I really can't loose sleep over that, for jpg it might be more critical.
@ bjanes, nice test and well executed, thanks for doing and posting that.