Luminous Landscape Forum

Site & Board Matters => About This Site => Topic started by: marcmccalmont on August 04, 2011, 08:38:18 pm

Title: Better than ETTR ?
Post by: marcmccalmont on August 04, 2011, 08:38:18 pm
The article on ETTR got me to thinking. My father and his chief engineer (Arnold M. McCalmont, Matthew Slate, Technical Communications Corporation) patented a signal processing technique  for voice encryption that might have an application in digital imaging. Their technique is clever and it improved recovered voice quality on HF SSB radio links which is unheard of with analog crypto gear, intelligibility usually gets worse. The idea is simple, high pass and low pass the analog signal, invert the lower half and read it out at twice the frequency (for the analog encryption they encoded the analog segments using time division multiplexing (TDM).
So ETTR is trying to put as much data as possible into the most significant bits, in effect lowering the noise floor.
My thought is to take the analog signal, high pass and low pass filter it, invert it, digitize all 4 segments (highlights. upper mid, lower mid and shadows), store each segment as a 16  bit word. I'm thinking if digitized at 2x the frequency the information would be stored in the most significant byte? In this way all four segments of the signal would have higher resolution. Your thoughts? I haven't thought it out thoroughly so don't be too critical just throwing the concept out to see if it has merit. Oh, all rights reserved Marc McCalmont 2011 just in case it might have commercial merit.
Marc
Title: Re: Better than ETTR ?
Post by: jeremypayne on August 04, 2011, 09:00:30 pm
As I understand it, the point of ETTR is to collect the maximum number of photons. 
Title: Re: Better than ETTR ?
Post by: marcmccalmont on August 04, 2011, 09:20:59 pm
good point! the captured photons are the captured photons no mater how they are recorded
marc
Title: Re: Better than ETTR ?
Post by: dreed on August 05, 2011, 03:52:37 am
So ETTR is trying to put as much data as possible into the most significant bits, in effect lowering the noise floor.
My thought is to take the analog signal, high pass and low pass filter it, invert it, digitize all 4 segments (highlights. upper mid, lower mid and shadows), store each segment as a 16  bit word. I'm thinking if digitized at 2x the frequency the information would be stored in the most significant byte? In this way all four segments of the signal would have higher resolution. Your thoughts? I haven't thought it out thoroughly so don't be too critical just throwing the concept out to see if it has merit.

This sounds like details that would be relevant to the analogue->digital converter on the sensor or in the camera regardless of exposure. The final step in the above needs to be converting those four 16bit values into one 16bit value or the size of a raw data file is going to quadruple - not something that most people would enjoy.

Quote
Oh, all rights reserved Marc McCalmont 2011 just in case it might have commercial merit.

Find an intellectual property lawyer and ask them what rights and protection you have for a design that you've detailed in a public forum.
Title: The point of ETTR is to maximize SNR
Post by: Guillermo Luijk on August 05, 2011, 04:39:23 am
As I understand it, the point of ETTR is to collect the maximum number of photons.

The point of ETTR is to maximize SNR. The best way to achieve this is by maximizing the number of photons, but if you cannot increase aperture and shutter and still have insufficient exposure, you can push ISO to maximize SNR (not all cameras will benefit equally; Canons are the ones which take more advantage of pushing ISO, the new Sony sensor found on Pentax K5 and Nikon D7000 gets a negligible improvement from pushing ISO).

In the following two images from a Canon 350D, the same number of photons were collected (they were shot at the same aperture and shutter, only changing ISO):

(http://www.guillermoluijk.com/article/iso/versus.jpg)


So the technique exposed by Marc would be of no advantage here. ETTR is not about number of levels (they are always sufficient, no matter how much you expose, because noise always dithers posterization), ETTR is only about improving SNR.

Regards
Title: Re: Better than ETTR ?
Post by: marcmccalmont on August 05, 2011, 04:41:01 am
This sounds like details that would be relevant to the analogue->digital converter on the sensor or in the camera regardless of exposure. The final step in the above needs to be converting those four 16bit values into one 16bit value or the size of a raw data file is going to quadruple - not something that most people would enjoy.

Yes it is an analog technique to increase the "resolution" of the digitized signal, the increase in file size might be worth it if the IQ improved enough
Title: Re: Better than ETTR ?
Post by: marcmccalmont on August 05, 2011, 04:46:38 am
one might argue that the shadows don't have enough levels and more in the lower greys might help too.
Marc
Title: Re: Better than ETTR ?
Post by: Guillermo Luijk on August 05, 2011, 05:14:00 am
one might argue that the shadows don't have enough levels and more in the lower greys might help too.

More levels could help in what? in displaying noise more accurately?.

Look at these images, left is 8-bit, right is only 5-bit:

(http://www.guillermoluijk.com/article/histogram/poster.gif)

However the extra levels in the first image are useless, because noise is greater than the gap between two 8-bit adjacent levels:

Title: Re: Better than ETTR ?
Post by: marcmccalmont on August 05, 2011, 07:02:52 am
I'm not proposing not to ETTR. ETTR does 2 things, lowers noise and for each stop of ETTR doubles your usable levels correct?
As sensors get better and lower noise I'm sure the middle greys where eyesight is most sensitive could be improved with more resolution (levels).
Granted more data below your noise floor is useless but most of your data in the highlights not in the midtones doesn't seem logical to me just the result of binary math?
Are more levels important? I see a slight improvement in print with a 16 bit work flow and a 16 bit print driver, small but noticeable.
Marc
Title: Re: Better than ETTR ?
Post by: michael on August 05, 2011, 08:08:03 am
Dissing having more levels may be fine from a theoretical perspective, and yes, noise is the main reason to use ETTR. But...

In the real world (rather than the theoretical one) it is frequently the case that one wants to manipulate the image (open up shadows, for example, to reveal  nuanced detail). If you've used ETTR and then "normalized" the image you now have many more tonal level in the shadows than you would otherwise have had.

It doesn't take much time in front of the screen to tell that the benefit of doing this is quite real. And, as with most such things, maybe not in the day to day, but most usefully in the extremes.

Michael
Title: Re: Better than ETTR ?
Post by: Graeme Nattress on August 05, 2011, 10:49:23 am
More tonal levels (I normally talk "code values") don't help if they're just more code values representing noise. Say you have a sensor, and you normally have a 12bit DAC on it, but you swap to a 16bit DAC. You now have vastly more code values in the shadows, but you don't necessarily get a better picture unless the sensor has low enough noise to actually put image data (rather than noise) into those newly added code values.

Graeme
Title: Re: Better than ETTR ?
Post by: bjanes on August 05, 2011, 11:06:55 am
Dissing having more levels may be fine from a theoretical perspective, and yes, noise is the main reason to use ETTR. But...

In the real world (rather than the theoretical one) it is frequently the case that one wants to manipulate the image (open up shadows, for example, to reveal  nuanced detail). If you've used ETTR and then "normalized" the image you now have many more tonal level in the shadows than you would otherwise have had.

It doesn't take much time in front of the screen to tell that the benefit of doing this is quite real. And, as with most such things, maybe not in the day to day, but most usefully in the extremes.

Michael

Michael,

You have to move away from the number of tonal levels affecting image quality and start considering noise and signal to noise. A high number of levels is advantageous in producing smooth tonal gradations, but the advantage is lost when the image noise exceeds the quantization step as shown earlier in this thread by Guillermo and discussed at length by Emil Martinec in his excellent treatise (http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/index.html) Noise, Dynamic Dange, and Bit Depth. The brightest f/stop of a 12 bit raw capture contains nowhere near 2048 levels but it does have a high SNR.

Exposing to the right increases the SNR over the entire exposure range and the improved SNR is often most appreciated in the deep shadows where noise limits the dynamic range of the capture. According to Emil, raw data are never posterized, since noise exceeds the quantization step, but posterization may be brought by processing (for example, application of a median filter).

In most cases, ETTR improves shadow image quality by increasing the SNR and not by increasing the number of encoded levels. Insufficient levels results in posterization. My experience with Nikon DSLRs (D700, D200, and D3) is that shadow image quality is usually affected by noise and not by posterization.

I encourage you to read Emil's essay which discusses your previous ETTR article in this section (http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p3.html#ETTR) and get back to us on Emil has screwed up things. However, I will not hold my breath while awaiting your reply.

Regards,

Bill
Title: Re: Better than ETTR ?
Post by: michael on August 05, 2011, 11:18:31 am
I've read Emil's treatise, and have great respect for his work. And, I also understand the theory.

But, in practice, I am confident that I see smoother tonal gradations on ETTR "normalized" shadow areas rather than native ones when I need to strongly open up such tonalities.

So, who do I believe? As the old joke has it, "The experts, or my lying eyes"?

Michael

Ps: Bumble bees can indeed fly, and prove it to themselves every day.
Title: Re: Better than ETTR ?
Post by: Graeme Nattress on August 05, 2011, 11:24:58 am
Michael, the question is not do you see an improvement in ETTR, but is that improvement due to more code values or lower noise? If you actually saw banding / posterization in the shadows then the answer is "ETTR is better because of more code values for shadows", whereas if you don't see banding / posterization, it's due to "ETTR lowers noise in the shadows".

I've experimented with bit-depth in RAW processing and indeed, if you reduce the bit depth precision too much (and at the wrong point in the processing) you can indeed induce banding / posterization. I could certainly see that happening if you put a DAC on the sensor that lacks sufficient precision for the task at hand - say you put a 10bit or 8bit DAC on a modern sensor.

Graeme
Title: Re: Better than ETTR ?
Post by: deejjjaaaa on August 05, 2011, 11:33:00 am
But, in practice, I am confident that I see smoother tonal gradations on ETTR "normalized" shadow areas rather than native ones when I need to strongly open up such tonalities.
may be the software is just not doing a great job, that's it... 2x2 = 4, but if you software will display 5 - are you going to believe it ?
Title: Re: Better than ETTR ?
Post by: fdisilvestro on August 05, 2011, 11:35:59 am
if you reduce the bit depth precision too much (and at the wrong point in the processing) you can indeed induce banding / posterization. I could certainly see that happening if you put a DAC on the sensor that lacks sufficient precision for the task at hand - say you put a 10bit or 8bit DAC on a modern sensor.

Graeme

Exactly!, that's the key issue. As long as you have sufficient bit depth, all that matters is to maximize SNR
Title: Re: Better than ETTR ?
Post by: Eric Myrvaagnes on August 05, 2011, 12:47:21 pm
So do I conclude that I'm right to be trying to use ETTR, even though I may be doing it for the wrong reason???  ???
Title: Re: Better than ETTR ?
Post by: ErikKaffehr on August 05, 2011, 01:05:39 pm
Hi,

Yes, ETTR is a bright idea, but avoid clipping. I had done some ETTR experiments lately and got bad clipping.

Best regards
Erik

So do I conclude that I'm right to be trying to use ETTR, even though I may be doing it for the wrong reason???  ???
Title: Re: Better than ETTR ?
Post by: michael on August 05, 2011, 01:18:55 pm
Graeme,

You may very well have hit the proverbial nail on the head.

I don't just use one or two high-end cameras. I use a dozen or more cameras a year, many of them low-end consumer grade for testing.

So, let's just say that since I have observed this phenomenon from time to time, I like to tip the odds in my favour, even if doing so may not always be necessary.

Once again for clarity. I most definitely understand the science. It just doesn't always jibe with my experience.

Michael
Title: Re: Better than ETTR ?
Post by: digitaldog on August 05, 2011, 02:18:17 pm
Yes, ETTR is a bright idea, but avoid clipping. I had done some ETTR experiments lately and got bad clipping.

Then that wasn’t proper ETTR, it was over exposure. ETTR is about optimal exposure for the data. If you clip and didn’t wish to, that was an exposure error.
Title: Re: Better than ETTR ?
Post by: ndevlin on August 05, 2011, 02:58:59 pm
Ok, does anyone disagree with the following:

When one exposes a zone II shadow as zone V (to use real photography terms  ;)) and then subsequently places that detail back down into zone II in post-processing, that shadow is much 'cleaner' than if it had simply been exposed at zone II to begin with.  No, there aren't any more tonal values in the shadow areas this way, but there is a qualitative difference.

The same results hold true in a slightly different way for higher tonal values where noise is less of an issue with a 'straight' capture to begin with. But there, too, my anecdotal experience is that the 'extra' data makes a difference.  

If anyone has a different experience of this, I would like to know.

It makes sense that this is a result of obtaining data with a better s/nr at capture. But the 'why' doesn't really matter if the result is as described, does it?

- N.
Title: Re: Better than ETTR ?
Post by: Graeme Nattress on August 05, 2011, 03:36:48 pm
Michael, I know what you mean about different cameras behaving differently, and also, you're forced to view the data through a raw converter which may not be performing optimally for that camera. However I don't know off-hand any cameras which have less bit-depth than their noise floor - usually it's the other way around where the bit depth of the ADC exceeds the noise capabilities of the sensor.

Graeme
Title: Re: Better than ETTR ?
Post by: Graeme Nattress on August 05, 2011, 03:41:19 pm
"But the 'why' doesn't really matter if the result is as described, does it?" - not really. It's results that count :-) However, I always like to get to the bottom of things and I want to know "why?"

Basically - if there's insufficient bit depth (or not enough code values used to describe that "zone") we see banding or posterization. That is a bad situation to be in as we know that the ADC or (lack of) processing precision has lost us valuable image data. This is what the "less code values per stop in linear light data" argument says as to why we should ETTR. I don't know of a camera that performs this way.

The SNR argument says that if you can't see banding or posterization in the shadows, then the bit depth is sufficient given the noise floor of the sensor, and ETTR works by capturing more light and giving a better SNR.

Either way you ETTR - but the "why" is different.

Graeme
Title: Re: Better than ETTR ?
Post by: fdisilvestro on August 05, 2011, 06:37:47 pm
Supporting SNR as the main reason for ETTR:

1.- Suppose (as a reduction to absurd) a ridiculously large bit depth, almost approaching infinity. Then, any stop value will have more than enough values so that the argument about number of values looses validity. The only argument left is SNR.

2.- Take for example a camera like a Nikon DSLR where you could select either 12 or 14 bit RAW. The highest f/stop in 12 bits will have 2048 levels, the same as 2 stops under in 14 bits mode. Do you think you will get the same quality doing ETTR in 12 bits than 2 stops below-ETTR in 14 bits mode, just because you get the same number of levels? I don't think so.


Title: Re: Better than ETTR ?
Post by: pedro.silva on August 06, 2011, 05:13:32 am
I don’t think that works as well as you think.

Supporting number of levels as the main reason for ETTR:

1.- Suppose (as a reduction to absurd) a ridiculously small noise, approaching zero. Then, any stop value will have no noise, so that the argument about noise looses validity. The only argument left is number of values (levels).

2.- The highest f/stop would have no noise, but so would the lowest f/stop (and all others in between).  Do you think you would get the same quality doing ETTR as 2 stops below-ETTR mode, just because you get the same absence of noise? I don't think so (you would get a combed histogram, possibly leading to visible banding). 

What we need is sound theory that applies to actual current cameras, and these both produce noise and work with a fairly small number of levels.  The current balance suggests that the main reason ETTR works is by increasing SNR, but that could change in the future. 

Anyway, all of this may become moot if cameras ever expose different sensels for different amounts of time (recording the exposure times of course), for then all sensels would be exposed right (ER™) automagically with the lowest noise possible, and the number of levels would only matter in the final, converted, image. 

On a side note, Guillermo and others seem to think there is no merit in recording noise more accurately.  I think there may be.  The more accurately noise is recorded, the better job noise removers can do. 

Cheers!

Supporting SNR as the main reason for ETTR:

1.- Suppose (as a reduction to absurd) a ridiculously large bit depth, almost approaching infinity. Then, any stop value will have more than enough values so that the argument about number of values looses validity. The only argument left is SNR.

2.- Take for example a camera like a Nikon DSLR where you could select either 12 or 14 bit RAW. The highest f/stop in 12 bits will have 2048 levels, the same as 2 stops under in 14 bits mode. Do you think you will get the same quality doing ETTR in 12 bits than 2 stops below-ETTR in 14 bits mode, just because you get the same number of levels? I don't think so.
Title: Re: Better than ETTR ?
Post by: ErikKaffehr on August 06, 2011, 08:09:14 am
Hi!

Most of the noise is coming from the variation of the number of incident photons. So the only way to make a sensor noise free is to make it very large. Whatever you do, there will be noise, however. Even if you detect all photons, it is well possible that there is no photon to detect. For that reason a noise free sensor is not possible, because light by itself has noise. It comes in quantum size packages called photons. If thousands of photons hit each sensor cell we have good statistics, that is low noise, if only a few photons hit the sensel we have poor statistics and noise.

Best regards
Erik

I don’t think that works as well as you think.

Supporting number of levels as the main reason for ETTR:

1.- Suppose (as a reduction to absurd) a ridiculously small noise, approaching zero. Then, any stop value will have no noise, so that the argument about noise looses validity. The only argument left is number of values (levels).

2.- The highest f/stop would have no noise, but so would the lowest f/stop (and all others in between).  Do you think you would get the same quality doing ETTR as 2 stops below-ETTR mode, just because you get the same absence of noise? I don't think so (you would get a combed histogram, possibly leading to visible banding). 

What we need is sound theory that applies to actual current cameras, and these both produce noise and work with a fairly small number of levels.  The current balance suggests that the main reason ETTR works is by increasing SNR, but that could change in the future. 

Anyway, all of this may become moot if cameras ever expose different sensels for different amounts of time (recording the exposure times of course), for then all sensels would be exposed right (ER™) automagically with the lowest noise possible, and the number of levels would only matter in the final, converted, image. 

On a side note, Guillermo and others seem to think there is no merit in recording noise more accurately.  I think there may be.  The more accurately noise is recorded, the better job noise removers can do. 

Cheers!

Title: Re: Better than ETTR ?
Post by: cunim on August 06, 2011, 10:06:00 am
At ambient temperatures, the proportion of noise varies with signal level.  It is higher at low signal levels.  That is why we use cooling with high bit depth quantization.  Cooled sensors linearize SNR across a very broad dynamic range so deep shadows have similar noise characteristics to midrange data.  Cryogenic pumps or liquid nitrogen combined with slow readout make for some pretty spectacular images.  Even low levels of cooling help, which is why small scientific grade cameras often use some kind of Peltier cooler.

Photographic cameras aren't cooled, both because of cost/mass issues and because the balance of thermal and read noise is different (can't use slow readout).  So, we look for signal processing approaches to noise reduction.  ETTR is one such (analog) approach.  It moves shadows into the detection range where noise is proportionally lower.  Michael's observations make sense.  Of course, there is no free lunch.  ETTR is a reduction in dynamic range.

Amplification (high iso) vs noise reduction is a whole nuther topic.  With monochrome detectors, noise reduction is better until you get near the photon counting range, and then amplification before quantization becomes more sensitive (though less efficient).  I have never really understood how it works in photographic cameras.
Title: Re: Better than ETTR ?
Post by: Graeme Nattress on August 06, 2011, 10:21:57 am
"ETTR is a reduction in dynamic range" - if you're not clipping then you're maximizing DR.

Graeme
Title: Re: Better than ETTR ?
Post by: pedro.silva on August 06, 2011, 10:38:53 am
No major disagreement there!

I never meant to imply that noise free sensors are possible, any more than Francisco meant that infinite bit depth is. I simply tried to show what I saw as a logical flaw in his reasoning.

However... "the only way to make a sensor noise free is to make it very large": that depends on how you define "noise free".  According to Poisson stats, the larger the value the larger the noise too (only the SNR gets better).  And better quantum efficiency could lead to better SNR too, not just larger sensors, rite? 

Hi!

Most of the noise is coming from the variation of the number of incident photons. So the only way to make a sensor noise free is to make it very large. Whatever you do, there will be noise, however. Even if you detect all photons, it is well possible that there is no photon to detect. For that reason a noise free sensor is not possible, because light by itself has noise. It comes in quantum size packages called photons. If thousands of photons hit each sensor cell we have good statistics, that is low noise, if only a few photons hit the sensel we have poor statistics and noise.

Best regards
Erik

Title: Re: Better than ETTR ?
Post by: fdisilvestro on August 06, 2011, 11:35:31 am
Cooled sensors linearize SNR across a very broad dynamic range so deep shadows have similar noise characteristics to midrange data.  Cryogenic pumps or liquid nitrogen combined with slow readout make for some pretty spectacular images.  Even low levels of cooling help, which is why small scientific grade cameras often use some kind of Peltier cooler.


Correct me if I'm wrong, but as far as I know, cooling the sensor reduces the read noise, but there is nothing you could do about shoot noise, which is an inherent property of light. Since photon noise is proportional to the square root of the photon count, then the higher the photon count, the higher the signal to noise to noise ratio

Even if you achieve zero read noise, you still have shoot noise. ETTR still holds no matter the explanation you prefer: the more levels, the more photons, the better SNR, you can actually derive one from another.

Suppose you could count individual photons, then that´s your maximum required bith depth, maximum values will be the same as maximum photons and since there is a direct relation between photon count and SNR then it will also be maximum SNR
Title: Re: Better than ETTR ?
Post by: cunim on August 06, 2011, 12:07:03 pm
Graeme, no need to get into a discussion DR.  LOTS of that elsewhere.  I will just make two statements and you can decide if one limits DR.

1.  I have 12 bit precision across a 10 stop range.
2.  I have 12 bit precision across 8 stops of a 10 stop range (ETTR).

Francisco, shot noise is a property of the detector package.  You reduce shot noise with cooling.  You reduce read noise in various ways, primarily by slowing the readout.  True 16-bit cameras tend to read out at about 1 kHz.

Really, none of this has much to do with whether or not to use ETTR.  That depends on the exposure lattitude that you need.  I was just suggesting that it is reasonable for shadows to open up better when ETTR is used. 
Title: Re: Better than ETTR ?
Post by: Bart_van_der_Wolf on August 06, 2011, 12:44:45 pm
Correct me if I'm wrong, but as far as I know, cooling the sensor reduces the read noise, but there is nothing you could do about shoot noise, which is an inherent property of light. Since photon noise is proportional to the square root of the photon count, then the higher the photon count, the higher the signal to noise to noise ratio

Hi Francisco,

No correction needed, you a correct.

Cooling only helps to reduce the spontaneous generation of free electrons, and typically makes sense for longer exposure times than 1 second (assuming normal room temperatures). If the heat build-up inside the camera is significant enough (dark current doubles approx. every 6 degrees Celsius, or Kelvin) it makes sense to cool the sensor if it exhibits extremely low read noise, otherwse the dark current is swamped by the read noise anyway.

Shot noise is purely determined by the statistical properties of the photon flux.

Cheers,
Bart
Title: Re: Better than ETTR ?
Post by: PierreVandevenne on August 06, 2011, 01:06:33 pm
_Read_ noise is _marginally_ affected by temperature. Cooling a sensor to extremely low temperature will lower it very slightly, until the sensor simply stops to work :-)

Dark current and read noise are two different things. I am sure you know that Bart, but I find the whole thread a bit imprecise.
Title: Re: Better than ETTR ?
Post by: ErikKaffehr on August 06, 2011, 01:20:59 pm
Hi,

Shot noise has nothing to do with sensor, except what is known as full well capacity, the number of incident photons a sensor cell can account for.

ETTR is mainly about maximizing the number of photons detected, that is utilizing full well capacity. Regarding read out noise you are probably right.

The practical effects of ETTR are twofold:

1) Noise in midtones are reduced, that actually may also mean that you can sharpen better as sharpening also enhances noise.

2) SNR (Signal Noise Ratio) in the shadows will be improved, so shadows respond better to "opening up"

As a remark I don't see any benefit of going to full 16 bit signal path. Sensor FWC (full well capacities) are rather going down than up. With present FWCs the lower bits will be dominated by noise anyway. Shoot noise will be significant and readout and other noises will be added to that.

Best regards
Erik


Francisco, shot noise is a property of the detector package.  You reduce shot noise with cooling.  You reduce read noise in various ways, primarily by slowing the readout.  True 16-bit cameras tend to read out at about 1 kHz.

Really, none of this has much to do with whether or not to use ETTR.  That depends on the exposure lattitude that you need.  I was just suggesting that it is reasonable for shadows to open up better when ETTR is used. 

Title: Re: Better than ETTR ?
Post by: John Camp on August 07, 2011, 01:06:25 pm
Ok, does anyone disagree with the following:
When one exposes a zone II shadow as zone V (to use real photography terms  ;)) and then subsequently places that detail back down into zone II in post-processing, that shadow is much 'cleaner' than if it had simply been exposed at zone II to begin with.  No, there aren't any more tonal values in the shadow areas this way, but there is a qualitative difference.
The same results hold true in a slightly different way for higher tonal values where noise is less of an issue with a 'straight' capture to begin with. But there, too, my anecdotal experience is that the 'extra' data makes a difference.
If anyone has a different experience of this, I would like to know.
It makes sense that this is a result of obtaining data with a better s/nr at capture. But the 'why' doesn't really matter if the result is as described, does it?
- N.

"But the 'why' doesn't really matter if the result is as described, does it?" - not really. It's results that count :-) However, I always like to get to the bottom of things and I want to know "why?" <BIG SNIP>Either way you ETTR - but the "why" is different.

Graeme

Okay, this is what I was getting at in the other thread -- the prescriptive mode. "Either way, you ETTR."

So generally, if I ETTR, at least modestly, checking the histogram to try to make sure that I haven't blown any channels, I'll USUALLY be better off than if I just go ahead and plug away on auto-exposure (assuming that I then take the RAW image into LR for adjustment.) Right? I think that's what Michael was saying in the original article, and that's all I was really trying to determine.




Title: Re: Better than ETTR ?
Post by: Wayne Fox on August 07, 2011, 03:30:03 pm
Graeme, no need to get into a discussion DR.  LOTS of that elsewhere.  I will just make two statements and you can decide if one limits DR.

1.  I have 12 bit precision across a 10 stop range.
2.  I have 12 bit precision across 8 stops of a 10 stop range (ETTR).


over simplistic and misleading.

ETTR is not about the dynamic range of your sensor.  It is about improving the quality of the data captured by the sensor by utilizing a different methodology in determining exposure settings than those built into cameras, which as Michael pointed out are engineered for use based on the characteristics of film dating back many decades. The dynamic range of the scene and that of the sensor to record the scene are what they are.

If in fact you can open your camera up two stops without clipping any highlights from the exposure setting your camera recommends and you end up with room to the left of your histogram, this simply means the scene itself has less dynamic range than your sensor, it doesn't mean you have limited your cameras ability to capture DR.

Additionally, if you can open your camera up two stops without clipping, and your data still extends all the way to the left of your histogram this implies you have more fully utilized the DR of your sensor, not decreased it, since some of the clipped shadow data is no longer clipped ... in effect you have increased the DR of your capture.
Title: Re: Better than ETTR ?
Post by: Guillermo Luijk on August 07, 2011, 07:12:26 pm
On a side note, Guillermo and others seem to think there is no merit in recording noise more accurately.  I think there may be.  The more accurately noise is recorded, the better job noise removers can do.
Can you show us an example where a noise remover did a better job thanks to having more levels?.

On the other hand, if you have more levels through ETTR, you also have better SNR, so talking about noise removers doing a better job is quite surrealistic because you cannot have more levels but the same SNR! ;D

In the article: DO RAW BITS MATTER? (http://www.guillermoluijk.com/article/rawbits/index.htm) I developed some RAW files with a decreasing number of bits (i.e. I rounded RAW numbers before demosaicing to emulate ADC with less bits).

For the Canon 40D, 12bits showed to be enough. The extra 2 bits didn't improve useful information recorded:

Canon 40D, 14 bits RAW development
(http://www.guillermoluijk.com/article/rawbits/40d_1.jpg)

Canon 40D, 12 bits RAW development
(http://www.guillermoluijk.com/article/rawbits/40d_2.jpg)


But the Pentax K5, being a much lower noise camera, really needed the 14 bits. A 12-bit RAW development started to display posterization in the deep shadows:

Pentax K5, 14 bits RAW development
(http://www.guillermoluijk.com/article/rawbits/k5_1.jpg)

Pentax K5, 12 bits RAW development
(http://www.guillermoluijk.com/article/rawbits/k5_2.jpg)
Title: Re: Better than ETTR ?
Post by: cunim on August 07, 2011, 07:20:02 pm
over simplistic and misleading.


Story of my life.  I plead guilty to being simplistic, but misleading - I don't think so.  

If I may summarize my reading of this thread - ETTR stays below clipping while raising the shadows.  People see an improvement for the shadow data as a result.  This discussion centers on why that improvement occurs.  My own belief is that signal is proportionally higher than noise (various forms of electrical noise, flare, etc.) in the shifted data.  Pretty simple.

The dynamic range compression is peripheral to all this but I will try to make up for the simplistic bit.  Without ETTR a camera might give us a 1000:1 range between white and black clipping - at our desired level of precision.  With ETTR it might be 800:1 because we concede that shadow areas are not as good as we would like.  Of course, the camera response does not change.  It still covers the same range it always did so I can understand the position that the device DR is the same.  However, the image luminance range is not.  It is narrower.  You can't use ETTR with full range data.

Contrast the photographic camera with a scientific imager in which it is possible to tweak various physical parameters to move all of your precision into a narrower range.   It seems to me that ETTR is the same in principle.  We can't alter the response of the camera to achieve higher precision over a limited range, but we can still see some benefits from altering exposures to exclude the bottom of the response curve.  To me, that is a reduction in DR.  I agree that the reduction should not matter as long as our images only contain the norrower range of data.





  
Title: Re: Better than ETTR ?
Post by: Ray on August 07, 2011, 11:39:31 pm
Here we are again on a perennially popular topic with an acronym suggestive of aliens from  outer space.  ;D

Getting the best and most appropriate exposure for a particular shot has always been a basic technical requirement for serious photographers.

However, it needs to be stressed that the best exposure is not necessarily the exposure which maximises the photon count and produces the lowest noise in the shadows, ie. an ETTR.

The conditions for an ETTR are generally constrained by DoF requirements, subject movement and the intensity of available light (in the absence of flash).

In fact, I would say that achieving an ETTR, in the sense of maximising the photon count, may be the last consideration.

Choosing the appropriate aperture for the desired DoF, and a shutter speed sufficient to freeze both camera and subject movement, is surely of greater priority.

Only after having selected an appropriate shutter speed and aperture should one then address the implications of ETTR, which may mean increasing shutter speed at base ISO to avoid overexposure, or increasing ISO. With a camera like the D7000 or K5, there's really no need to increase ISO. If the desired aperture and minimum shutter speed for a sharp result, also result in an underexposure at base ISO, then so be it. It can't be helped.

Of course, if one has the luxury of time on one's side, if the subject is static, and the camera is on tripod, then there is surely no problem regarding 'correct' exposure.

The problem of ETTR arises when one doesn't have sufficient time to manually get the settings right for a particular scene because one is trying to 'capture the moment'. In these circumstances, an adjustable feature in the camera that would guarantee an ETTR could be useful.

However, such a feature would also have its own problems. It would be another camera adjustment to get right, and when it wasn't right, the shot might be ruined.

To give you an example. Supposing the automatic ETTR exposure adjustment was set at its maximum setting which ignores very small percentages of the frame, which we would describe as specral highlights. Supposing you are walking through the wilderness and suddenly encounter a rare or interesting bird, or animal, in the shade of some foliage, with significant areas of sky visible through the foliage.

I would suggest there would be two likely results. (1) You forget your camera is set to ignore only specral highlights for an ETTR, take the shot, the bird flies away at the sound of the shutter, and you are left with a lovely shot of deep blue sky shining through the foliage, and a noisy bird in the shadows.

(2) You remember the ETTR setting is not appropriate for the shot, and try to estimate the area of the frame that consists of bright sky which is of secondary importance, so you can make the appropriate adjustment in the camera's ETTR feature. Alas! The bird has noticed you and flies away before you take the shot.

The only solution I see to this problem is to bracket every shot you take, unless you have the luxury of time on your hands to be confident you have made the right settings for an ETTR.

There can be other advantages of bracketing exposure (or ISO) when trying to capture the moment. One of the other shots, either the overexposure or the underexposure, may be preferred irrespective of its degree of closeness to the ideal ETTR, simply because the subject has moved to a more interesting position, or adopted a more interesting facial expression during that one whole second that many cameras need for 3 frames.
Title: Re: Better than ETTR ?
Post by: Wayne Fox on August 08, 2011, 01:04:48 am
The dynamic range compression is peripheral to all this but I will try to make up for the simplistic bit.  Without ETTR a camera might give us a 1000:1 range between white and black clipping - at our desired level of precision.  With ETTR it might be 800:1 because we concede that shadow areas are not as good as we would like.  Of course, the camera response does not change.  It still covers the same range it always did so I can understand the position that the device DR is the same.  However, the image luminance range is not.  It is narrower.  You can't use ETTR with full range data.
It is no secret that at black clipping we have weak data.  If our dynamic range is less than the sensor, moving all the data to the right records that data with more precision and less noise.  We do not lose any data in the highlights.  The data is not full range, that's why you can use EttR.  No one ever stated that you should use EttR with every image you take, and I believe anyone who uses it will say it has little use if the DR of the scene equals or exceeds the DR of the sensor.  However, in a great many shooting situations with sensor dynamic ranges of todays cameras, you will have plenty of head room to move your data to the right.  Yes, there may be no data at the black clipping point, but so what? You still have the same relationship of the white  point of the scene to the black  point of the scene.  The fact they don't match the clip points of the sensor really doesn't matter.



Here we are again on a perennially popular topic with an acronym suggestive of aliens from  outer space.  ;D

Getting the best and most appropriate exposure for a particular shot has always been a basic technical requirement for serious photographers.

However, it needs to be stressed that the best exposure is not necessarily the exposure which maximises the photon count and produces the lowest noise in the shadows, ie. an ETTR.

The conditions for an ETTR are generally constrained by DoF requirements, subject movement and the intensity of available light (in the absence of flash).

In fact, I would say that achieving an ETTR, in the sense of maximising the photon count, may be the last consideration.

Choosing the appropriate aperture for the desired DoF, and a shutter speed sufficient to freeze both camera and subject movement, is surely of greater priority.

Only after having selected an appropriate shutter speed and aperture should one then address the implications of ETTR, which may mean increasing shutter speed at base ISO to avoid overexposure, or increasing ISO. With a camera like the D7000 or K5, there's really no need to increase ISO. If the desired aperture and minimum shutter speed for a sharp result, also result in an underexposure at base ISO, then so be it. It can't be helped.

Of course, if one has the luxury of time on one's side, if the subject is static, and the camera is on tripod, then there is surely no problem regarding 'correct' exposure.

The problem of ETTR arises when one doesn't have sufficient time to manually get the settings right for a particular scene because one is trying to 'capture the moment'. In these circumstances, an adjustable feature in the camera that would guarantee an ETTR could be useful.

However, such a feature would also have its own problems. It would be another camera adjustment to get right, and when it wasn't right, the shot might be ruined.
I don't think anyone suggests that EttR has replaced aperture/shutter speed/ISO as the priority settings in determining an exposure.  All that is suggested is that an exposure based on EttR often will leave you with better raw data than the setting chosen by your camera, if you have the headroom and don't have to compromise those other areas.

Unfortunately there is absolutely no easy way to use EttR with current cameras.  You have to shoot, examine histo, and adjust (and your guessing at that because the histo isn't telling you enough about the raw data).  But using metering/exposure techniques designed for non-linear film is also not optimal.  It works because it almost always errs on the underexposure side, and certainly there are times and photographic situations where speed is critical.  But even in somewhat time sensitive shooting situations, an exposure system that based it's estimate on EttR would make it effective in more situations than one might think.

It does seem that designing a metering system based on EttR might have been challenging until LiveView was introduced. But it seems quite possible now, and as suggested in the article could certainly be included as an option.  Cameras with live histograms make EttR really quite easy, seems some engineer could analyze the histogram in firmware and recommend or choose a setting based on that much like I can by viewing the histo and changing the settings on my point and shoot. Of course first the camera makers need to understand that histogram's based on the raw data are important, and at least provide that as an option.
Title: Re: Better than ETTR ?
Post by: Bart_van_der_Wolf on August 08, 2011, 04:33:13 am
Can you show us an example where a noise remover did a better job thanks to having more levels?.

I don't have a camera with variable bit-depth, but I can imagine that as the precision of the noise quantization improves, it also becomes a little easier to remove the noise. The difference will be marginal though, so it might benefit e.g. astrophotography (by averaging multiple calibrated exposures) more than, say, sports photography.

Quote
In the article: DO RAW BITS MATTER? (http://www.guillermoluijk.com/article/rawbits/index.htm) I developed some RAW files with a decreasing number of bits (i.e. I rounded RAW numbers before demosaicing to emulate ADC with less bits).

Yes, that's an interesting article.

Quote
For the Canon 40D, 12bits showed to be enough. The extra 2 bits didn't improve useful information recorded:

Actually I think the difference is noticeably more smooth without the coarser quantization posterization, but perhaps I am hyper-sensitive to it. It was the same when I switched from the EOS 1Ds2 to the 1Ds3. There may be other factors involved, but  I was struck by how smooth the gradients had become. But then I am also one of the unhappy few who can sometimes see the increase in noise from underexposing by 1/3rd EV. Let's call it a curse ...

But quantization precision (bit depth) is only marginally relevant to the ETTR concept, which is all about improving the S/N ratio in all tonal areas of the image. That also means that relatively low scene contrast offers the best potential because an averaging metering will technically 'underexpose' the highlights more and offer some headroom. A high contrast scene will only add a dilemma about how much of the specular highlights we can sacrifice to keep the exposure level at the mid- and lower tones as high as possible. In that case bracketing may be the best solution if the scene dynamics allow.

Cheers,
Bart
Title: Re: Better than ETTR ?
Post by: Ray on August 08, 2011, 08:05:52 am
Unfortunately there is absolutely no easy way to use EttR with current cameras.  You have to shoot, examine histo, and adjust (and your guessing at that because the histo isn't telling you enough about the raw data).

That's not quite true, Wayne. There is a reasonably easy way of using ETTR without experimenting first with a shot then examining the histogram.

Set the camera to spot meter mode and full manual mode. Take a reading of what you see as the brightest part of the scene, for example a white cloud, a blue sky, a galvanised iron roof, a white wall, a white shirt etc., then increase exposure by a certain number of stops depending on the model of camera.

As I recall with my Canon 5D, that increase was 3 stops which is easy to calculate. With some makes of camera it might be 2.5 stops, slightly more difficult to calculate.

I also recall being amazed at how accurate this method could be. But again it's not ideally suitable when a shot has to be taken quickly to capture the moment. For this reason I haven't persevered with the method, but I think with practice one could get to the stage of being very quick and accurate in locating the brightest part of a scene and making the necessary exposure increase with one's thumb on the wheel without moving the camera from one's eye.
Title: Re: Better than ETTR ?
Post by: digitaldog on August 08, 2011, 10:32:08 am
I don't think anyone suggests that EttR has replaced aperture/shutter speed/ISO as the priority settings in determining an exposure.  All that is suggested is that an exposure based on EttR often will leave you with better raw data than the setting chosen by your camera, if you have the headroom and don't have to compromise those other areas.

Exactly! Whenever we run into these ETTR discussions, some post theoretical problems with the practice (like this one concerning DOF), or how it clips highlights (not highlights you want to retain IF you ETTR correctly) or how people will now ruin their images due to slower shutter speeds to gain more exposure. Its as if suggesting ETTR implies all the aspects of sound image capture we’ve been practicing for 100+ years is to be ignored or isn’t valid any longer. ETTR is about idealized exposure for raw data when you have the time and desire for idealized data, all other photographic practices still in effect.
Title: Re: Better than ETTR ?
Post by: fike on August 08, 2011, 11:08:57 am
My experience with EttR has been mixed.  I have found that sometimes when I have a very narrow dynamic range (flat light), I can make a crummy lighting situation better by having more levels in the high range to be stretched in post processing. 

On the other hand, I find more frequently that I don't like the results (particularly in skies). Sometimes this is because I have used the luminance histogram and missed the fact that I actually clipped one of the channels and not the others.  This often creates unattractive cyan highlights in skies...yes, yes, I was too aggressive with EttR. I know that, but it can be easy to miss if you don't do some substantial experimenting with exposure and the RGB and luminance histogram. 

When you have time, photographers should substantially underexpose the image to see if there are any spikes way off to the right of the graph.  Of course, we should be able to see this with our eyes, but generally too much dallying around means you miss the shot. 

As a woodland photographer, shadows are key, but perhaps more key is not blowing out the very small areas of light filtering between leaves.  I have learned to accept that sometimes blacks are a part of the image.  Going to extremes to rescue detail in those blacks doesn't always enhance the image particularly if I find very small highlights blooming in the branches of the trees.  Yes, again I recognize that I was too aggressive with EttR, but this is my main point that when I have used EttR in woodland photography, I very frequently blow the shot with EttR. 

I think someone said it above...If the dynamic range of the image exceeds the dynamic range capability of the camera (always in woodlands photography) then EttR doesn't help.  When dynamic range is less than the range of the camera, EttR can vastly improve an image.  The trouble is that in the moment of capture, it can be easy to misjudge these things.
Title: Re: Better than ETTR ?
Post by: Ray on August 08, 2011, 11:53:47 am
Its as if suggesting ETTR implies all the aspects of sound image capture we’ve been practicing for 100+ years is to be ignored or isn’t valid any longer. ETTR is about idealized exposure for raw data when you have the time and desire for idealized data, all other photographic practices still in effect.

If you have the time and desire for idealised data, there should be no problem at all in this digital age. Michael in his recent article on the topic of ETTR was lamenting the fact that manufacturers have not yet designed a camera that will guarantee an automatic ETTR. That's where the difficulty lies, when you might miss the shot due to the time it takes to work out the settings for an idealised exposure.
Title: Re: Better than ETTR ?
Post by: digitaldog on August 08, 2011, 12:23:40 pm
Michael in his recent article on the topic of ETTR was lamenting the fact that manufacturers have not yet designed a camera that will guarantee an automatic ETTR.

If they would even make an attempt, provide a useful histogram I’d be happy. I’m not expecting any guarantees, just an effort.
Title: Re: Better than ETTR ?
Post by: ErikKaffehr on August 08, 2011, 04:50:14 pm
Hi,

After reading Michael's original article on ETTR my approach has been to maximize the exposure, using the histogram. Fortunately enough the Sony's I have now have both RGB and luminance histograms and blinking zones for over and under exposure. I essentially found both quite reliable.

In my view ETTR means is that we try to utilize the full histogram, thus essentially optimizing available DR and minimizing noise relative to signal. So I don't really see how the DR of the subject matters, whatever the tonal range of the subject we want a "full histogram without clipping".

For me, ETTR essentially mean that we don't care about the placement of midtones, because that will be done later in postprocessing. Quite obviously, DR is maximal at minimum ISO, and that is what I essentially always use when possible. My experience has been that DR is almost never an issue for me, except perhaps when shooting inside building and wanting to handle shadows detail and window detail. In those situation I'd say HDR may be a good recourse.

I started do some more testing on ETTR after reading Michaels article, but essentially found that I get into clipping if I try to move outside the in camera histogram.

I'd also say that i absolutely concur with Michael's suggestion for an ETTR option on modern cameras. Especially on modern EVIL cameras it should really be a piece of cake. On the other hand I guess that most of the processing in the cameras is done in the ASICs (Application Specific Integrated Circuit) and additional functionality may not always be easy to add.

Best regards
Erik


My experience with EttR has been mixed.  I have found that sometimes when I have a very narrow dynamic range (flat light), I can make a crummy lighting situation better by having more levels in the high range to be stretched in post processing. 

On the other hand, I find more frequently that I don't like the results (particularly in skies). Sometimes this is because I have used the luminance histogram and missed the fact that I actually clipped one of the channels and not the others.  This often creates unattractive cyan highlights in skies...yes, yes, I was too aggressive with EttR. I know that, but it can be easy to miss if you don't do some substantial experimenting with exposure and the RGB and luminance histogram. 

When you have time, photographers should substantially underexpose the image to see if there are any spikes way off to the right of the graph.  Of course, we should be able to see this with our eyes, but generally too much dallying around means you miss the shot. 

As a woodland photographer, shadows are key, but perhaps more key is not blowing out the very small areas of light filtering between leaves.  I have learned to accept that sometimes blacks are a part of the image.  Going to extremes to rescue detail in those blacks doesn't always enhance the image particularly if I find very small highlights blooming in the branches of the trees.  Yes, again I recognize that I was too aggressive with EttR, but this is my main point that when I have used EttR in woodland photography, I very frequently blow the shot with EttR. 

I think someone said it above...If the dynamic range of the image exceeds the dynamic range capability of the camera (always in woodlands photography) then EttR doesn't help.  When dynamic range is less than the range of the camera, EttR can vastly improve an image.  The trouble is that in the moment of capture, it can be easy to misjudge these things.
Title: Re: Better than ETTR ?
Post by: douglasf13 on August 08, 2011, 07:13:00 pm
As Ken mentioned in the other thread, what about color issues when you expose mid tones too far right?

Title: Re: Better than ETTR ?
Post by: Bart_van_der_Wolf on August 08, 2011, 08:02:55 pm
As Ken mentioned in the other thread, what about color issues when you expose mid tones too far right?

Hi Douglas,

Mid-tones shouldn't be a problem with a good Raw converter. With Adobe ACR or Lightroom it can apparently depend on the camera profile. Of course when you clip too much of the highlight data, it may be impossible to reconstruct the highlight color.

Cheers,
Bart
Title: Re: Better than ETTR ?
Post by: douglasf13 on August 08, 2011, 08:19:55 pm
  Andrey Tverdokhleb and Iliah Borg, the developers of Raw Photo Processor, say color is very much an issue when exposing past mid tones, at least with most newer cameras. To quote Andrey in reference to the Sony A900 on another forum:

"No, I meant using camera light meter (or external one if you care), but don't push histogram to the right and use camera light meter as it's intended, i.e. expose most important part of a picture around camera midpoint.  ETTR had some reasons for old cameras with low DR - noise was too close to the midpoint and we had to do this to minimize it. With late cameras which have over 9 stops of DR ETTR is very harmful for colors - midpoint is the most colorful place in A900 gamut and noise is not an issue there any more. In A900 gamut slowly narrowing down from midpoint to shadows and very quickly narrowing down from midpoint to highlights. This means that brightest stop of the camera range has most of colors gone forever and they cannot be restored with negative exposure compensation. I'd say all color critical parts should be below top 1.5 stops. Veiling glare from lens and sensor are the culprits here."

"Squeezing scene with high DR into sensor or film range is a totally valid approach when needed, same as exposing for shadows. ETTR however assumes that it's always better to shift histogram to the right, even when your scene is only 6 stops wide and sensor is 9.5 stops wide. You probably already noticed before that slightly underexposed shots can be amazingly colorful even after exposure correction and I definitely noticed that ETTR shots can be very dull after correction even if there was no clipping. So my point is that ETTR is not always better and shouldn't be used unconditionally. The whole approach that you need to pay attention only to highlights is very limiting - what's really important is were we place critical part of a picture on a sensor range. This critical part can be anywhere - in shadows, highlights or in the middle and we should understand that moving it closer to the middle gray will improve it's appearance and try our best. That's it "

"Regarding ETTR I already mentioned this, but let me rephrase - ETTR as universal approach for all kinds of shooting is wrong and unfortunately most of people treat it this way no matter what and how they shoot. It's easy to understand and there is even some technical explanation behind it, but in fact it doesn't tell the whole story. ETTR as approach when we are trying to open shadows without clipping highlights is a valid technique when needed and as long as we understand what we gain and what we loose there is absolutely nothing wrong with that."


Iliah and Andrey have managed to put together a converter with better output than most of the other expert's converters, and I find their opinion compelling on this issue of ETTR.
Title: Re: Better than ETTR ?
Post by: marcmccalmont on August 08, 2011, 08:56:26 pm
Are there any graphs published that show gamut vs level at various iso's? to both confirm this and optimize your exposure?
Marc
Title: Re: Better than ETTR ?
Post by: Ray on August 08, 2011, 10:06:35 pm
In my view ETTR means is that we try to utilize the full histogram, thus essentially optimizing available DR and minimizing noise relative to signal.

Erik,
I think we should distinguish between the two types of camera designs regarding higher-than-base ISO performance, and between the two methods of shooting regarding available time to plan the shot and use tripod.

With the D7000 and a few other models, including most MFDBs I think (at least the older models), there is actually only one concern regarding attempts to achieve that 'idealised amount of RAW data', as Andrew put it, and that concern is overexposure at base ISO.

This simplifies the ETTR situation significantly for hand-held shots and/or moving subjects whenever it's clear that the desired combination of shutter speed and aperture in the available light requires an increase in ISO.

With Canon cameras, an ETTR at ISO 400 produces a better SNR than the same shot showing an underexposure of 2 stops at base ISO. With the D7000 it doesn't, not because ISO 400 is worse on the D7000, but because ISO 100 is better.

In such circumstances, there is no need to 'try to utilize the full histogram' as you put it. One can happily shoot all day in manual mode at base ISO and be completely free of any obsession with ETTR. The only concern is overexposure, and shutter speed can easily be changed without moving the camera from one's eye.

As I recall, the first DSLRs that were available had a rather poor DR compared with negative film, and especially B&W film. This factor, in combination with a digital sensor's lack of the smooth roll-off or toe which is characteristic of film, created problems for the first DSLR users, of a lack of 'headroom' regarding exposure.

Attempts to maximise the DR in the capture would often lead to the clipping of at least one color in the highlights. Playing it safe could result in disturbing noise in the shadows and lower midtones.

In my view we no longer have to be so obsessive about ETTR when using modern DSLRs, not just because the DR and SNR characteristics of modern DSLRs are much improved, but also because memory is now so cheap.

When in doubt, bracket exposure, or bracket ISO in circumstances where there is a risk that the 'overexposed' shot, which may turn out to be the ideal exposure, has too slow a shutter speed to freeze movement.

Title: Re: Better than ETTR ?
Post by: Schewe on August 09, 2011, 12:17:53 am
"Regarding ETTR I already mentioned this, but let me rephrase - ETTR as universal approach for all kinds of shooting is wrong and unfortunately most of people treat it this way no matter what and how they shoot. It's easy to understand and there is even some technical explanation behind it, but in fact it doesn't tell the whole story. ETTR as approach when we are trying to open shadows without clipping highlights is a valid technique when needed and as long as we understand what we gain and what we loose there is absolutely nothing wrong with that."

I have no problem with the above...I don't suggest ALWAYS doing ETTR...only when it's appropriate. But, I don't really agree with the warnings regarding color. I've not experienced that with ACR/LR. Course, I'm pretty good at adjusting both tone and color with both. Can't comment on Raw Photo Processor cause I've never used it...maybe it's more of a problem for Raw Photo Processor than ACR/LR?

Look, if you know what you are doing, you'll know when and how to use ETTR...if you don't, go right ahead and flail about like regular people and leave image quality on the table–you're choice.
Title: Re: Better than ETTR ?
Post by: Ray on August 09, 2011, 12:42:24 am
But, I don't really agree with the warnings regarding color. I've not experienced that with ACR/LR.

Nor have I, and I agree that such issues depend on one's skill with ACR adjustments and post processing in Photoshop.

For example, I find it easier to create a dramatic sky in post processing if I've underexposed the sky at least a little, probably because I need more skill with Photoshop.

I can also appreciate that it may be the case if just one channel has been clipped in a highlight area, even in small parts of the area, and reconstruction of that clipped channel has taken place in ACR, then the colors may be slightly different as a consequence.

The most obvious example I can think of is a blue sky where the red channel is partially clipped. The result is a sky that can look more cyan than it should.
Title: Re: Better than ETTR ?
Post by: Guillermo Luijk on August 09, 2011, 03:50:25 am
The most obvious example I can think of is a blue sky where the red channel is partially clipped. The result is a sky that can look more cyan than it should.
It's difficult that the red channel clips in the RAW file. If it does, the green channel will be clipped as well most of the times. So surely if the red channel got clipped and not the green channel, it happened at post processing.

This scene:
(http://www.guillermoluijk.com/article/spot/nube.jpg)

had the following RAW histogram, where the green channel was more than 1 stop ahead of the red channel:
(http://www.guillermoluijk.com/article/spot/nube.gif)
(Medición puntual = spot metering)

A Canon 5D was used to spot meter the higlights of the scene. The RAW histogram shows at least 3 stops of headroom before saturation.

More spot metering examples here (http://www.guillermoluijk.com/article/spot/index.htm).
Title: Re: Better than ETTR ?
Post by: Ray on August 09, 2011, 06:48:02 am
It's difficult that the red channel clips in the RAW file. If it does, the green channel will be clipped as well most of the times. So surely if the red channel got clipped and not the green channel, it happened at post processing.


You may be right. When the sky turns to cyan, both the red and green channels may be clipped, but I get the impression the red channel may be clipped first, or to a greater degree.

Here's an image of a temple in Kathmandu, with blown sky. The sky is brighter on the right, and more blown. Taking readings from the left-hand corner I get values of 134, 151, 197.

From the right-hand corner, I get readings of 207, 227, 251. The percentage of red on the left is greater, but also the percentage of green. The percentage of blue on the right is greater.  However, the reduction of the percentage of red on the right is greater than the reduction in the percentage of green.
Title: Re: Better than ETTR ?
Post by: Guillermo Luijk on August 09, 2011, 09:23:29 am
Taking readings from the left-hand corner I get values of 134, 151, 197. (...)

Readings from the RAW developer are irrelevant, you don't know what happened to your RAW values before they were displayed (exposure correction, white balance, highlight (recovery) strategies,...). Just open your RAW file into Rawnalyze and inspect the genuine RAW histograms to find out.

A RAW developer is not a tool suited to analyze RAW files, it is a tool designed to develop them.

Regards
Title: Re: Better than ETTR ?
Post by: fike on August 09, 2011, 09:27:53 am
Skies shifting towards cyan is my major problem with EttR.  Of course this means that I did EttR WRONG, but regardless it is an occupational hazard of the strategy that you will occasionally misjudge and clip one color channel while the general luminance histogram looks pretty good.
Title: Re: Better than ETTR ?
Post by: Eric Myrvaagnes on August 09, 2011, 09:32:06 am
Look, if you know what you are doing, you'll know when and how to use ETTR...if you don't, go right ahead and flail about like regular people and leave image quality on the table–you're choice.
That sums it up perfectly for me. Thanks, Jeff.
Title: Re: Better than ETTR ?
Post by: Ray on August 09, 2011, 10:02:32 am
Readings from the RAW developer are irrelevant, you don't know what happened to your RAW values before they were displayed (exposure correction, white balance, highlight (recovery) strategies,...). Just open your RAW file into Rawnalyze and inspect the genuine RAW histograms to find out.

A RAW developer is not a tool suited to analyze RAW files, it is a tool designed to develop them.

Regards


That's true. I don't know what happens to my RAW files before they are displayed. I get the impression that attempts are made by ACR to reconstruct any clipped channels. In the case of grey clouds, I believe even greater reconstruction of clipped channels is possible.
However, the fact is that ACR and Photoshop are the programs I use to develop my images. I don't see much advantage in using one program to analyse my RAW images and another program to develop them.
Title: Re: Better than ETTR ?
Post by: Bart_van_der_Wolf on August 09, 2011, 10:09:24 am
However, the fact is that ACR and Photoshop are the programs I use to develop my images. I don't see much advantage in using one program to analyse my RAW images and another program to develop them.

Hi Ray,

It might teach a valuabe lesson (without having to repeat it for every file), e.g. that Highlight Recovery should only be used after an adjustment of the exposure (and perhaps the brightness) slider(s). When Rawnalyse tells you that there are no clipped highlights, then why use the HR tool?

Cheers,
Bart
Title: Re: Better than ETTR ?
Post by: fdisilvestro on August 09, 2011, 11:57:53 am
I think it is very useful to know if the original RAW file has clipped channels or not. The behavior of LR / ACR depends on that. If there are clipped values, the highlight recovery is automatically invoked when applying negative exposure values with the exposure slider.

See this old LuLa thread (http://www.luminous-landscape.com/forum/index.php?topic=36161.msg296848#msg296848) with interesting info about this (from that message and the following ones)

Other things to consider:

-As far as I know, LR / ACR applies R=G=B for highlight recovery, which is rather limited in some cases (i.e. color propagation may be a better strategy in some situations).

-Some cameras have different saturation levels for each channel at base ISO, this happens to most Nikons. For example, the D300 at base ISO, the green channel clips at about 3830 (12 bits) instead of the supposed 4096. If this is not handled properly, clipped highligts will not look neutral. (LR / ACR deals fine with this)
Title: Re: Better than ETTR ?
Post by: douglasf13 on August 09, 2011, 12:06:51 pm
I have no problem with the above...I don't suggest ALWAYS doing ETTR...only when it's appropriate. But, I don't really agree with the warnings regarding color. I've not experienced that with ACR/LR. Course, I'm pretty good at adjusting both tone and color with both. Can't comment on Raw Photo Processor cause I've never used it...maybe it's more of a problem for Raw Photo Processor than ACR/LR?

Look, if you know what you are doing, you'll know when and how to use ETTR...if you don't, go right ahead and flail about like regular people and leave image quality on the table–you're choice.

I don't want to tug on Superman's cape here or anything with you, Jeff, but we're talking about Iliah Borg sharing these color opinions.  If anything, it may because of he has more discerning color expectations than you do, and I only mean that in regards to him probably having higher color expectations than ANYONE, and so it's tough to draw the line between theoretical and practice.  FWIW, RPP is what most of us use when we decide that ACR is lacking in terms of both color and detail, and the output difference between the two is quite remarkable, albeit RPP is much more difficult to use.  I own Lightroom, and use it for cataloguing and quick processing, but I send the raw to RPP for more critical use.  You certainly should give it a try sometime.
Title: Re: Better than ETTR ?
Post by: Schewe on August 09, 2011, 12:42:15 pm
If anything, it may because of he has more discerning color expectations than you do, and I only mean that in regards to him probably having higher color expectations than ANYONE, and so it's tough to draw the line between theoretical and practice.

Hum...really? I actually got a chuckle out of that.
Title: Re: Better than ETTR ?
Post by: deejjjaaaa on August 09, 2011, 01:03:40 pm
Hum...really? I actually got a chuckle out of that.

how do you know that do not "leave image quality on the table" (c) Schewe if you did not try RPP ?
Title: Re: Better than ETTR ?
Post by: Schewe on August 09, 2011, 01:29:43 pm
how do you know that do not "leave image quality on the table" (c) Schewe if you did not try RPP ?

I have no interest in testing/using other 3rd party raw processors...I'm kinda invested in ACR/LR ya know?
Title: Re: Better than ETTR ?
Post by: deejjjaaaa on August 09, 2011, 01:55:35 pm
I have no interest in testing/using other 3rd party raw processors...I'm kinda invested in ACR/LR ya know?
nobody doubts the business side of that...
Title: Re: Better than ETTR ?
Post by: douglasf13 on August 09, 2011, 02:17:57 pm
One would think that trying other 3rd party converters would help in making the one you're involved with better. I'm a paying customer of LR3, but I still export raws to RPP all the time, and it certainly isn't because I want to add more steps to my workflow. Granted, I guess if I'd never tried RPP in the first place, my ignorance would be bliss.

Honestly, it makes me a little nervous if the people working with Adobe haven't thoroughly investigated other converters like RPP, as I've been hoping they'd use it as an example on how to improve their converter in the future. My mistake.
Title: Re: Better than ETTR ?
Post by: Schewe on August 09, 2011, 02:45:27 pm
Honestly, it makes me a little nervous if the people working with Adobe haven't thoroughly investigated other converters like RPP, as I've been hoping they'd use it as an example on how to improve their converter in the future. My mistake.

Actually, it's not at all unusual to specifically NOT use other products in the tech industry...makes it a lot easier to testify that another product had no influence in the development of a competing product. Same reason why I've personally never tested anything from NIK software while being involved in the development of PhotoKit Sharpener (still haven't).
Title: Re: Better than ETTR ?
Post by: Alan Goldhammer on August 09, 2011, 02:45:42 pm
One would think that trying other 3rd party converters would help in making the one you're involved with better. I'm a paying customer of LR3, but I still export raws to RPP all the time, and it certainly isn't because I want to add more steps to my workflow. Granted, I guess if I'd never tried RPP in the first place, my ignorance would be bliss.

Honestly, it makes me a little nervous if the people working with Adobe haven't thoroughly investigated other converters like RPP, as I've been hoping they'd use it as an example on how to improve their converter in the future. My mistake.
It might be nice if the developers would port RPP to Win7 so the other half of the photo world could give it a try.  Until that happens, I'm an Adobe captive (though there is nothing wrong with that mind you).
Title: Re: Better than ETTR ?
Post by: douglasf13 on August 09, 2011, 03:21:08 pm
Actually, it's not at all unusual to specifically NOT use other products in the tech industry...makes it a lot easier to testify that another product had no influence in the development of a competing product. Same reason why I've personally never tested anything from NIK software while being involved in the development of PhotoKit Sharpener (still haven't).

Interesting point which I hadn't considered.  Of course, the guys RPP speak a lot to the difference between converters and how one should take the converter being used into account when making an exposure.  In fact, to quote Iliah Borg from a couple of weeks ago,

"...Do you think different raw converters allow for the same dynamic range and the same distribution of dynamic range?

Have you ever attempted to use, say, D7000 exposing at ISO 200 like it is ISO 800 and process it in different raw converters?

If I expose for ACR or LR I can allow some highlights to be blown out, but G-d forbid to underexpose. Contrary to that with, say, RPP. So different raw converters do mean different exposure strategies."


  Jeff, if you're not actively testing other converters, I would think that your opinion on the ETTR matter should be notated as being relevant only in the case of using ACR, no?


Title: Re: Better than ETTR ?
Post by: douglasf13 on August 09, 2011, 03:37:48 pm
It might be nice if the developers would port RPP to Win7 so the other half of the photo world could give it a try.  Until that happens, I'm an Adobe captive (though there is nothing wrong with that mind you).

Yeah, unfortunately, it's only an OSX program, and it can't be easily converted.  According to the makers, "I do not see Colorsync being implemented for Linux.  RPP relies on several OS X mechanisms, and porting it to any OS that does not support those is a major effort, literally re-writing RPP from the scratch is needed."

You may want to give Raw Therapee a shot.  It is also very good.
Title: Re: Better than ETTR ?
Post by: Alan Goldhammer on August 09, 2011, 03:45:39 pm
Yeah, unfortunately, it's only an OSX program, and it can't be easily converted.  According to the makers, "I do not see Colorsync being implemented for Linux.  RPP relies on several OS X mechanisms, and porting it to any OS that does not support those is a major effort, literally re-writing RPP from the scratch is needed."
Too bad.  I was always taught to keep machine dependent code separate from the underlying program code so that porting it would not be so difficult.  I guess I will just have to take a pass at this.
Title: Re: Better than ETTR ?
Post by: Schewe on August 09, 2011, 04:44:54 pm
  Jeff, if you're not actively testing other converters, I would think that your opinion on the ETTR matter should be notated as being relevant only in the case of using ACR, no?

Correct...I never said anything to imply that I was referring to ALL raw processors. In fact, the only other raw processors I've looked into is the camera company's offerings and in the case of my P-65+, Capture One (which handles ETTR pretty much the same way as ACR/LR).

The ONLY raw processor I claim to be an expert on is ACR/LR...I kinda have to be an expert to write a book on the subject. (and I DON'T claim to be an expert on C1...just an average user :~)
Title: Re: Better than ETTR ?
Post by: fdisilvestro on August 09, 2011, 04:45:25 pm

You may want to give Raw Therapee a shot.  It is also very good.

I'll second that. Raw therapee has a RAW pre-processing tab where you can "normalize" the RAW data before any other processing. This basically multiplies the RAW values by a factor, something like a first step towards a ISO as a Metadata implementation
Title: Re: Better than ETTR ?
Post by: ErikKaffehr on August 09, 2011, 04:46:06 pm
Hi,

There is also raw therapy...

Too bad.  I was always taught to keep machine dependent code separate from the underlying program code so that porting it would not be so difficult.  I guess I will just have to take a pass at this.
Title: Re: Better than ETTR ?
Post by: Ray on August 09, 2011, 09:06:57 pm
Hi Ray,

It might teach a valuabe lesson (without having to repeat it for every file), e.g. that Highlight Recovery should only be used after an adjustment of the exposure (and perhaps the brightness) slider(s). When Rawnalyse tells you that there are no clipped highlights, then why use the HR tool?

Cheers,
Bart

Hi Bart,
Perhaps I already do that intuitively. When an image looks as though it's overexposed, I first reduce the brightness which is set at a default +50, then bring back exposure. If the image still shows clipping, I will then use the HR tool, then reduce highlights with the tone curve.

If I can disguise the fact there are clipped highlights, or to put it another way, kid myself and kid others that there are no clipped highlights, then I don't find it particularly useful to learn that one or more channels are in fact still clipped, although it is of some academic interest. Generally, as long as it doesn't look clipped, that's fine by me.

However, if I were analysing the spectrum of an image from a distant star in order to obtain accurate information relevant to the verification of some scientific theory, I imagine I would be very concerned about the clipping of channels, whether apparent or not.

I've generally found ACR to be either better than other converters at recovering highlights, or at least as good, whenever I've taken the trouble to make a comparison.

Making more extreme adjustments to my previous example, it seems clear that the blue sky is definitely blown out to a degree which ACR cannot rectify. However, I doubt that any other converter could do a more convincing job of reconstructing that lost data.
Title: Re: Better than ETTR ?
Post by: Guillermo Luijk on August 10, 2011, 03:31:08 am
I don't see much advantage in using one program to analyse my RAW images and another program to develop them.

If you compare the input to ACR (RAW data) with the output from it (RAW development), you can try to reverse engineer how ACR works in different situations, and use that knowledge in the future. If you only have the output, you can just sit back and stare at your monitor for a long, long time... you'll never know what happened.


Making more extreme adjustments to my previous example, it seems clear that the blue sky is definitely blown out to a degree which ACR cannot rectify. However, I doubt that any other converter could do a more convincing job of reconstructing that lost data.

I think we could be mistifying too much how ACR (or any RAW developer) works in the highlights. As far as I know, when a single (or more) RAW channel gets clipped, most RAW converters just replicate luminance (R=G=B) from the available information in the non-clipped channels. There is no mystery here (-H 2 option in DCRAW does the same), the rest is a matter of contrasting the clipped area so that textures are more visible.

With these last settings, it seems your image still has colour in the sky (not sure if some parts are B&W though), so perhaps you had nothing clipped in the RAW file and it was just your previous processing (your previous ACR settings) that produced the cyan sky. Think for example that setting Exposure 0.0 doesn't mean exposure is not changed. There is the Baseline exposure (http://forums.dpreview.com/forums/read.asp?forum=1021&message=38334395) metadata in your RAW file (generally meaning overexposing), and there is the white balance which is another change in exposure but for individual channels.

If you don't want to analyse that RAW file perhaps you could upload it to some fileserver and I'd do it for you. We could then leave Plato's cave (http://en.wikipedia.org/wiki/Allegory_of_the_Cave) where ACR prisoners live, jump the wall and look into the real world of RAW. Just for academic purposes of course.

(http://img7.imageshack.us/img7/6735/cavel.jpg)
Title: Re: Better than ETTR ?
Post by: HSway on August 10, 2011, 04:54:13 am
... ETTR is about optimal exposure for the data. If you clip and didn’t wish to, that was an exposure error.


Hello all,

this post appears my first post on Luminous Landscape but the site is not that unfamiliar to me. I’d hope to be more active on this platform.

Reading the exchange that has progressed further since, the above post reflects about exactly my approach, definition and practice in relation to maximum exposure possible for given scene and high value of DR. Strictly speaking, exposing below the highlights ceiling (whatever we decide, and whatever our approach to shines and reflections is) is limiting the sensor capacity that records our image. That way I actually don’t understand ETTR as a special measure but a routine. More like optimizing DR span that we usually tune in line with our approach/vision worked relative to a size of the displayed image or the print.
Apart from rather rare occasions such as misty conditions for example, where the spread of light takes different proportions due to diffusions and where we can drive our exposure choices more by our preferences (quality of tonal levels, noise levels possibly) than what I’d call most of photographers and also my needs.

Best regards,

Hynek
Title: Re: Better than ETTR ?
Post by: PierreVandevenne on August 10, 2011, 09:12:06 am
It's the weight of tradition and the fact that manufacturers didn't want to shock a non technical audience.

A modern camera, while keeping aperture and speed as they are (well, it could be interesting to vary those continuously or in very small steps) should simply expose the scene as high as it is possible in the zone of linear response of its sensor at its optimal gain level.  All the rest, either in firmware or software, is just post-processing anyway.
Title: Re: Better than ETTR ?
Post by: Eric Myrvaagnes on August 10, 2011, 09:42:46 am
We could then leave Plato's cave (http://en.wikipedia.org/wiki/Allegory_of_the_Cave) where ACR prisoners live, jump the wall and look into the real world of RAW. Just for academic purposes of course.

(http://img7.imageshack.us/img7/6735/cavel.jpg)

;D Terrific, Guillermo!
Title: Re: Better than ETTR ?
Post by: francois on August 10, 2011, 10:23:50 am
;D Terrific, Guillermo!
I didn't know that JPEG, ACR, PS and other stuff was already used by the Greeks…
Title: Re: Better than ETTR ?
Post by: eleanorbrown on August 10, 2011, 11:55:00 am
Jeff probably an unfair question but as a user of a P65+ I would be interested to know if you prefer Capture  One or LR for you Phase RAW files. I use both and find sometimes one does better, sometimes the other. Thanks, Eleanor

Correct...I never said anything to imply that I was referring to ALL raw processors. In fact, the only other raw processors I've looked into is the camera company's offerings and in the case of my P-65+, Capture One (which handles ETTR pretty much the same way as ACR/LR).

The ONLY raw processor I claim to be an expert on is ACR/LR...I kinda have to be an expert to write a book on the subject. (and I DON'T claim to be an expert on C1...just an average user :~)
Title: Re: Better than ETTR ?
Post by: Schewe on August 10, 2011, 01:11:34 pm
Jeff probably an unfair question but as a user of a P65+ I would be interested to know if you prefer Capture  One or LR for you Phase RAW files.

A bit OT but I'll answer...it depends on what I'm shooting. If I'm in the studio and shooting tethered, I use C1 so I can control the camera functions and shoot from the laptop. Then I'll usually go ahead and process files in C1 (but also import the raws into LR for organization). If I'm shooting a lot like on a multi-day shoot in the field, I'll use LR for the importing and processing. I'm really good at ACR/LR and competent in C1, so I can basically get equal results out of either. I do find C1's current local tools to be lacking though (I look forward to more functionality there) and I think ACR/LR's sharpening and noise reduction is prolly better–but I'm prolly biased :~).

I much prefer printing from Lightroom even over Photoshop and I think C1's printing isn't great yet...fine for contact sheets. I also think C1's asset management is primitive...it'll be interesting to see what Phase One does with asset management now that they have Expression Media. iView Media Pro was a really good early asset management app that MSFT let falter. I hope P1 can bring it back...
Title: Re: Better than ETTR ?
Post by: Eric Myrvaagnes on August 10, 2011, 01:52:04 pm
I didn't know that JPEG, ACR, PS and other stuff was already used by the Greeks…
I think Plato was the first to urge photographing averything in RAW. He also insisted that only Philosophers should be allowed to use PhotoShop.
Title: Re: Better than ETTR ?
Post by: Ray on August 10, 2011, 06:58:53 pm

If you don't want to analyse that RAW file perhaps you could upload it to some fileserver and I'd do it for you. We could then leave Plato's cave (http://en.wikipedia.org/wiki/Allegory_of_the_Cave) where ACR prisoners live, jump the wall and look into the real world of RAW. Just for academic purposes of course.



Guillermo,
I'd be interested in any quick method of resconstructing a blown sky, as in my example which is fairly typical of the problem of a shift from a natural blue in the darkest part of the sky to an unnatural cyan, then sometimes to a complete blow-out in the brightest part of the sky.

I believe this is due to a mismanagement of Extra Terrestrial Tyrannosaurus Rex.

I notice you have a Hotmail address. Is Hotmail still restricted to 10MB limits for attachments? The RAW file is 13.7MB.
Title: Re: Better than ETTR ? (Stouffer wedge tests)
Post by: ErikKaffehr on August 11, 2011, 12:24:46 am
Hi,

I posted some DNGs from Stouffer Wedges for download here: http://echophoto.dnsalias.net/ekr/images/StoufferWedge/

Preliminary finding I have done with LR indicates that both cameras tested have a latitude for ETTR of 1.5 step when exposure is adjusted so we are just short of blinking highlights on step 10 of total 41 steps.

http://echophoto.dnsalias.net/ekr/images/StoufferWedge/

Best regards
Erik
Title: Re: Better than ETTR ?
Post by: Guillermo Luijk on August 11, 2011, 02:55:38 am
I notice you have a Hotmail address. Is Hotmail still restricted to 10MB limits for attachments? The RAW file is 13.7MB.

I think it is 25MB now, so it should work.


Primary finding I have done with LR indicates that both cameras tested have a latitude for ETTR of 1.5 step when exposure is adjusted so we are just short of blinking highlights on step 10 of total 41 steps.

Something to consider about this is that a digital camera measures light as a mix of all wavelengths I guess, but then it produces RAW files with 3 channels, each of them weighted towards some part of the spectrum. So to define highlight headroom some considerations are to be made:


My conclusion is that the only reliable source to define highlight headroom is the pure RAW data, but unfortunately no absolut headroom can be defined for all situations since light is very different from one scene to another. Finally each user should learn how his RAW development software behaves if one or two RAW channels clipped (if no RAW channel is clipped there is no problem, if the 3 channels were blown surely all will produce pure white, no matter what WB setting was used).
Title: Re: Better than ETTR ?
Post by: HSway on August 11, 2011, 05:08:15 am
Guillermo,
I'd be interested in any quick method of resconstructing a blown sky, as in my example which is fairly typical of the problem of a shift from a natural blue in the darkest part of the sky to an unnatural cyan, then sometimes to a complete blow-out in the brightest part of the sky


I considered a gentle cyan hue in these contra light scenes as a sort of natural (deliberately) part of the sky transitions under such conditions. Leaving aside the big topic what the human eye and the camera see and crucially how both sides see it in many respects and also in these conditions, I just tried suppressing some cyan tones in the sky with strong highlights with what happened to be my last edit. I think I will involve these adjustments more in the future as I like the results of more uniformed sky in this sense.
I just looked at the referred image and while I see it’s a different case, the exchange brought me an interesting impulse, thanks.

Best,

Hynek
Title: Re: Better than ETTR ?
Post by: Ray on August 11, 2011, 09:58:23 am

I considered a gentle cyan hue in these contra light scenes as a sort of natural (deliberately) part of the sky transitions under such conditions. Leaving aside the big topic what the human eye and the camera see and crucially how both sides see it in many respects and also in these conditions, I just tried suppressing some cyan tones in the sky with strong highlights with what happened to be my last edit. I think I will involve these adjustments more in the future as I like the results of more uniformed sky in this sense.
I just looked at the referred image and while I see it’s a different case, the exchange brought me an interesting impulse, thanks.

Best,

Hynek


On reflection, and after a bit of experimentation, I get the impression that the degree of transition from blue to cyan in a partially blown sky has been reduced in CS5. Or perhaps it's the case that the number of controls in ACR, to enable such reduction, has been increased.
Title: Re: Better than ETTR ?
Post by: eleanorbrown on August 11, 2011, 10:04:41 am
Thanks Jeff. A bit OT too but printing through LR is second to none. I gave up using Imageprint for LR print module . Eleanor



I much prefer printing from Lightroom even over Photoshop and I think C1's printing isn't great yet...fine for contact sheets. I also think C1's asset management is primitive...it'll be interesting to see what Phase One does with asset management now that they have Expression Media. iView Media Pro was a really good early asset management app that MSFT let falter. I hope P1 can bring it back...
Title: Re: Better than ETTR ?
Post by: bjanes on August 11, 2011, 10:04:55 am
I've read Emil's treatise, and have great respect for his work. And, I also understand the theory.

But, in practice, I am confident that I see smoother tonal gradations on ETTR "normalized" shadow areas rather than native ones when I need to strongly open up such tonalities.

So, who do I believe? As the old joke has it, "The experts, or my lying eyes"?

Michael

Ps: Bumble bees can indeed fly, and prove it to themselves every day.


Michael,

Are you seeing better tonal gradations in the shadows because of increased SNR with ETTR or because of reduced posterization due to an increased number of levels? As mentioned elsewhere in this thread, noise dithers an insufficient number of levels in nearly all cases with current cameras, preventing posterization. What do you see that indicates that an increased number of levels is giving you better results rather than improved SNR?

That bumble bees can not fly was based on faulty theory. Where are Emil and Guillermo wrong?

Regards,

Bill
Title: Re: Better than ETTR ?
Post by: Guillermo Luijk on August 11, 2011, 02:38:06 pm
I notice you have a Hotmail address. Is Hotmail still restricted to 10MB limits for attachments? The RAW file is 13.7MB.

Here the areas where a RAW channel was clipped (note that the R channel was the only one intact across the entire scene):

(http://www.guillermoluijk.com/misc/anim.gif)
Looking at the gray bands in the sky (each gray tone in the colourscale represents 1EV), exposing 1 stop less would have been enough to avoid any clipping.

My belief that one clipped channel automatically means B&W output was wrong (although I have seen that clearly in other RAW files with a lot more texture). Also there are big differences in the HL strategies among the two RAW developers, the cyan colour produced by ACR was replaced by some kind of magenta cast in DCRAW. This cyan/magenta effect seems to extend in a region of influence outside the clipped zones:

(http://www.guillermoluijk.com/misc/anim2.gif)
Funily a mix of both skies (70% to DCRAW, 30% to ACR) produces a more pleasant image to my like.

Regards
Title: Re: Better than ETTR ?
Post by: fike on August 11, 2011, 04:28:48 pm
That is exactly the phenomenon that I have seen repeatedly, and I don't like it. The risk of getting this cyan highlight in the sky has made me far less willing to use EttR unless I am absolutely certain that I have a very compressed histogram--rainy or foggy days in particular. 

I don't agree that avoiding this clipping is just a matter of always ensuring you haven't overexposed a highlight in one channel.  As you move the exposure to the right, you are doing more than simply moving the same shape of graph to the right.  As you move right, you are changing the magnitude and slopes of each channel.  As a result what is happening outside of the histogram area as you approach the highlight line on the right, may be a mystery to you. 

Title: Re: Better than ETTR ?
Post by: digitaldog on August 11, 2011, 04:32:53 pm
I don't agree that avoiding this clipping is just a matter of always ensuring you haven't overexposed a highlight in one channel. 

Why not? I’d suspect the color effects here are the creation of the data from the other channels. IOW, don’t clip. Anything.
Title: Re: Better than ETTR ?
Post by: fike on August 11, 2011, 04:44:48 pm
Because I don't believe that on the back of the camera you have enough information to know that you haven't lost data off the right side of the histogram. 

Yesterday I was sitting in the woods, experimenting with my camera while considering this discussion.  Something that surprised me was that as I increased the exposure to move to the right the shape of the histogram changes--the slope and magnitude of each peak changed.  This suggests that the histogram isn't representing a linear transformation of the exposure data as you move to the right.  What I am inferring is that it is possible (even likely) that as you move the histogram to the right (as close to the edge as you can) there may be activity that would be shown outside the graph if it were to extend beyond the edge.  This activity may be a result of the change in the magnitude and slope of that I described above. 

I guess the response to that idea is I should watch the blinking highlight alert, but we have all seen that those are not particularly accurate and that many raw converters can bring back some of that data.  So what you end up doing is playing a game of chicken with the right edge.  What we never know is how far off that edge we have gone in any particular channel and whether or not the histogram would show that discrete information if we decreased the exposure enough.

Title: Re: Better than ETTR ?
Post by: Guillermo Luijk on August 11, 2011, 05:31:59 pm
Because I don't believe that on the back of the camera you have enough information to know that you haven't lost data off the right side of the histogram.
(...)
as I increased the exposure to move to the right the shape of the histogram changes--the slope and magnitude of each peak changed.  This suggests that the histogram isn't representing a linear transformation of the exposure data as you move to the right.

Fike, I totally agree with what you say, but we must be clear: the reason is not the principle of ETTR (which is conceptually correct), nor the way sensors work (they are very linear devices). The one and only reason for this uncertainty is camera makers are making machines oriented to the JPEG shooter, not allowing the RAW shooter know about exposure accurately. Camera makers are making us RAW shooters prisoners in Plato's cave, trying to decipher what the shadows in the wall mean.

These are RAW histograms obtained from a series of shots equally spaced 1 stop apart over a white wall with uniform lighting, and represented in log scale. They move at totally regular 1EV intervals (as expected), and they don't change their shape (as expected; only the B channel spreaded slightly once all its surrounding R and G photocaptors reached saturation):

(http://www.guillermoluijk.com/misc/sequence.gif)
Title: Re: Better than ETTR ?
Post by: Alan Goldhammer on August 11, 2011, 07:01:50 pm
Which is why camera makers should provide the option to give us RAW histograms on the LCD screen.
Title: Re: Better than ETTR ?
Post by: HSway on August 12, 2011, 03:26:59 am
On reflection, and after a bit of experimentation, I get the impression that the degree of transition from blue to cyan in a partially blown sky has been reduced in CS5. Or perhaps it's the case that the number of controls in ACR, to enable such reduction, has been increased.


Without doubt it’s evolving and opening up better and better possibilities of editing through its tools and the engine, another factor that makes it a backbone of the digital photography darkroom.

Best,

Hynek
Title: Re: Better than ETTR ?
Post by: Ray on August 12, 2011, 08:19:29 am
Here the areas where a RAW channel was clipped (note that the R channel was the only one intact across the entire scene):


Interesting! An excellent demonstration of the situation, Guillermo. Thanks. It makes sense that the red channel would be the last to be clipped in a blue sky, unless it were a sunrise or sunset.

I guess someone at Adobe has decided that a reconstruction of a blown sky that leans towards cyan is more acceptable than one which leans towards magenta.

Regards
Title: Re: Better than ETTR ?
Post by: bjanes on August 21, 2011, 10:11:14 am
Interesting! An excellent demonstration of the situation, Guillermo. Thanks. It makes sense that the red channel would be the last to be clipped in a blue sky, unless it were a sunrise or sunset.

I guess someone at Adobe has decided that a reconstruction of a blown sky that leans towards cyan is more acceptable than one which leans towards magenta.

Ray,

The Adobe software may not be as intelligent as you think. Rather than identifying a blown sky, ACR may have merely identified a clipped area. When reconstructing blown areas, ACR pushes blown areas to neutral. The red channel is the only one intact in the photo, so ACR may have added blue and green, resulting in a cyan cast.

DCRaw has various highlight recovery options. See Guillermo's post (http://www.guillermoluijk.com/tutorial/dcraw/index_en.htm) on highlight recovery and look at the reconstructions of blown facial highlights in his example. ACR pushes them towards white, whereas DCRaw looks at the adjacent colors in determining the reconstruction.

ETTR is a good thing, but when one pushes it to extreme, clipping and data loss will occur. The prudent photographer avoids data loss rather than depending on implementation of highlight recovery in software. With today's high performance sensors, a bit of underexposure can be tolerated if bracketing is not feasible. Since blown highlights are often relatively neutral in color cast, highlight recovery can often rescue inadvertently overexposed images. However, I think that successful reconstruction has more to do with the SNR than the abundance of levels in the highlights as claimed by some experts. Shot noise and rendering of the raw file will usually be sufficient to dither the image and prevent banding.

Regards,

Bill
Title: Re: Better than ETTR ?
Post by: ejmartin on August 21, 2011, 06:43:54 pm
I've generally found ACR to be either better than other converters at recovering highlights, or at least as good, whenever I've taken the trouble to make a comparison.

Making more extreme adjustments to my previous example, it seems clear that the blue sky is definitely blown out to a degree which ACR cannot rectify. However, I doubt that any other converter could do a more convincing job of reconstructing that lost data.

May I have the raw file to play with?

Here is an example that works particularly well with RawTherapee's (in the dev version, 4.0) recently revamped color propagation method of highlight recovery:

(http://theory.uchicago.edu/~ejm/pix/20d/posts/ojo/HLR-maskedgirl.jpg)
Title: Re: Better than ETTR ?
Post by: ejmartin on August 21, 2011, 07:07:47 pm
Hi Ray,

It might teach a valuabe lesson (without having to repeat it for every file), e.g. that Highlight Recovery should only be used after an adjustment of the exposure (and perhaps the brightness) slider(s). When Rawnalyse tells you that there are no clipped highlights, then why use the HR tool?

Cheers,
Bart

It depends how ACR does HR.  The raw file may have no clipped highlights, but WB can send some channels past the white point.  A good HR tool should be able to distinguish channels that are clipped because of WB vs those that are clipped because the raw data is clipped.
Title: Re: Better than ETTR ?
Post by: Bart_van_der_Wolf on August 21, 2011, 07:28:52 pm
It depends how ACR does HR.  The raw file may have no clipped highlights, but WB can send some channels past the white point.

Hi Emil,

A good point, althought the potentially best method to avoid data clipping due to WB would be a bit more -EV correction at the Raw conversion stage.

Quote
 A good HR tool should be able to distinguish channels that are clipped because of WB vs those that are clipped because the raw data is clipped.

Exactly, fully agree with that.

Cheers,
Bart
Title: Re: Better than ETTR ?
Post by: ejmartin on August 21, 2011, 08:27:44 pm
Hi Emil,

A good point, althought the potentially best method to avoid data clipping due to WB would be a bit more -EV correction at the Raw conversion stage.

The tricky bit is that -EV correction to save unclipped channels runs the risk of messing up clipped channels, bringing them down into the visible range, with disastrous hue shifts.
Title: Re: Better than ETTR ?
Post by: Ray on August 21, 2011, 09:34:08 pm
May I have the raw file to play with?

Here is an example that works particularly well with RawTherapee's (in the dev version, 4.0) recently revamped color propagation method of highlight recovery:


Hi Emil,
I've sent you the RAW image as requested plus another one with an even greater degree of blown sky, as detailed below.

Thanks for your interest.

Regards,   Ray
Title: Re: Better than ETTR ?
Post by: ejmartin on August 21, 2011, 10:17:40 pm
Here's RawTherapee on the first one.  The Color Propagation recovery tool is still experimental -- I had to adjust the pre-demosaic CA correction manually to keep the CA in the image from infecting the highlight recovery.  I'll have to fix that in the next iteration.  The blown area is still a little too pink for my taste.

The second image is too blown to recovery anything in the sky -- there is no unblown region to inpaint from.

(http://theory.uchicago.edu/~ejm/pix/20d/posts/ojo/ray-blownsky.jpg)
Title: Re: Better than ETTR ?
Post by: Ray on August 21, 2011, 11:09:26 pm
Here's RawTherapee on the first one.  The Color Propagation recovery tool is still experimental -- I had to adjust the pre-demosaic CA correction manually to keep the CA in the image from infecting the highlight recovery.  I'll have to fix that in the next iteration.  The blown area is still a little too pink for my taste.

The second image is too blown to recovery anything in the sky -- there is no unblown region to inpaint from.

(http://theory.uchicago.edu/~ejm/pix/20d/posts/ojo/ray-blownsky.jpg)

Hi Emil,

That's amazing! Are you involved in the development of RAW Therapee? Version 4 looks as though it could be very useful. I look forward to using it.

Cheers!   Ray
Title: Re: Better than ETTR ?
Post by: ejmartin on August 21, 2011, 11:17:57 pm
Hi Emil,

That's amazing! Are you involved in the development of RAW Therapee? Version 4 looks as though it could be very useful. I look forward to using it.


Yes, I started revamping the image processing pipeline when it went open source a little over a year ago.  We're slowly getting our act together...
Title: Re: Better than ETTR ?
Post by: ErikKaffehr on August 22, 2011, 03:49:41 am
Hi,

Very much impressed...

Best regards
Erik

Yes, I started revamping the image processing pipeline when it went open source a little over a year ago.  We're slowly getting our act together...
Title: Re: Better than ETTR ?
Post by: Guillermo Luijk on August 22, 2011, 11:46:58 am
Impressive!