Luminous Landscape Forum

Site & Board Matters => About This Site => Topic started by: Fine_Art on October 28, 2014, 05:56:20 pm

Title: The Optimum Digital Exposure
Post by: Fine_Art on October 28, 2014, 05:56:20 pm
The article makes perfect sense.

Why don't our cameras, especially any over $500, have an ETTR exposure mode built in? Surely it is just a software addition to the metering reading?

People should be able to specify 100% within the matrix or 99.5% within the matrix in ETTR mode. The manufacturers should understand exposure enough to build this in.
Title: Re: The Optimum Digital Exposure
Post by: markd61 on October 28, 2014, 07:47:34 pm
It seems that I have been doing this for a few years now. I evolved into this out of a desire to maintain shadow detail.
 Never mind that a final image had deep info-free shadows!
I found that my files were naturally being "overexposed" by about a stop on average yet my processed images seemed very good. Rarely do I have blown highlights and if I do it is in scenes where the outdoor sun has introduced backbreaking contrast.

I agree that it seems that the camera manufacturers "should" be able to have an exposure matrix that could detect the maximum exposure and the minimum exposure and create an exposure that would then be optimal. However as a layperson these things always seem to be a matter of a weekend's worth of effort rather than the gigantic task they actually are. ;)
Title: Re: The Optimum Digital Exposure
Post by: Ray on October 28, 2014, 08:09:09 pm
The article makes perfect sense.

Why don't our cameras, especially any over $500, have an ETTR exposure mode built in? Surely it is just a software addition to the metering reading?

People should be able to specify 100% within the matrix or 99.5% within the matrix in ETTR mode. The manufacturers should understand exposure enough to build this in.

I suspect that part of the reason is that the inexperienced amateur would get confused to sometimes see such dark images produced by his camera. Often, the effect of a built-in ETTR exposure mode would be to cause a significant underexposure of the most important part of the image in order to avoid blown highlights in less significant parts of the image, such as patches of sky visible through gaps between the branches and foliage of a tree.

The experienced photographer will tend to make an assessment regarding which parts of the scene he wants to be as noise-free as possible. He may decide to sacrifice less significant detail in a white wall or sky in order to get more pleasing and noise-free results for a subject in the shade. A camera in ETTR mode could not make such decisions.

Such a feature could also be very negative for certain brands of cameras, such as Canon, which have as much as 2 EV lower dynamic range than Nikon at base ISO. It would be just too easy for even a complete amateur to shoot the same high-contrast scene with a Canon and Nikon DSLR, with both cameras in ETTR mode, and see how much noisier the shadows are in the Canon images. So there would clearly be a big disincentive for Canon to introduce such a feature, unless it were to simultaneously raise the DR performance of its cameras.  ;)

Also, if Canon were to significantly improve the DR of its cameras, there would be less need to develop an ETTR mode, just as there is already a reduced need for Nikon to do so, because of the wide DR of its current DSLRs.
Title: Re: The Optimum Digital Exposure
Post by: Torbjörn Tapani on October 28, 2014, 08:23:22 pm
My favourite post on ETTR comes from Pekka Potka, it works with EVF and I first read about it in this post about E-P3: http://www.pekkapotka.com/journal/2011/12/20/expose-to-the-right-ettr-with-e-p3.html?rq=ettr

No need for bracketing.

Maybe Pekka Potka is not so well known but I can recommend his blog http://www.pekkapotka.com/journal/ 
Title: Re: The Optimum Digital Exposure
Post by: JohnBrew on October 28, 2014, 09:23:43 pm
Interesting. I've been trying to work with this idea in mind lately, however, I find it very camera specific. Or sensor specific, if you like. On my Leica M8.2 (CCD) you absolutely cannot overexpose, because if the highlights are blown - they are forever blown. With my D800 (CMOS), which has a totally different sensor, I usually overexpose by a 1/3 stop.
Title: Re: The Optimum Digital Exposure
Post by: michael on October 28, 2014, 09:27:20 pm
The article makes perfect sense.

Why don't our cameras, especially any over $500, have an ETTR exposure mode built in? Surely it is just a software addition to the metering reading?

People should be able to specify 100% within the matrix or 99.5% within the matrix in ETTR mode. The manufacturers should understand exposure enough to build this in.

I have been evangelizing for an ETTR exposure mode now for some years, whenever I meet with camera company product development people. In Live View mode the camera knows exactly what every pixel is reading. Piece of cake. Easy as pie.

One should even be able to specific a % of "blown" pixels to accommodate specular highlights.

What happens when I proposethis? Some nod sagely and say yes, good idea..."we will study", which in some parts of the world means..."we didn't think of it so fuggetaboutit". Incredibly there are some that actually just don't get it. They just nod, look like they understand, but you know from their eyes that they don't. Just incredible.

Arde we surprised then that we still have cameras from major manufacturers that don't have blinkies or zebras for judging ETTR exposure oneself, and some that don't even have histograms on instant review. No names, because I don't want to start a pissing match, but you know who they are!

Michael

Title: Re: The Optimum Digital Exposure
Post by: michael on October 28, 2014, 09:30:35 pm
Interesting. I've been trying to work with this idea in mind lately, however, I find it very camera specific. Or sensor specific, if you like. On my Leica M8.2 (CCD) you absolutely cannot overexpose, because if the highlights are blown - they are forever blown. With my D800 (CMOS), which has a totally different sensor, I usually overexpose by a 1/3 stop.

John. You can't "overexpose" a sensor. Once the wells are full the photons fall on the floor. It's a brick wall.

But some cameras don't give accurate histograms. Most are not based on the raw data but on an sRGB JPG. This is how you lose a stop or more of your data.

You need to run some tests and compare what the camera says is overexposed with what really is when the histogram is viewed in a raw processing program.

Michael
Title: Re: The Optimum Digital Exposure
Post by: BernardLanguillier on October 28, 2014, 09:46:03 pm
Interesting. I've been trying to work with this idea in mind lately, however, I find it very camera specific. Or sensor specific, if you like. On my Leica M8.2 (CCD) you absolutely cannot overexpose, because if the highlights are blown - they are forever blown. With my D800 (CMOS), which has a totally different sensor, I usually overexpose by a 1/3 stop.

John,

I don't think this has to do with CCD vs CMOS. This probably is just about how the ISO value was calibrated.

Overall this article is a good summary, but I am a bit unclear what is new relative to the original ETTR concept?

Yes, we can use bracketing in order to increase the chances we get one image optimally exposed (from an ETTR standpoint). Now, with the latest Exmor sensors, where is so much DR available anyway that under-exposing one stop relative to ideal ETTR has a fairly limited real world impact.

Cheers,
Bernard
Title: Re: The Optimum Digital Exposure
Post by: deejjjaaaa on October 28, 2014, 10:15:47 pm
> I have developed an exposure method

...

> Method #1 - Using a Spot Meter

http://www.rawdigger.com/houtouse/lightmeter-calibration

no ACR no LR no books, no "OneZoneTM Digital Exposure Method" - thank you

> The “Blinkies” on the back of your camera occur about 1 stop before the highlight warning in your software – highlight “Clipping”!

blinkies (or zebra) in my setup for E-M1 and A-7 are happening exactly where rawdigger says raw clipping is (with << 1/3EV precision)  ... but granted I live with not so nice OOC JPGs to get there.

Title: Re: The Optimum Digital Exposure
Post by: BobD on October 28, 2014, 10:38:04 pm
I have been evangelizing for an ETTR exposure mode now for some years, whenever I meet with camera company product development people. In Live View mode the camera knows exactly what every pixel is reading. Piece of cake. Easy as pie.

What happens when I proposethis? Some nod sagely and say yes, good idea..."we will study" ... Incredibly there are some that actually just don't get it. They just nod, look like they understand, but you know from their eyes that they don't. Just incredible.

Michael


Micheal, I totally agree this can so easily be built in to as a menu choice in digital cameras.  I can only re-new the discussion and hope to move the ball a little further to that end. However, observing your pleas to camera manufactures over the years with such little response (and incredibly that they "just don't get it") I can only wonder if we will ever see it.

Years ago, when I worked in R&D at Polaroid, I witness one of the great visionaries Dr. Land work for years and years on instant movies... code name "Sesame". Whenever we thought we were getting closer to a viable product... it was "not ready yet" and again more years were added to the development of Sesame. It was finally ready for market about 2 years after the introduction of Betamax and VHS tape!

I appreciate you relentless pursuit of pushing this industry forward. Your recent video stating that there has been very little new in the development with digital cameras is so true. An "ETTR exposure mode feature" in digital cameras is easily doable. We'll see.  My hope is it doesn't become a digital "Sesame"... my guess is, if it does happen, it will probably be heralded as a brilliant NEW idea!

Thank you for all you efforts and the forum you have created for voices and thoughts to be shared.
Bob
Title: Re: The Optimum Digital Exposure
Post by: Chrisso26 on October 28, 2014, 10:58:00 pm
I have my Sigma Merrill set to overexpose +0.3 for every shot. Isn't this effectively ETTR.
On the BM Pocket Camera I have zebras set to 95%. I set my exposure so the brightest part of the image is not quite triggering the 95% zebras. I know it's not exactly accurate, but my graded results have been pretty good so far.
Title: Re: The Optimum Digital Exposure
Post by: Fine_Art on October 28, 2014, 11:08:28 pm
Part of the package the manufacturers need to figure out is:
the photog still gets to adjust +/- ev from ETTR for things like chrome, clouds, etc.
their jpg processing engine should produce a "normal" exposure from the ETTR data.

They may be ignoring the input thinking about their jpgs for casual users.
Title: Re: The Optimum Digital Exposure
Post by: Fine_Art on October 28, 2014, 11:16:33 pm
I suspect that part of the reason is that the inexperienced amateur would get confused to sometimes see such dark images produced by his camera. Often, the effect of a built-in ETTR exposure mode would be to cause a significant underexposure of the most important part of the image in order to avoid blown highlights in less significant parts of the image, such as patches of sky visible through gaps between the branches and foliage of a tree.

The experienced photographer will tend to make an assessment regarding which parts of the scene he wants to be as noise-free as possible. He may decide to sacrifice less significant detail in a white wall or sky in order to get more pleasing and noise-free results for a subject in the shade. A camera in ETTR mode could not make such decisions.

Such a feature could also be very negative for certain brands of cameras, such as Canon, which have as much as 2 EV lower dynamic range than Nikon at base ISO. It would be just too easy for even a complete amateur to shoot the same high-contrast scene with a Canon and Nikon DSLR, with both cameras in ETTR mode, and see how much noisier the shadows are in the Canon images. So there would clearly be a big disincentive for Canon to introduce such a feature, unless it were to simultaneously raise the DR performance of its cameras.  ;)

Also, if Canon were to significantly improve the DR of its cameras, there would be less need to develop an ETTR mode, just as there is already a reduced need for Nikon to do so, because of the wide DR of its current DSLRs.

You are right for Canon. However, if an Exmor using company like Nikon does make an exposure toggle "Expose grey/ Expose ETTR", they will take even more customers from Canon. It is in all their best interests to not be late to the party once it starts.
Title: Re: The Optimum Digital Exposure
Post by: BobD on October 29, 2014, 12:13:27 am
Part of the package the manufacturers need to figure out is:
the photog still gets to adjust +/- ev from ETTR for things like chrome, clouds, etc.
their jpg processing engine should produce a "normal" exposure from the ETTR data.

They may be ignoring the input thinking about their jpgs for casual users.

The default can be for the "casual user".  However, there are so many pages of menus - if the have a menu item for for "back focus" there can surly be a menu item for ETTR!
Title: Re: The Optimum Digital Exposure
Post by: BernardLanguillier on October 29, 2014, 02:40:29 am
I appreciate you relentless pursuit of pushing this industry forward. Your recent video stating that there has been very little new in the development with digital cameras is so true. An "ETTR exposure mode feature" in digital cameras is easily doable. We'll see.  My hope is it doesn't become a digital "Sesame"... my guess is, if it does happen, it will probably be heralded as a brilliant NEW idea!

Bob,

What is your view about the value of the highlight priority metering mode of the D810?

Since it is linked with the spot meter mode of the camera, it seems like a pretty good solution, isn't it?

Cheers,
Bernard
Title: Re: The Optimum Digital Exposure
Post by: ErikKaffehr on October 29, 2014, 03:11:44 am
Hi,

In my view the article has a very low noise to fact ratio, but I also feel that it has several serious issues. The foremost issue is that it ignores the fact that both ACR and Lightroom do excessive highlight recovery, and the histogram on LR (and I presume ACR) takes that recovery into account. So you can get a pretty good histogram in LR or ACR, where much of the highlight information is actually blown out.

The only histogram that can be trusted is a raw histogram like the ones in RawDigger, an excellent learning tool.

My experience is that the histograms on both my Sonys and the P45 are quite OK.

Exposing based on spot metered highlights may be a good idea. But those highlights may be hard to locate and to measure.

I would say that an exposure program based highlight preservation make may a lot (or at least some) sense.

Optimal exposure strategy may differ between sensors. Canon sensors may need a different exposure strategy than Sony Exmore sensors, due to different handling of pre ADC analogue gain for different ISOs.

Best regards
Erik
Title: Re:
Post by: Torbjörn Tapani on October 29, 2014, 07:35:18 am
Erik, not really I think. ETTR is the same for every sensor. Meaning to gather as much light as possible. ISO and shadow priority is a different matter.
Title: Re: Sv: Re: The Optimum Digital Exposure
Post by: Torbjörn Tapani on October 29, 2014, 07:43:34 am
I have my Sigma Merrill set to overexpose +0.3 for every shot. Isn't this effectively ETTR.
On the BM Pocket Camera I have zebras set to 95%. I set my exposure so the brightest part of the image is not quite triggering the 95% zebras. I know it's not exactly accurate, but my graded results have been pretty good so far.
Not exactly. ETTR might mean you underexpose from camera metering in some rare case. It all depends on relevant highlights.
Title: Re: The Optimum Digital Exposure
Post by: bjanes on October 29, 2014, 08:23:50 am
In my view the article has a very low noise to fact ratio, but I also feel that it has several serious issues. The foremost issue is that it ignores the fact that both ACR and Lightroom do excessive highlight recovery, and the histogram on LR (and I presume ACR) takes that recovery into account. So you can get a pretty good histogram in LR or ACR, where much of the highlight information is actually blown out.

The only histogram that can be trusted is a raw histogram like the ones in RawDigger, an excellent learning tool.

I agree that the RawDigger histogram is the preferred tool for histogram analysis, but one can obtain a reasonable approximation of the status of the raw RGB channels after white balance in ACR or LR by setting the color space to ProPhotoRGB and using PV2010 with the sliders on the main panel zeroed out (set to 0).

My experience is that the histograms on both my Sonys and the P45 are quite OK

A major limitation of the histograms on my Nikon D800e is that they are based on the JPEG preview image, and the widest color space is Adobe RGB. The luminance histogram may not detect channel clipping in the blue or red channel, since it is heavily weighted for the green channels. The RGB histograms may show saturation clipping in the AdobeRGB space when the raw channels are intact. For daylight white balance, the red multiplier is near 2.0 and the red channel histogram shows the red channel values after white balance, effectively moving the histogram to the right by one stop. When photographing red flowers, one may need to underexpose by several stops to eliminate clipping in the red histogram. The camera needs to offer ProPhotoRGB as one of the camera renderings.

Regards,

Bill
Title: Re: The Optimum Digital Exposure
Post by: Bart_van_der_Wolf on October 29, 2014, 08:42:05 am
When photographing red flowers, one may need to underexpose by several stops to eliminate clipping in the red histogram.

I agree with Bill, flowers can have amazing saturation thus presenting not only overexposure risk, but some also risk of underexposure, e.g. with near zero Blue and very high Red channel signals (at the same time).

RawTherapee at least offers a solution by allowing to change the per channel pre-demosaicing signal levels, I wish other converters allowed such manual control.

Cheers,
Bart
Title: Re: The Optimum Digital Exposure
Post by: MHMG on October 29, 2014, 08:42:51 am
John. You can't "overexpose" a sensor. Once the wells are full the photons fall on the floor. It's a brick wall.

You can't overexpose a sensor, but you can overexpose an image. And when you do, highlight color and tonal detail cannot be recovered from the RAW file because what Michael said, those photons "fell on the floor" :)

In the "good ole" days of the film era and the classic zone system, the rule for making optimal negatives was to "expose for the shadows and develop for the highlights". It made total sense but begged the use of a spot meter to perform the task consistently under real world lighting not studio conditions. For the modern RAW "digital negative" workflow this classic rule can be stated as just the reverse, i.e.,  "Expose for the highlights and develop for the shadows". "ETTR" is just another way of saying the same thing, and it's also how we had to expose color reversal film except that we had no real control over the development of the shadows with color reversal film. With todays RAW image editors we do.

I agree with Michael and others that the camera engineers could indeed build a decent ETTR mode into the camera metering system, but I suspect part of their reluctance to do so is that it would require a total rethink on how to construct a good looking OOC jpeg from that ETTR RAW data. Still, it's possible, and if anyone is likely to do it, my guess would be Fuji will get there first.

Re: the article:  It makes some good points, but the notion that "A good digital exposure looks as if it has been dipped in skim milk!” will lead to totally clipped highlights with many of todays digital cameras, IMHO.  This exposure advice is probably too simplistic because there are so many camera and RAW editor variables that factor into the determination of the optimal RAW exposure. Neither the camera blinkies nor the clipping warnings in LR are a precise indicator as to when full color and tone is being destroyed in critical highlight values of the scene. Today's cameras' "smart metering modes" like my D810's matrix and highlight metering modes are still trying to play nice with OOC jpeg and "Dlighting" jpeg processing methods. As such they will miss the optimal RAW exposure far too many times to make me comfortable with those modes out in the field. I resort to, wait for it, spot metering mode with liberal use of AE LOCK and then recompose the frame, or as runner up, the center weighted mode. Both of these modes aren't trying to outthink me, but as the "thinking" photographer I have to pay more attention to correct exposure to master those older "traditional" metering modes :)

Lastly, re the gamut warnings in LR. I find that it is entirely possible to set the RGB model to Prophoto, enable a particular camera profile, and then set the various sliders such that I can make the gamut clipping warnings go away even on images where delicate highlight colors and tones in the RAW file are still truly clipped. I have found that the single best analytical tool to really get a handle on camera-specific RAW image exposure is to use the Xrite color Passport target and pay particular attention to the light pastel color patches in the target that Xrite added to the overall target for white balance corrections. These very light tones of subtle difference in hue provide a very sensitive test for when delicate highlight colors are getting overexposed to the extent that they are impossible to fully recover with "magic" settings in LR or other RAW editors. When the camera sensor is right on the brink of clipping, recovery of those delicate colors may still be partially recoverable but definitely not fully recoverable. For my D810, it occurs with about 1.3 stop extra exposure to reach that point when metering carefully from an 18% reflectance gray card. On my Fuji S3, it takes about 3 stops overexposure to begin to lose those delicate highlight colors in RAW processing. Moral of the story is that ETTR tolerances are highly camera dependent, and a hand-held spot meter is still a wise choice if you are serious about ETTR optimal exposures.

cheers,
Mark
http://www.aardenburg-imaging.com
Title: Re: The Optimum Digital Exposure
Post by: michael on October 29, 2014, 08:52:02 am
Thanks for your comments Bob.

I'm just waiting for one of the companies to file a patent for it. I have emails going back at least 10 years that would count as "prior art".

Cheers,

Michael
Title: Re: The Optimum Digital Exposure
Post by: bjanes on October 29, 2014, 09:20:53 am
I agree with Bill, flowers can have amazing saturation thus presenting not only overexposure risk, but some also risk of underexposure, e.g. with near zero Blue and very high Red channel signals (at the same time).

RawTherapee at least offers a solution by allowing to change the per channel pre-demosaicing signal levels, I wish other converters allowed such manual control.

Cheers,
Bart

Bart,

The ability to set per channel WB multipliers in the raw converter (as can be done with RawTherapee, which also has an option to show the raw histogram) would be helpful. Rather than leaving the green multiplier at 1.0 and having red and blue multipliers greater than unity, one could set the multiplier for the most exposed channel to unity, and the other multipliers to less than unity, as Guillermo discusses here (http://www.guillermoluijk.com/tutorial/dcraw/index_en.htm) for dcraw.

If the raw converter had a true exposure slider affecting all channels linearly, couldn't one accomplish something similar by merely decreasing the exposure. With ACR and LR, the exposure slider with PV2010 seems to work reasonably well for this purpose.

Bill
Title: Re: The Optimum Digital Exposure
Post by: dreed on October 29, 2014, 09:39:06 am
The developers at "Magic Lantern (http://www.magiclantern.fm/)" have included in their arsenal of addons a mode where you can activate an "Automatic ETTR (http://www.magiclantern.fm/forum/index.php?topic=10808.0)" that uses a histogram built from raw data (http://www.magiclantern.fm/forum/index.php?topic=5149.0) to calculate the exposure.

From my experience, this works exactly as advertised.

Unfortunately it is only available for Canon's EOS line of DSLRs.
Title: Re: The Optimum Digital Exposure
Post by: Bart_van_der_Wolf on October 29, 2014, 09:43:54 am
Bart,

The ability to set per channel WB multipliers in the raw converter (as can be done with RawTherapee, which also has an option to show the raw histogram) would be helpful. Rather than leaving the green multiplier at 1.0 and having red and blue multipliers greater than unity, one could set the multiplier for the most exposed channel to unity, and the other multipliers to less than unity, as Guillermo discusses here (http://www.guillermoluijk.com/tutorial/dcraw/index_en.htm) for dcraw.

Exactly what I'm talking about. Whitebalancing and 'exposure' optimization bundled together. Of course that's harder to do in Raw capture, but then one would need to get flexible exposure per channel (while recording white balance as multipliers). When the data is still in linear gamma space, not really all that hard to do.

Quote
If the raw converter had a true exposure slider affecting all channels linearly, couldn't one accomplish something similar by merely decreasing the exposure. With ACR and LR, the exposure slider with PV2010 seems to work reasonably well for this purpose.

Yes, after capture, it's easy enough when still in linear gamma space. Depending on the software, there may be some complications with color profiles ('scene referred' versus 'output referred'), also shown by the PV2010 (and later) (non-)predictability. But then, what's one more challenge after eliminating several others.

Cheers,
Bart
Title: Re: The Optimum Digital Exposure
Post by: deejjjaaaa on October 29, 2014, 09:53:13 am
I wish other converters allowed such manual control.
and they do, for example RPP or Iridient have per channel multipliers (RPP only and Iridient as an option)... there are many different raw converters in this world
Title: Re: The Optimum Digital Exposure
Post by: deejjjaaaa on October 29, 2014, 09:56:56 am
but one can obtain a reasonable approximation of the status of the raw RGB channels after white balance in ACR or LR by setting the color space to ProPhotoRGB and using PV2010 with the sliders on the main panel zeroed out (set to 0).
there might be hidden exposure corrections in LR/ACR ... so unless in addition you use (or verify that your particular combination of camera model, profile, iso, etc does not do this) a particular .dcp profile for a particular camera model (and even for a particular ISO there - because Adobe might use different hidden expocorrections for different ISOs) ACR and LR are simply not the right tool to be bothered with
Title: Re: The Optimum Digital Exposure
Post by: bjanes on October 29, 2014, 10:23:11 am
there might be hidden exposure corrections in LR/ACR ... so unless in addition you use (or verify that your particular combination of camera model, profile, iso, etc does not do this) a particular .dcp profile for a particular camera model (and even for a particular ISO there - because Adobe might use different hidden expocorrections for different ISOs) ACR and LR are simply not the right tool to be bothered with

That is a good point, but with my D800e at base ISO and with the Adobe Standard profile, there appears to be no hidden expo corrections as can be verified by comparing the ACR histogram to the RawDigger histogram. While not ideal for some purposes, ACR/LR is the raw converter that many of us use for routine work.

Bill
Title: Re: Re: Re:
Post by: Guillermo Luijk on October 29, 2014, 11:13:32 am
Erik, not really I think. ETTR is the same for every sensor. Meaning to gather as much light as possible. ISO and shadow priority is a different matter.

Wrong. See what happens on a Canon sensor when you fiddle with ISO at constant exposure (aperture/shutter), i.e. constant gathered light:

(http://www.guillermoluijk.com/article/iso/versus.jpg)

On Sony sensors that improvement is negligible.

ETTR tools:

- Aperture, shutter and ISO for Canon sensors.
- Aperture and shutter for Sony sensors.

The rest will fall somewhere in between.
Title: Re: Re: Re:
Post by: deejjjaaaa on October 29, 2014, 11:29:24 am
On Sony sensors that improvement is negligible.

A7s..........
Title: Re: The Optimum Digital Exposure
Post by: bjanes on October 29, 2014, 11:49:47 am
Exactly what I'm talking about. Whitebalancing and 'exposure' optimization bundled together. Of course that's harder to do in Raw capture, but then one would need to get flexible exposure per channel (while recording white balance as multipliers). When the data is still in linear gamma space, not really all that hard to do.

Yes, after capture, it's easy enough when still in linear gamma space. Depending on the software, there may be some complications with color profiles ('scene referred' versus 'output referred'), also shown by the PV2010 (and later) (non-)predictability. But then, what's one more challenge after eliminating several others.

Cheers,
Bart

Bart,

Thanks for your comments. I am posting the results of a recent experiment with white balance for a red flower using the Nikon D800e for information and further comment. The WB was set to Daylight in the camera (at around 1:30 pm, CDT at 42 N lat on 10/25/2015, -5h GMT) with the Nikon standard Picture Control and file space set to AdobeRGB. The wb multipliers for RGGB are 1.973, 1.0, 1.0, 1.375 as reported by RawDigger's EXIF tool.

The metered exposure was 1/100 s @ f/9. The Red channel shows severe clipping.

(http://bjanes.smugmug.com/Photography/Red-Flower-WB/i-gr7L5WN/1/L/Prev-1-L.png)

However, the raw histogram shows underexposure in all channels, with the red channel about 1.5 EV below saturation.

(http://bjanes.smugmug.com/Photography/Red-Flower-WB/i-6ZNkkkX/0/O/Img-1-RD.png)

For a proper ETTR exposure, it is necessary to increase the exposure, rather than decreasing it as would be indicated by the red channel histogram on the camera. Doubling of the exposure gives a better ETTR exposure, but the red channel is still 0.5 EV below clipping, which would allow for specular highlights if any were present. The red histogram was shifted 1 EV to the right, as expected with the 1 EV exposure increment.

(http://bjanes.smugmug.com/Photography/Red-Flower-WB/i-2LPH5cV/0/O/Img-4-Full-RD.png)

In ACR with PV2012, a negative exposure of -0.65 EV gives a good histogram. Perhaps PV 2010 would give more accurate color, but I used PV2012 since that is what most photographers would likely be using.

(http://bjanes.smugmug.com/Photography/Red-Flower-WB/i-WbbmBGB/1/O/Img-4_ACR%20NegExp%20Dialog.png)

Bill
Title: Re: The Optimum Digital Exposure
Post by: BobD on October 29, 2014, 12:12:45 pm
Bob,

What is your view about the value of the highlight priority metering mode of the D810?

Since it is linked with the spot meter mode of the camera, it seems like a pretty good solution, isn't it?

Cheers,
Bernard

Bernard,
If I'm understanding this feature correctly, it looks like using this function is diametrically opposed to what is needed. It appears as if they are basing the exposure on the highlights which drives the the exposure to the left - underexposing.
"Although spot metering targets a selected area, targeting can be difficult when your subject is in motion. Matrix metering balances exposure over the entire frame, including the background, and may consequently leave highlights overexposed, or "washed out." In contrast, highlight-weighted metering automatically detects and meters highlights for optimal exposure with less washout (and less fiddling with the camera)." As Michael said in an earlier post, "it appears as if they just don't get"!

If you're using the OneZone Method #2: +1.3 EV plus and minus 2/3 EV, I would use the matrix metering.
Using your in camera spot meter can be a little tricky. Here is how I address of spot metering in my book:
"Although you can use your camera’s spot meter, there are a couple of things to consider when doing so:
•   First, the Spot Size: The spot is relatively large and varies in size with focal length of the lens being used. This can be useful if you are using a zoom lens. When you zoom large you narrow the spot meter area and can get a more accurate reading.
Second, the Accuracy of Reading: If your zoom lens does not have a constant f-stop throughout the entire zoom range, zooming larger in order to narrow the spot meter area will produce an inaccurate exposure when you zoom smaller to recompose. (For more information, see more In-Camera Spot Metering Considerations in the Appendix)"


Bob
Title: Re: The Optimum Digital Exposure
Post by: BobD on October 29, 2014, 12:20:06 pm
I agree with Bill, flowers can have amazing saturation thus presenting not only overexposure risk, but some also risk of underexposure, e.g. with near zero Blue and very high Red channel signals (at the same time).
RawTherapee at least offers a solution by allowing to change the per channel pre-demosaicing signal levels, I wish other converters allowed such manual control.

Cheers,
Bart
I also agree that flowers have amazing saturation. However, this can be also remedied in ACR (LR or PS) using the TAT tool to lower the red/increase the blue saturation or by using the Tone Curve by adjusting the red and blue channels.
Title: Re: Re: Re:
Post by: Torbjörn Tapani on October 29, 2014, 12:47:30 pm
Wrong. See what happens on a Canon sensor when you fiddle with ISO at constant exposure (aperture/shutter), i.e. constant gathered light:

[image]

On Sony sensors that improvement is negligible.

ETTR tools:

- Aperture, shutter and ISO for Canon sensors.
- Aperture and shutter for Sony sensors.

The rest will fall somewhere in between.

But if you keep constant exposure (shutter/aperture) then it's not ETTR in the ISO100 example. Am I missing something here?

Was the image ETTR:ed at ISO 100, and then the same exposure made in ISO1600 to produce that result? Then you probably have one of those examples where ETTR would yield lower exposure than metered. Say we have some highlight out of frame that was ETTR:ed for.

But sure. Other than base ISO is good for separation in shadows. This is true even for the near ISO-less D7000 and D800. Which we discussed recently: http://www.luminous-landscape.com/forum/index.php?topic=94383.0

It's absolutely possible I'm flat out wrong but enlighten me.
Title: Re: Re: Re:
Post by: Bart_van_der_Wolf on October 29, 2014, 01:20:04 pm
But if you keep constant exposure (shutter/aperture) then it's not ETTR in the ISO100 example. Am I missing something here?

Correct. All the images show is that for some cameras that increasing the ISO can be beneficial for shadow noise, although it also reduces the dynamic range.

ETTR is all about collecting as many photons as possible, without clipping, to improve Dynamic Range. That's typically low ISO territory.

Cheers,
Bart
Title: Re: The Optimum Digital Exposure
Post by: bjanes on October 29, 2014, 01:29:33 pm
I also agree that flowers have amazing saturation. However, this can be also remedied in ACR (LR or PS) using the TAT tool to lower the red/increase the blue saturation or by using the Tone Curve by adjusting the red and blue channels.

This could be the topic of another thread, but with the image I used as an example, there is no trouble bringing the image into the gamut of ProPhotoRGB with ACR or into the ProPhoto like working space of LR, and I think the best approach would be to optimize image in those spaces. For printing, one would probably not want to eliminate all clipping of the red channel, but to reduce saturation until important image details are not burned out as judged from soft proofing. With LR, one could create a virtual copy for proofing and edit the image in real time. With Photoshop and ACR, one could render into the printer space and perform adjustments there. How would you handle this situation?

Regards,

Bill
Title: Re: The Optimum Digital Exposure
Post by: williamchutton on October 29, 2014, 01:53:33 pm
Since 2011 I followed the succinct advice presented by LuLA member ejmartin (post #88).

http://www.luminous-landscape.com/forum/index.php?topic=56947.msg463071#msg463071

I don't use Canon cameras, so ejmartin's advice has served me well.

Bill Claff's brand specific data on photographic dynamic range shadow improvement vs ISO is interesting.

https://home.comcast.net/%7ENikonD70/Charts/PDR_Shadow.htm
Title: Re: The Optimum Digital Exposure
Post by: kwalsh on October 29, 2014, 03:27:11 pm
Worth mentioning if you are already going to deal with in camera previews that are blown out/over exposed to do ETTR you might as well make those images green and more useful.  Which is to say set a custom WB (sometime called UniWB) that will make your in camera JPEG based histograms more closely match the RAW histograms.  For most cameras this means AWBing on a magentaish target with the end result being green previews.  Depending on the camera the technique to do this can be easy or hard, one shot or iterative.  But in the end you have a custom WB setting that will give you close to RAW histograms.  Google it.

Also I've got to say that writing and encouraging people to go to 99% on SW like LR as a general practice is foolhardy.  Bad things happen near saturation and it can be extremely unforgiving.  A little more shadow noise is easily handled in post processing.  Blown highlights and exciting color errors as a result of non-linearity near saturation capacities of wells is much worse to deal with.  So in that vein I find the "method 2" which is very close to what I use to be much more practical and sensible.  Most people are going to be much better served by being a half stop below "99% optimal" than actually running up against saturation.  Especially when that saturation point is determined rather erroneously from a LR histogram.

The ETTR concept is very sensible.  Going to 99% is introducing new problems without actually improving things in the shadows and mid-tones hardly at all.
Title: Re: Re: Re: Re: Re:
Post by: Guillermo Luijk on October 29, 2014, 04:12:14 pm
But if you keep constant exposure (shutter/aperture) then it's not ETTR in the ISO100 example. Am I missing something here?

Was the image ETTR:ed at ISO 100, and then the same exposure made in ISO1600 to produce that result? Then you probably have one of those examples where ETTR would yield lower exposure than metered. Say we have some highlight out of frame that was ETTR:ed for.

But sure. Other than base ISO is good for separation in shadows. This is true even for the near ISO-less D7000 and D800. Which we discussed recently: http://www.luminous-landscape.com/forum/index.php?topic=94383.0

It's absolutely possible I'm flat out wrong but enlighten me.

What I demonstrated is that ISO is another tool to achieve ETTR on Canon sensors when aperture/shutter don't suffice because of the lighting conditions and DOF/motion blur requirements.

So yes, ETTR is about ISO as well, not only about collecting photons.

Regards
Title: Re: Re: Re:
Post by: ErikKaffehr on October 29, 2014, 04:19:50 pm
Hi,

Guillermo's post illustrates what I was thinking about. The way I see it, ETTR is about maximising the number of electrons captured. But on Canons and some other cameras, increasing ISO will reduce readout noise, so increasing ISO and doing ETTR may make some sense.

On "isoless" cameras I would say that keeping base ISO and exposing as much as possible is the key to achieve maximum image quality.

Best regards
Erik

Wrong. See what happens on a Canon sensor when you fiddle with ISO at constant exposure (aperture/shutter), i.e. constant gathered light:

(http://www.guillermoluijk.com/article/iso/versus.jpg)

On Sony sensors that improvement is negligible.

ETTR tools:

- Aperture, shutter and ISO for Canon sensors.
- Aperture and shutter for Sony sensors.

The rest will fall somewhere in between.
Title: Re: Re: Re: Re: Re:
Post by: Torbjörn Tapani on October 29, 2014, 04:34:27 pm
What I demonstrated is that ISO is another tool to achieve ETTR on Canon sensors when aperture/shutter don't suffice because of the lighting conditions and DOF/motion blur requirements.

So yes, ETTR is about ISO as well, not only about collecting photons.

Regards

Ok. I think I can understand what you mean by that. But I think that is a different situation. Actually it is more like the situation I had in the discussion in the link about unity gain, trying to optimize ISO for when you could not raise exposure(shutter/aperture) due to other constraints.

But if the situation is that you benefit from raising ISO and expose less with aperture/shutter in some case, then I agree, then ETTR is about ISO as well. But in my mind ETTR is about collecting photons.
Title: Re: Re: Re:
Post by: Torbjörn Tapani on October 29, 2014, 04:38:52 pm
Hi,

Guillermo's post illustrates what I was thinking about. The way I see it, ETTR is about maximising the number of electrons captured. But on Canons and some other cameras, increasing ISO will reduce readout noise, so increasing ISO and doing ETTR may make some sense.

On "isoless" cameras I would say that keeping base ISO and exposing as much as possible is the key to achieve maximum image quality.

Best regards
Erik


I think I can understand that view. Not conviced tho. Could you demonstrate a situation where you would raise ISO before you increase exposure (shutter/apterure) with that kind of sensor?
Title: Re: Re: Re: Re: Re: Re: Re:
Post by: Guillermo Luijk on October 29, 2014, 04:39:01 pm
But if the situation is that you benefit from raising ISO and expose less with aperture/shutter in some case, then I agree, then ETTR is about ISO as well. But in my mind ETTR is about collecting photons.

It is not that situation. It is not when you DECIDE to expose less with aperture/shutter, but when you CANNOT capture more photons (e.g. you are using max aperture and slowest shutter to prevent motion blur, and that is still not enough to achieve ETTR).
Title: Re: Re: Re: Re: Re:
Post by: ErikKaffehr on October 29, 2014, 04:39:45 pm
Hi,

I absolutely agree on that!

Best regards
Erik


But in my mind ETTR is about collecting photons.
Title: Re: Re: Re: Re: Re: Re: Re:
Post by: Guillermo Luijk on October 29, 2014, 04:42:36 pm
Hi,

I absolutely agree on that!

Best regards
Erik

And I absolutely disagree. ETTR is about shifting the histogram to the right (maximise RAW levels before clipping). To achieve that on a digital camera there are three variables: aperture, shutter AND ISO, the last one has no connection to captured photons. As simple as that :)
Title: Re: Re: Re: Re: Re: Re: Re:
Post by: Torbjörn Tapani on October 29, 2014, 04:58:07 pm
And I absolutely disagree. ETTR is about shifting the histogram to the right (maximise RAW levels before clipping). To achieve that on a digital camera there are three variables: aperture, shutter AND ISO, the last one has no connection to captured photons. As simple as that :)

Then we can agree to disagree about the definition of ETTR, that is fine by me. As long as we all expose as much as possible first. Then raise ISO.

I would optimize shadow detail with increased ISO in my Nikons as well up to a point. Not to shift the histogram but to take advantage of better shadow S/N ratio.

 

Title: Re: Re: Re:
Post by: deejjjaaaa on October 29, 2014, 04:59:15 pm
I think I can understand that view. Not conviced tho. Could you demonstrate a situation where you would raise ISO before you increase exposure (shutter/apterure) with that kind of sensor?
his situation is = when your best /best = most possible light/ exposure (remember that ISO is not a part of exposure - it is part of your decision about exposure, along w/ other variables) in a particular situation (motion blur, DOF, shutter shock, etc, etc) does not lead to unacceptable clipping (in raw) then w/ some cameras (Canon for example) it makes sense to increase ISO settings as that leads to greater post exposure pre ADC amplification /analog gain/ that leads to better S/N (in /deep/ shadows)... to a certain extent of course /that's why people check their camera model to find out what is the reasonable limit in ISO/
Title: Re: Re: Re: Re: Re: Re: Re: Re: Re:
Post by: Guillermo Luijk on October 29, 2014, 05:04:05 pm
Not to shift the histogram but to take advantage of better shadow S/N ratio.

Both are the same thing Torb, the purpose of ETTR is to reduce visible noise :)
Title: Re: Re: Re:
Post by: deejjjaaaa on October 29, 2014, 05:04:48 pm
On "isoless" cameras I would say that keeping base ISO and exposing as much as possible is the key to achieve maximum image quality.

there is a school of thought that says that some raw converters (along with their "camera profiles" that drive the color transforms/etc) are not doing a good job in deep shadows... so sometimes your decisions (like w/ clipping - some converters are not good w/ inventing eye pleasing fake colors/transitions where you have clipping in raw) depend on your raw converter as well.
Title: Re: The Optimum Digital Exposure
Post by: Torbjörn Tapani on October 29, 2014, 05:05:11 pm
deejjjaaaa, then we are doing the exact same thing. Only in a Canon you have lower read noise with increased ISO and analog gain to much higher ISOs. You would do the same thing with a D4. In other Nikons, I have a D7000 and D800 you would not gain much above where analog gain ends. About 800 and 1600 ISO respectively.
Title: Re: Re: Re: Re: Re: Re: Re:
Post by: Bart_van_der_Wolf on October 29, 2014, 05:10:30 pm
It is not that situation. It is not when you DECIDE to expose less with aperture/shutter, but when you CANNOT capture more photons (e.g. you are using max aperture and slowest shutter to prevent motion blur, and that is still not enough to achieve ETTR).

Hi Guillermo,

Let's not get too deep into semantics, but EXPOSE to the right (ETTR) is all about photons, exposure, and allows to optimize Dynamic Range. Boosting gain on a limited given amount of photons may be beneficial, but IMHO it's something else.

Cheers,
Bart
Title: Re: The Optimum Digital Exposure
Post by: Fine_Art on October 29, 2014, 05:12:32 pm
Thanks for your comments Bob.

I'm just waiting for one of the companies to file a patent for it. I have emails going back at least 10 years that would count as "prior art".

Cheers,

Michael


I believe, from having gone through the patent process, that "prior art" would be prior patents or physical objects that have been made. Without question you have copyright, which is some ways is more powerful, as it does not expire so fast. For example, I went to do a wall patent. A complex set of materials with a method of fast placement. My lawyer, a MSc in engineering, said yes it works, it is novel, etc. All is a go. The problem became a big company, and they are really big in construction, would bury me in a legal fight that forces my abandonment of the patent. So I have building designs that I submit in permits for building. The copyright on the design is more powerful than if I tried for a patent. Anyone submitting a permit worldwide can automatically be in copyright infringement, while an international patent is a whole new process with more fees.
Title: Re: Re: Re: Re: Re: Re: Re:
Post by: Bart_van_der_Wolf on October 29, 2014, 05:17:15 pm
Then we can agree to disagree about the definition of ETTR, that is fine by me. As long as we all expose as much as possible first. Then raise ISO.

Yes, that's the idea. More photons will reduce shot noise, and if there are enough to almost saturate the sensor wells we have done well. In case there are restrictions to the number of photons, then there are some camera models that benefit in the Photon starved shadows by making sure that the read noise doesn't start contributing to the total noise more than necessary, and boosting amplification/gain with the ISO setting can help (more on some cameras than others).

Quote
I would optimize shadow detail with increased ISO in my Nikons as well up to a point. Not to shift the histogram but to take advantage of better shadow S/N ratio.

Yep.

Cheers,
Bart
Title: Re:
Post by: Guillermo Luijk on October 29, 2014, 05:19:10 pm
I am being practical. I have zero interest to find out why the first guy who used the term ETTR (was Michael BTW?) didn't call it HTTR (Histogram To The Right), for instance.

The true is that on many cameras (Canons, many Nikons,...) there is a way to improve a lot image quality when exposure cannot be increased but we can still push RAW levels through ISO without clipping.

If that is not ETTR for you, ok take the term HTTR as a more general rule that covers ETTR and takes it beyond! :)

In fact in Spanish the term ETTR is usually translated as a terrific expression: "Derecheo del histograma", i.e. "Histogram rightization" or simply... HTTR. It's ugly but I believe it makes sense.
Title: Re: The Optimum Digital Exposure
Post by: telyt on October 29, 2014, 06:21:33 pm
The article makes perfect sense.

Why don't our cameras, especially any over $500, have an ETTR exposure mode built in? Surely it is just a software addition to the metering reading?

People should be able to specify 100% within the matrix or 99.5% within the matrix in ETTR mode. The manufacturers should understand exposure enough to build this in.

Another automated feature to ignore and/or disable.  Unless the software knows what highlights are not important and can go to 255/255/255 it will take some user intervention.  Might as well use manual exposure.
Title: Re: The Optimum Digital Exposure
Post by: duane_bolland on October 29, 2014, 06:27:41 pm
I think I found a minor mistake in the article.  Under method #2: "The exposure that shows no red clipping in highlight detail areas is the optimum exposure of the set. In this case EV+1.3."  No, I think that should be the EV+.6 exposure.  It goes on to say that either (+1.3 or +.6) would work in this case.  But clearly the +.6 has the least clipping.

Nice article.  It was insightful.  I push my exposure to the right often and essentially never toast it beyond salvage.   
Title: Re: The Optimum Digital Exposure
Post by: Fine_Art on October 29, 2014, 06:45:25 pm
I think I found a minor mistake in the article.  Under method #2: "The exposure that shows no red clipping in highlight detail areas is the optimum exposure of the set. In this case EV+1.3."  No, I think that should be the EV+.6 exposure.  It goes on to say that either (+1.3 or +.6) would work in this case.  But clearly the +.6 has the least clipping.

Nice article.  It was insightful.  I push my exposure to the right often and essentially never toast it beyond salvage.   

You cant see the details so you do not know why he picks +1.3. Is it a wet roof? He is talking about specular highlights. We all agree the central idea is right.
Title: Re: Re: Re: Re: Re: Re: Re:
Post by: bjanes on October 29, 2014, 08:28:35 pm
Hi Guillermo,

Let's not get too deep into semantics, but EXPOSE to the right (ETTR) is all about photons, exposure, and allows to optimize Dynamic Range. Boosting gain on a limited given amount of photons may be beneficial, but IMHO it's something else.

Cheers,
Bart

Bart, I agree entirely with you here. However, those of us who have been following the ETTR issue over the years will recall that the rationale for ETTR was initially based on the false premise that moving the histogram to the right made use of the greater number of theoretical levels in the upper ranges of the file: half of the levels are in the brightest f/stop of a linearly encoded raw file. I recall many heated exchanges on online forums regarding this topic. Emil Martinec debunked this rationale in his marvelous treatise (http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p3.html#ETTR) posted on the University of Chicago web site. Increasing the ISO moves the histogram to the right, but does not change the exposure and has little effect with an ISO-less camera. With many older Nikons and many Canons where the electronics downstream to sensor limit dynamic range at low ISO, increasing the ISO will reduce the read noise and improve the SNR but this benefit maxes out at ISO 1600 or thereabouts (see Emil's Fig 12a for the Canon 1D3). Further increases in ISO will move the histogram further to the right and brighten the LCD image preview, but will limit highlight headroom with no improvement in image quality.

The concept of maximizing SNR by increasing exposure is an important concept, but the acronym ETTR is somewhat misleading; however, it has become so firmly established in the photography lexicon that I doubt that it will be renamed.

Bill
Title: Re: Re: Re: Re: Re: Re: Re:
Post by: Fine_Art on October 29, 2014, 09:30:49 pm
Bart, I agree entirely with you here. However, those of us who have been following the ETTR issue over the years will recall that the rationale for ETTR was initially based on the false premise that moving the histogram to the right made use of the greater number of theoretical levels in the upper ranges of the file: half of the levels are in the brightest f/stop of a linearly encoded raw file. I recall many heated exchanges on online forums regarding this topic. Emil Martinec debunked this rationale in his marvelous treatise (http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p3.html#ETTR) posted on the University of Chicago web site. Increasing the ISO moves the histogram to the right, but does not change the exposure and has little effect with an ISO-less camera. With many older Nikons and many Canons where the electronics downstream to sensor limit dynamic range at low ISO, increasing the ISO will reduce the read noise and improve the SNR but this benefit maxes out at ISO 1600 or thereabouts (see Emil's Fig 12a for the Canon 1D3). Further increases in ISO will move the histogram further to the right and brighten the LCD image preview, but will limit highlight headroom with no improvement in image quality.

The concept of maximizing exposure by increasing exposure is an important concept, but the acronym ETTR is somewhat misleading; however, it has become so firmly established in the photography lexicon that I doubt that it will be renamed.

Bill

So lets talk about what limits exposure. The need to control the look of motion. If not for that, we would all shoot base ISO, long exposure, capturing as many photons as possible. Is there anything else? What about several shots stacked? If you can mask moving areas, hopefully adding the captures of dark areas, you are further ahead. That assumes you need to bother. Low ambient light conditions can still be a problem.
Title: Re: The Optimum Digital Exposure
Post by: jpegman on October 29, 2014, 11:16:14 pm
I think a lot of people are getting lost in the trees here.  Seems to me the premise is pretty simple.

1. If you are 2 stops underexposed (90%) you are using only 25% of your sensor data and effectively have thrown away 75% of your data; If you want to stop at the 97% level and don't want to go to the 99% level because it too risky you have discarded 50% of your sensor data - your choice! George Jardine documented this in his article referenced in the DiNatale's LuLa post (http://mulita.com/blog/?p=3358#Tonal-Compression) -it's real no matter what camera or sensor or raw converter one chooses - it's the nature of current digital capture technology!

2. The article was written for someone using Adobe ACR and works fine with my Canon 6D. If you use something else your mileage may vary and you will have to work it out for yourself with your specific software and tools.

3. Can you get a "full range image with only 25%-50% of your data - sure if you are willing to accept the 25%-50% loss in YOUR shadow data areas. If no one pixel peeps in the shadows - they may never know what they are missing.

4. Before this article, I would have never thought that pushing the white rabbit in snow even further than classic compensation recommended would result in a better image, and shooting the black cat in a coal bin image so the image was medium grey or LIGHTER would have been total heresy - yet our modern raw converters can bring it back in spades!

5. As Mark absolutely correctly noted (reply #20): "For the modern RAW "digital negative" workflow this classic rule can be stated as just the reverse, i.e.,  "Expose for the highlights and develop for the shadows". "ETTR" is just another way of saying the same thing, and it's also how we had to expose color reversal film except that we had no real control over the development of the shadows with color reversal film. With todays RAW image editors we do." As

6. How you get there is just a technique and the methodology presented seems both technically correct and even if it appears somewhat simplistic, it is doable (With no additional equipment than what you already have) and very easy to automate using camera custom functions and Lightroom stacks where the only cost is 2 extra exposures using extra digital card storage space. One doesn't have to do it for every shot just the ones you want to have both maximum data and maximum preserved shadow quality.

7. I have spent the last 2 days going over the calibrations and field testing and all I can say is IT WORKS!

8. As they say, the proof is in the results.

Jpegman
Title: Re: The Optimum Digital Exposure
Post by: BobD on October 30, 2014, 12:30:42 am
I think I found a minor mistake in the article.  Under method #2: "The exposure that shows no red clipping in highlight detail areas is the optimum exposure of the set. In this case EV+1.3."  No, I think that should be the EV+.6 exposure.  It goes on to say that either (+1.3 or +.6) would work in this case.  But clearly the +.6 has the least clipping.

Duane, I'm not sure it is a mistake... just choices. I thought this might be confusing but I wanted to bring up the concept that "Highlight Clipping" is acceptable when there is specular reflection or bright sunlit snow (a.k.a. Zone "X").  Notice the "Note" under the example:
"Note: There can be some red highlight warning in an optimum exposure if the photo includes sunlit bright snow, specular reflections, etc.. In the example above you might choose the middle +0.6 EV exposure, however, the red “Highlight Clipping” on the roof in the first exposure, +1.3 EV, could also be used. The bright snow covered roofs in bright sunlight would look natural being 100% white with no texture. Remember the expression of the Artist’s Vision is what matters."

Although "you might choose the middle +0.6 EV exposure"... "the first exposure, +1.3 EV, could also be used."  When ever I have choices like this example I usually choose the brighter exposure. Hope that helps.
Title: Re: The Optimum Digital Exposure
Post by: BobD on October 30, 2014, 12:46:38 am
I think a lot of people are getting lost in the trees here.  Seems to me the premise is pretty simple.
...8. As they say, the proof is in the results. Jpegman
Jpegman,
Thanks for your clarification and perspective.  I am an empirical kind of guy, so when I read an article (like the one I wrote here)... I try to think it through but I always need to try it.

As you say the proof is in the doing.  I am not looking for people to adopt my method buy adapt the concept with their photography.

I can only hope that those that try it have the same quality results that I have experienced.

Thanks Again
Title: Re: Re: Re: Re: Re: Re: Re:
Post by: deejjjaaaa on October 30, 2014, 01:34:11 am
but this benefit maxes out at ISO 1600 or thereabouts (see Emil's Fig 12a for the Canon 1D3).
Emil's tested cameras long time ago... nowadays Aptina's approach ( http://www.aptina.com/products/technology/DR-Pix_WhitePaper.pdf ) in some cameras can kick in @ higher "ISO" (= Sony A7s for example)... so ISO1600 or thereabouts is not set in stone, is it ?
Title: Re: The Optimum Digital Exposure
Post by: trichardlin on October 30, 2014, 03:21:03 am
...

Why don't our cameras, especially any over $500, have an ETTR exposure mode built in? Surely it is just a software addition to the metering reading?
...

Imagine a family on vacation.  With hundreds or even thousands of pictures taken in this 'optimal' mode.  The poor dad has to individually adjust all of them in post to get usable pictures to show friends and families.  It's just too much work to get arguably marginal improvement in picture quality (depends on the scene and what's important in the picture).  In addition, you could suddenly lose a couple of stops, resulting in more blurry photos due to longer shutter speed.  So, if you are not careful, ETTR can give you more bad photos than not using ETTR.

Title: Re: Re: Re: The Optimum Digital Exposure
Post by: Guillermo Luijk on October 30, 2014, 04:05:31 am
Imagine a family on vacation.  With hundreds or even thousands of pictures taken in this 'optimal' mode.  The poor dad has to individually adjust all of them in post to get usable pictures to show friends and families.  It's just too much work to get arguably marginal improvement in picture quality (depends on the scene and what's important in the picture).  In addition, you could suddenly lose a couple of stops, resulting in more blurry photos due to longer shutter speed.  So, if you are not careful, ETTR can give you more bad photos than not using ETTR.

Having the AETTR feature doesn't mean you have to use it all times, as simple as that. That family could spend their vacation happily producing tons of nice JPEGs without worrying about ETTR or even RAW.
Title: Re: The Optimum Digital Exposure
Post by: ErikKaffehr on October 30, 2014, 04:06:52 am
Hi,

Just an example:

This is an image that in all probability was exposed using both blinkies and histogram on my P45+. Histogram in Lightroom indicates significant underexposure:

(http://echophoto.dnsalias.net/ekr/Articles/PostExposure/P45+/ImportedImage.jpg)

Below is the same image in RawDigger, note that green channel is very close to clipping. Any additional exposure of original raw image would cause reconstruction of skym as green channel would be clipped.This is what the sensor sees:
(http://echophoto.dnsalias.net/ekr/Articles/PostExposure/P45+/RawImage.jpg)

Developed image below. More exposure would be beneficial for shadows, but may have reconstruction artefacts in sky.
(http://echophoto.dnsalias.net/ekr/Articles/PostExposure/P45+/DevelopedImage.jpg)

Best regards
Erik
Title: Re: The Optimum Digital Exposure
Post by: laughingbear on October 30, 2014, 05:08:06 am
Greetings,

if there was one single article that nailed it for me, then it was this one:

http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p3.html#ETTR (http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p3.html#ETTR)

in conjunction with the Niagara falls example from Jeff Schewe here:

http://schewephoto.com/ETTR/ (http://schewephoto.com/ETTR/)

Best
Georg
Title: Re: The Optimum Digital Exposure
Post by: Jonathan Cross on October 30, 2014, 05:52:26 am
What a fascinating article and thread.  I have been conscious of ETTR that Michael espoused and has rightly promoted for a long time and I try to put it into practice most of the time.  I am also vastly impressed by the efforts contributors to this thread have made to get the very best quality images.  There are times, however,  e.g. photographing wild life or 'grabbing the moment' when ETTR is difficult to employ.  I am also aware, rightly or wrongly, of images, particularly b&w, where there are areas of deep black that add to the visual impact.

All this is a preamble to my view that, 'It all depends.'.  Yes, of course ETTR is a good idea as it gives flexibility in post processing, BUT...  To me part of the joy of photography is to try to get it right in camera and not have to do huge amounts of post processing.  Getting it right in camera, involves having an idea of how I want the final image to look.  If I am visualizing a b&w image or a graphic one with use of black as an important component, then my exposure considerations will be different to when trying to get an image of a scene with a wide range of subtle tonal and colour variation.  To me ETTR is an important consideration, but not the only one.

Thanks to all LULA contributors for making me think!

Jonathan
Title: Re: The Optimum Digital Exposure
Post by: BobD on October 30, 2014, 09:38:31 am
There are times, however,  e.g. photographing wild life or 'grabbing the moment' when ETTR is difficult to employ.

To me part of the joy of photography is to try to get it right in camera and not have to do huge amounts of post processing.

"There are times, however,  e.g. photographing wild life or 'grabbing the moment' when ETTR is difficult to employ."
This is the beauty the OneZone method #2. You set your exposure for EV +1-1/3 then bracket +/-2/3 of the stop!. If you put these settings in one of your camera presets, when the time comes for "grabbing the moment" you just simply hold your finger on the shutter button (capturing 3 exposures) then go on to the next scene.

I also understand "...the joy of photography is to try to get it right in camera".
However, the optimum exposure IS getting it right in the camera (placing the brightest part of the scene at 99+% brightness in your software). Think of the "huge amount of post processing" being a commitment to your craft.

I know many photographers think getting the "final" exposure in the camera it is a badge of honor... that badge can get heavy.

A more correct statement may be ...to (you) part of the joy of photography is to get "the final exposure" in the camera. But you do realize that this is not the optimum and the results  are that "digital signature" that we have unfortunately come to accept.

Thanks for your thoughts.
Bob
Title: Re: The Optimum Digital Exposure
Post by: BobD on October 30, 2014, 09:51:36 am
Imagine a family on vacation.  With hundreds or even thousands of pictures taken in this 'optimal' mode.  The poor dad has to individually adjust all of them in post to get usable pictures to show friends and families.  It's just too much work to get arguably marginal improvement in picture quality (depends on the scene and what's important in the picture).  In addition, you could suddenly lose a couple of stops, resulting in more blurry photos due to longer shutter speed.  So, if you are not careful, ETTR can give you more bad photos than not using ETTR.


I agree... this is the time for Jpegs or iPhones.  As I say in my book:
Where the OneZone May Not Be Useful
There are shooting situations where the OneZone may not be practical - such as event photography. If you are “Wi-Fi-ing” hundreds of images to the sports photo editor or shooting hundreds of photos at a wedding ceremony the OneZone method will be unwieldy...
...you can take advantage of having a camera with custom presets. Simply move off the  Custom setting that has the OneZone settings, shoot and upload the  JPGs as normal, then return to the custom setting containing the OneZone settings.

If you place you OZ Method #2 in one of your camera presets... then when "poor dad"comes across a wonderful scenic, can just go into the custom setting mode take the 3 OneZone brackets set then quickly go back to shooting JPEG.

Thanks for your input
Bob

Title: Re: Re: Re: Re: Re: Re: Re:
Post by: Ray on October 30, 2014, 10:25:08 am
And I absolutely disagree. ETTR is about shifting the histogram to the right (maximise RAW levels before clipping). To achieve that on a digital camera there are three variables: aperture, shutter AND ISO, the last one has no connection to captured photons. As simple as that :)

I have to agree with Guillermo's logic here. The expression ETTR clearly refers to exposures which shift the histogram to the right without clipping wanted highlights. What else could it mean other than 'Expose To The Right' of the histogram? It certainly doesn't mean 'expose to the right of my left ear lobe', or 'expose to the right of my little toe'.  ;D

Although it's true that an ETTR at base ISO will result in the maximum number of photons captured without clipping relevant parts of the scene, the fact is, when using a higher-than-base ISO, the same principles of ETTR apply, but apply with a significant reduction in the number of captured photons, compared with an ETTR at base ISO.
Title: Re: The Optimum Digital Exposure
Post by: deejjjaaaa on October 30, 2014, 10:49:58 am
Imagine a family on vacation.  With hundreds or even thousands of pictures taken in this 'optimal' mode.  The poor dad has to individually adjust all of them in post to get usable pictures to show friends and families.

the poor dad uses OOC JPG

Title: Re: Re: Re: The Optimum Digital Exposure
Post by: deejjjaaaa on October 30, 2014, 10:53:43 am
Having the AETTR feature doesn't mean you have to use it all times, as simple as that. That family could spend their vacation happily producing tons of nice JPEGs without worrying about ETTR or even RAW.
and not only that... if a manufacturer now suggests poor dad some exposure that prev. poster thinks is good for a raw conversion unaltered, then that manufacturer can simply write a tag in raw file hinting to raw converters how to dial it back, during raw conversion, exposure-wise, from ETTR exposure back to the one camera is using today... then poor dad just need a raw converter support to use that tag automatically - so it is a solvable problem @ code level (but certainly as usual w/ all issues related to automated batch raw conversion w/o manual attention to each shot).
Title: Re: Re: Re: Re: Re: Re: Re:
Post by: Bart_van_der_Wolf on October 30, 2014, 11:23:04 am
I have to agree with Guillermo's logic here. The expression ETTR clearly refers to exposures which shift the histogram to the right without clipping wanted highlights. What else could it mean other than 'Expose To The Right' of the histogram? It certainly doesn't mean 'expose to the right of my left ear lobe', or 'expose to the right of my little toe'.  ;D

Although it's true that an ETTR at base ISO will result in the maximum number of photons captured without clipping relevant parts of the scene, the fact is, when using a higher-than-base ISO, the same principles of ETTR apply, but apply with a significant reduction in the number of captured photons, compared with an ETTR at base ISO.

It depends on what one's Expose to the right goal is. If the goal is a right aligned histogram, then there are several routes possible, even postprocessing. If one's goal is to optimize overall image quality/dynamic range, then increasing  the number of Photons is the path.

Boosting ISO / gain has some merits, but will not produce the same high level of Dynamic range. If the subject contrast is moderate, it might be a good alternative, especially on certain Canon camera models. In some fields of photography, where relatively short exposure time is important, it offers another trade-off to choose from. But that's IMHO not about ETTR, it's rather about optimizing noise characteristics of mostly shadows (and the more limited DR dictates shorter exposures to avoid clipping).

Cheers,
Bart
Title: Re: Re: Re: Re: Re: Re: Re:
Post by: deejjjaaaa on October 30, 2014, 11:48:53 am
But that's IMHO not about ETTR, it's rather about optimizing noise characteristics of mostly shadows
from a practical standpoint optimizing exposure (first of all) and - if possible - gain (after exposure consideration) shall be always considered together ...

PS: probably some different abbr. shall be used - like Ettr&GO ( exposure ttr & gain optimization ) or whatever
Title: Re: The Optimum Digital Exposure
Post by: williamchutton on October 30, 2014, 11:56:29 am
When the single goal is shifting the histogram to the right, it is possible to do more harm than good. Simply increasing ISO alone will shift the histogram. But the analog exposure of the sensor when the shutter is open is not increased. The SNR and DR range may not be optimal compared to using a different shutter time and, or aperture with a lower ISO parameter.

This is how come LuLA member ejmartin's advice from 2011 to maximize exposure (the analog signal recorded by the sensor when the shutter is open) is less ambiguous than the ETTR acronym. Member ejmartin suggested the new "mantra" should be ME. ME often produces a histogram that coincidentally resembles an ETTR histogram.
Title: Re: The Optimum Digital Exposure
Post by: bjanes on October 30, 2014, 01:30:25 pm
1. If you are 2 stops underexposed (90%) you are using only 25% of your sensor data and effectively have thrown away 75% of your data; If you want to stop at the 97% level and don't want to go to the 99% level because it too risky you have discarded 50% of your sensor data - your choice! George Jardine documented this in his article referenced in the DiNatale's LuLa post (http://mulita.com/blog/?p=3358#Tonal-Compression) -it's real no matter what camera or sensor or raw converter one chooses - it's the nature of current digital capture technology!


This business about the number of possible tonal levels in the brightest f/stop or the brightest 2 f/stops is what Emil debunked in the referenced article. The important thing here is the signal to noise ratio (SNR), which varies with the square root of the number of photons collected. Two stops underexposed means the SNR is cut in half. The number of possible perceived tones will be limited by noise, not the reduced number of encodable tones using only 25% of the range of the ADC.

Bill
Title: Re: The Optimum Digital Exposure
Post by: MarkL on October 30, 2014, 05:08:46 pm
ErikKaffehr's exaple is interesting. I know I have had shots where the histogram in LR shows no clipping but colour information has clearly been lost.

A similar example is here: http://diglloyd.com/blog/2014/20140812_2112-NikonD810-MiningShack.html
Title: Re: Re: Re: The Optimum Digital Exposure
Post by: trichardlin on October 30, 2014, 05:56:53 pm
Having the AETTR feature doesn't mean you have to use it all times, as simple as that. That family could spend their vacation happily producing tons of nice JPEGs without worrying about ETTR or even RAW.

Of course.  Remember we were just guessing why camera manufactures don't build this function into the camera.  If only 1% of the skilled customers need this function, it probably won't make it into the final product, especially if the function is tricky to implement or confusing.

Title: Re: The Optimum Digital Exposure
Post by: jpegman on October 30, 2014, 06:04:55 pm
This business about the number of possible tonal levels in the brightest f/stop or the brightest 2 f/stops is what Emil debunked in the referenced article. The important thing here is the signal to noise ratio (SNR), which varies with the square root of the number of photons collected. Two stops underexposed means the SNR is cut in half. The number of possible perceived tones will be limited by noise, not the reduced number of encodable tones using only 25% of the range of the ADC.

Bill

Specific numbers aside -(I spent quite some time going over Emil's paper and was more confused when I was finished about what he was (or was not) trying to say. Glad I didn't have him as an prof at Tech!

However, academics aside, when you collect as many photons via ETTR as you can on the sensor, your data is increased - common sense.
The impact is easy to understand - More input data = More RAW converter data for image processing. The result I SEE is less shadow noise!

Empirical testing shows what Bob demonstrated in his "optimum ISO Comparison" figure where the same sensor/subject/light when "Optimally" underexposed by 2 stops has better s/n than the low ISO (200) image and much better than the "2 stop non-ETTR" image.
What is there to argue about specific number like 25%, 50% or just better images if you want them. I rather be in the company of Bruce Frasier or George Jardine with their imperfect math rather than Emil's analytical corner.

The important thing her is NOT what actually drives the SNR, but, the proposed process will improve your images visually - which is how I evaluate my images. My simplistic conclusion is - if it makes my images better, than I use it until a better paradigm comes along to replace it!

Jpegman
Title: Re: Re: Re: Re: Re: Re: Re:
Post by: Ray on October 30, 2014, 07:22:21 pm
It depends on what one's Expose to the right goal is. If the goal is a right aligned histogram, then there are several routes possible, even postprocessing. If one's goal is to optimize overall image quality/dynamic range, then increasing  the number of Photons is the path.

Not really, Bart. An exposure is an exposure and is something that takes place only once at the time the shot is taken. No amount of post processing can change the exposure. We haven't invented time travel yet.  ;)

The concept of ETTR, regardless of the number of photons captured, and regardless of the ISO setting used, implies that the histogram should be aligned to the right to achieve the minimum amount of noise in relation to the other considerations that always apply when taking a shot, such as lighting conditions, subject movement and depth of field.

If one's goal is to optimise overall image quality, then increasing the number of photons captured is not necessarily the path. It might result in a blurry image due to the use of a shutter speed which is too slow, for example.

Title: Re: Re: Re: The Optimum Digital Exposure
Post by: BernardLanguillier on October 30, 2014, 08:22:53 pm
Of course.  Remember we were just guessing why camera manufactures don't build this function into the camera.  If only 1% of the skilled customers need this function, it probably won't make it into the final product, especially if the function is tricky to implement or confusing.

How about the highlight spot exposure capability of the D810?

Cheers,
Bernard
Title: Re: Re: Re: Re: Re: Re: Re:
Post by: Fine_Art on October 30, 2014, 09:07:25 pm
Not really, Bart. An exposure is an exposure and is something that takes place only once at the time the shot is taken. No amount of post processing can change the exposure. We haven't invented time travel yet.  ;)

The concept of ETTR, regardless of the number of photons captured, and regardless of the ISO setting used, implies that the histogram should be aligned to the right to achieve the minimum amount of noise in relation to the other considerations that always apply when taking a shot, such as lighting conditions, subject movement and depth of field.

If one's goal is to optimise overall image quality, then increasing the number of photons captured is not necessarily the path. It might result in a blurry image due to the use of a shutter speed which is too slow, for example.



Lets put the sequence down (manual). The photographer takes the shot based on limits of A and E, which are desired depth of field and motion blur. The sensor has an unchangeable full well capacity of photons.  That is converted to a charge which the A/D converters convert to a closest number. 

ETTR is about letting the raw file use the largest numbers, which makes the whole image have the finest gradations. It is not about the original number of photons exposed. You can use a gain (higher ISO) to get to those finer bins.
Title: Re: Re: Re: The Optimum Digital Exposure
Post by: dreed on October 30, 2014, 10:20:05 pm
How about the highlight spot exposure capability of the D810?

This and a similar feature in Canon DSLRs results in a different tone curve being used to render the JPEG from raw. It has no impact on the way in which raw data (or the exposure) is made.
Title: Re: Re: Re: The Optimum Digital Exposure
Post by: dreed on October 30, 2014, 10:22:11 pm
Of course.  Remember we were just guessing why camera manufactures don't build this function into the camera.  If only 1% of the skilled customers need this function, it probably won't make it into the final product, especially if the function is tricky to implement or confusing.

Then how do you justify the presence of AFMA (auto-focus micro adjust) settings? These are even harder to use properly than any auto-ETTR would.
Title: Re: The Optimum Digital Exposure
Post by: Jonathan Ratzlaff on October 30, 2014, 10:28:28 pm
This is an interesting article and discussion.  For those of us who grew up shooting film there is an analogy. The standard metering on the camera is analogous to shooting transparency film, where the fear was blowing out highlights and letting the shadows fall where they will.  While shooting negative film we used to expose for the shadows because the exposure latitude of the film would hold the highlights.  The problem there was the narrow latitude of the printing paper.  However we used to overexpose negative film by one or two stops as a matter of course when doing weddings to ensure proper shadow detail..
Now with the high dynamic range of the sensor we again have to go back to exposing for shadows.  The empirical part is learing how the sensor handles the highlights and how much we can force it. It's akin to learning how a given film behaves.

The good part is that we no longer have to deal with the constraints of photo printing paper as we can make the adjustments later.  Once again, Jeff's image of Niagara Falls was the first example of this that I ran across.

Now all I have to do is start applying it.
Title: Re: Re: Re: Re: Re: Re: Re:
Post by: bjanes on October 30, 2014, 11:25:01 pm
Not really, Bart. An exposure is an exposure and is something that takes place only once at the time the shot is taken. No amount of post processing can change the exposure. We haven't invented time travel yet.  ;)

The concept of ETTR, regardless of the number of photons captured, and regardless of the ISO setting used, implies that the histogram should be aligned to the right to achieve the minimum amount of noise in relation to the other considerations that always apply when taking a shot, such as lighting conditions, subject movement and depth of field.

If one's goal is to optimise overall image quality, then increasing the number of photons captured is not necessarily the path. It might result in a blurry image due to the use of a shutter speed which is too slow, for example.

Ray,

I think that is not quite correct regarding the histogram to the right. Let's assume that you are shooting with a so called ISOless camera like the D800 in dim light. You would select an aperture small enough for adequate depth of field and a high enough shutter speed to freeze action. Once you have selected the exposure as determined by f/stop and shutter speed, you could take the shot at base ISO. In this case, the histogram will be far to the left. You then increase "exposure" in the raw converter. Its called exposure in ACR or LR, but it is really increasing the gain. Or you could increase the ISO on the camera to move the histogram to the right and take the shot with the same shutter speed and f/stop. The final result would be the same, except that with the higher ISO you have less highlight headroom.

Bill
Title: Re: Re: Re: The Optimum Digital Exposure
Post by: BernardLanguillier on October 30, 2014, 11:49:31 pm
This and a similar feature in Canon DSLRs results in a different tone curve being used to render the JPEG from raw. It has no impact on the way in which raw data (or the exposure) is made.

I don't think that this is correct. The highlight priority mode in the D810 attempts to preserve highlight clipping in the area where the spot metering is performed.

So it does have an impact on exposure, not just about the way the image is rendered.

Cheers,
Bernard
Title: Re: The Optimum Digital Exposure
Post by: BobD on October 31, 2014, 01:04:52 am
Now with the high dynamic range of the sensor we again have to go back to exposing for shadows...

"Now with the high dynamic range of the sensor we again have to go back to exposing for shadows...[/i]]"
OK, call it macaroni...so lets expose for the shadows.  And how do we do that in digital photograph?... by placing the the brightest desired value in our scene at the [Optimum White Point]... 99+% in our software!

There - "Macaroni". And our shadows are not placed but are dragged and fall on the chip based on the dynamic range of the scene.
If that DR is greater then our sensor's DR then we need HDR... a option we got to much too quickly if you ask me!

Maybe we should stop calling it ETTR - to generic and more a description than a measurement.  Let's call it what it is - the Optimum White Point which is a measurement... 99+% brightness in your camera raw processing software.

(Forgive me... I keep slipping back to my analytical days at Polaroid where our experiment had to be proved then defined in procedures for other to repeat the result.)
Title: Re: Re: Re: Re: Re: Re: Re:
Post by: Bart_van_der_Wolf on October 31, 2014, 05:02:38 am
Not really, Bart. An exposure is an exposure and is something that takes place only once at the time the shot is taken. No amount of post processing can change the exposure.

Well Ray, that's what the ISO control does as well, it changes the number of exposure photons into a different ADU or DR (Analog to Digital Unit, Digital number), AKA "gain" control. The exposure is unchanged.

The ISO (gain) setting will determine if e.g. each 4 photons will change the resulting ADU in the Raw data by 1 unit (ISO 100), or 1 photon will change the ADU by 1 unit (ISO 400). The exposure (in photons) was the same, the conversion rate was changed. The same can be done in software after the fact. It won't be exactly the same depending on the electronics involved (which can also add noise), or different moments of signal amplification can be used (analog or digital), etc., but the effect will be approx. the same in many cases. There still may be a small ISO/gain increase benefit up to, say, ISO 800. However, it does limit Dynamic range and potentially clips highlights. Doing it in software doesn't risk clipping the highlights.

Cheers,
Bart
Title: Re: Re: Re: Re: Re: Re: Re: Re: Re:
Post by: Guillermo Luijk on October 31, 2014, 05:31:32 am
Boosting ISO / gain has some merits, but will not produce the same high level of Dynamic range.

In the situation ISO would be used to achieve ETTR that is not correct Bart. ISO should be used for ETTR only when aperture/shutter (photons) don't suffice. In that situation at base ISO you have empty levels on the right side of the histogram. Imagine you have one empty stop for a certain situation; pushing ISO from ISO100 to ISO200 will make us GAIN (not loose), nearly one extra stop of Dynamic Range on Canon cameras.

BTW exposure is about collecting photons... why then software developers call 'Exposure' slider something that only affects digital RAW  values?. I don't mean I don't agree exposure is only about light, just to show that digital photography has stablished new paradigms in photography that need to be reinterpreted. A sensor is not a film anymore and ISO is not a sensitivity anymore.
Title: Re: Re: Re: The Optimum Digital Exposure
Post by: Guillermo Luijk on October 31, 2014, 05:41:01 am
However, academics aside, when you collect as many photons via ETTR as you can on the sensor, your data is increased - common sense.
The impact is easy to understand - More input data = More RAW converter data for image processing. The result I SEE is less shadow noise!

You confuse 'more data' with 'more information' which is the important thing in practice. The advantage of ETTR is not having more data but having more information through SNR improvement. A good example of more data but no more information (no SNR improvement) is pushing ISO on an underexposed shot on a Sony sensor. Or in nearly any sensor beyond ISO1600, because you get more RAW levels but the same SNR so there is no practical advantage.

Emil explains this fantastically in the article that produced your headache. A simple test to illustrate:

(http://www.guillermoluijk.com/article/ettr3/poster.gif)

Image on the left has 4 times more data but the same information and robustness against postprocessing. The reason is noise dithering, the same process that takes place in RAW files.
Title: Re: The Optimum Digital Exposure
Post by: jjj on October 31, 2014, 05:46:14 am
Here's a thought. Rather than arguing semantics and theoretical stuff, why don't people show more photos that demonstrate their point of view as Guilermo + Erik usefully did.

There's also another angle on this. Does increased data etc make for a better looking photograph as opposed to a an image that is technically better?
I recall trying ETTR some years back and whilst in theory the image may have been 'better' I prefered the rendition of the 'lower quality' images.
When I have some time I shall retest, as I have different cameras now and LR/ACR has changed considerably too.
Title: Re: The Optimum Digital Exposure
Post by: l_d_allan on October 31, 2014, 06:30:42 am
The developers at "Magic Lantern (http://www.magiclantern.fm/)" have included in their arsenal of addons a mode where you can activate an "Automatic ETTR (http://www.magiclantern.fm/forum/index.php?topic=10808.0)" that uses a histogram built from raw data (http://www.magiclantern.fm/forum/index.php?topic=5149.0) to calculate the exposure.

From my experience, this works exactly as advertised.

Agree. My observation is that use of ML's A-ETTR provides results that are very close to what is advocated in this article. There are parameters to take into account scenes that have negligible to large amounts of spectral highlights that are allowed to be blown out. The ML design seems to closely cooperate with ACR/LR's PV2012 to have a parameter to allow clipping in one channel, or the Green channel(s).

I'd appreciate feedback from the author or experienced practitioners of what the author is describing / advocating, who have Canon DSLR's with ML installed, on how close ML's A-ETTR implements the article's "best practice" for optimal exposure.

I think of ML's A-ETTR as "OneZone Method #3". Or not?

As others have mentioned, I've found RawDigger to be really helpful to better understand what is going on with the sensor, without all the adjustment "behind your back" that PV2012 does. FastRawViewer from the same developer is also very useful, although simplified with less capability.

Quote
Unfortunately it is only available for Canon's EOS line of DSLRs.

Also, agree. ML is perhaps the biggest reason I will stay with Canon DSLR's rather than switching to Nikon or Sony. Their brilliant Dual-ISO really helps with dynamic range, although at the cost of some resolution and additional post processing.

Title: Re: The Optimum Digital Exposure
Post by: jjj on October 31, 2014, 06:37:20 am
Surely, the sort of person who would use/appreciate ETTR would be shooting manually to get the desired result.
Using auto to get something that needs to be done so very precisely seems contradictory.
Title: Re: The Optimum Digital Exposure
Post by: Ray on October 31, 2014, 06:44:55 am

There's also another angle on this. Does increased data etc make for a better looking photograph as opposed to a an image that is technically better?


You should know by now, Jeremy, that all appearances such as 'good-looking', 'ugly', 'beautiful', and so on, exist only in the mind of the beholder.

Technically better is unequivocally technically better, provided the science is sound.

There's no accounting for taste.  ;D
Title: Re: Re: Re: Re: Re: Re: Re: Re: Re:
Post by: Bart_van_der_Wolf on October 31, 2014, 06:53:54 am
In the situation ISO would be used to achieve ETTR that is not correct Bart. ISO should be used for ETTR only when aperture/shutter (photons) don't suffice. In that situation at base ISO you have empty levels on the right side of the histogram. Imagine you have one empty stop for a certain situation; pushing ISO from ISO100 to ISO200 will make us GAIN (not loose), nearly one extra stop of Dynamic Range on Canon cameras.

Dear Guillermo,

Please show me were on this respected website (http://www.dxomark.com/Cameras/Canon/EOS-5D-Mark-III---Measurements) the gain in DR is visible..., I see only decline (see attachement). Of course, Dynamic range is specified as total range, not when underexposing images (which is where S/N ratios for specific luminance levels matter). As I've said before, and we agree on that (I think), with underexposure there are some benefits to (modest ISO increases, more effective on some cameras than others).

Cheers,
Bart
Title: Re: Re: Re: Re: Re: Re: Re: Re: Re:
Post by: Guillermo Luijk on October 31, 2014, 07:02:48 am
Dear Guillermo,

Please show me were on this respected website (http://www.dxomark.com/Cameras/Canon/EOS-5D-Mark-III---Measurements) the gain in DR is visible..., I see only decline (see attachement). Of course, Dynamic range is specified as total range, not when underexposing images (which is where S/N ratios for specific luminance levels matter). As I've said before, and we agree on that (I think), with underexposure there are some benefits to (modest ISO increases, more effective on some cameras than others).

Cheers,
Bart

Bart, this really surprises me coming from you: one thing is the MAXIMUM DR that a sensor could capture for a given ISO value, which DxO measurements represent perfectly and is always decreasing with increasing ISO.

Another story is the particular situation we have here with ETTR: the RAW data are underexposed at base ISO (right side of the histogram empty, you forgot to red highlight that important point in your quote) and we cannot collect more photons with shutter/aperture. In this situation, pushing ISO:
1. Doesn’t make use loose any highlight information because the histogram is empty there, we are just getting closer to ETTR
2. Provides better SNR in the shadows
If we don’t loose HL DR, but gain shadows DR, we are gaining total CAPTURED DR.

MAXIMUM DR is not always the same as CAPTURED DR, they are only the same when perfect ETTR is achieved.
Title: Re: The Optimum Digital Exposure
Post by: Bart_van_der_Wolf on October 31, 2014, 07:04:18 am
You should know by now, Jeremy, that all appearances such as 'good-looking', 'ugly', 'beautiful', and so on, exist only in the mind of the beholder.

Indeed, when did excessive noise ever interfere with our ability to make significant tonal adjustments ...,  Oh wait, it does (which explains many of those noise ridden 'creative' images/masterpieces). Maybe that's why ACR has a masking option in its Detail/sharpening panel, hmm.

Quote
There's no accounting for taste.  ;D

Apparently.

Cheers,
Bart
Title: Re: Re: Re: Re: Re: Re: Re: Re: Re:
Post by: Bart_van_der_Wolf on October 31, 2014, 07:18:38 am
MAXIMUM DR is not always the same as CAPTURED DR

Correct, but you didn't specify that:
Quote
pushing ISO from ISO100 to ISO200 will make us GAIN (not loose), nearly one extra stop of Dynamic Range on Canon cameras

By only mentioning Dynamic range, you cause confusion amongst some readers. You should, IMHO of course, rather call it improved S/N ratio in the shadows (or "underexposed DR"  to make the distinction clear). That might indeed lead to an improvement of the remaining underexposed dynamic range because the noise floor is lowered, for some cameras more than others. Context is important, so it's better to be specific than to assume that people read the entire post. And even then, I may have misinterpreted your " Imagine you have one empty stop for a certain situation;" premise.

Once photon shot noise dominates (due to higher exposure levels), we'll have a better starting point to optimize the technical image quality than by underexposing and boosting ISO. Of course we sometimes do not have the luxury of choice to achieve both, optimal shutterspeed/aperture and enough light to get ETTR exposure levels. That's why people sometimes use additional light-sources or reflectors if possible.

I think we are essentially saying the same thing, but language backgrounds and trying to drive a certain point home, may interfere a bit.

Cheers,
Bart
Title: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re:
Post by: Guillermo Luijk on October 31, 2014, 07:47:17 am
I may have misinterpreted your " Imagine you have one empty stop for a certain situation;" premise.

I think you did. That premise is the key in the situation described: usage of ISO to ETTR when aperture/shutter don't suffice.
Title: Re: Re: Re: The Optimum Digital Exposure
Post by: bjanes on October 31, 2014, 08:02:47 am
You confuse 'more data' with 'more information' which is the important thing in practice. The advantage of ETTR is not having more data but having more information through SNR improvement. A good example of more data but no more information (no SNR improvement) is pushing ISO on an underexposed shot on a Sony sensor. Or in nearly any sensor beyond ISO1600, because you get more RAW levels but the same SNR so there is no practical advantage.

Emil explains this fantastically in the article that produced your headache. A simple test to illustrate:

(http://www.guillermoluijk.com/article/ettr3/poster.gif)

Image on the left has 4 times more data but the same information and robustness against postprocessing. The reason is noise dithering, the same process that takes place in RAW files.

Guillermo,

An excellent demonstration of how noise dithering can mitigate posterization when a reduced number of levels is available. Even without noise, fewer levels are needed than generally recognized, and the number of needed levels is predicted by the Weber-Fechner law. A good explanation is given by Norman Koren (http://www.normankoren.com/digital_tonality.html).

The brightest f/stop of a 14 bit digital file contains half of the 16384 levels or 8192. However, only 70 levels are needed to prevent visible posterizaiton. Since 8192 levels are not needed for any practical purpose, some of them could be discarded and this is what Nikon does when recording visually loss less raw files, as Emil explains in his treatise.

Bill
Title: Re: The Optimum Digital Exposure
Post by: jjj on October 31, 2014, 08:05:29 am
Indeed, when did excessive noise ever interfere with our ability to make significant tonal adjustments ...,  Oh wait, it does (which explains many of those noise ridden 'creative' images/masterpieces). Maybe that's why ACR has a masking option in its Detail/sharpening panel, hmm.
I used to use Kodak recording film [1000 ISO and not known for it's lack of grain] and then pushed it in Acuspeed, a developer good at increasing film speed [3200ISO in this case] but not without increasing film grain. The reason being was that to my mind the end result looked good, because of the grain or what would now be called noise.

Something I've noticed over the years that those who obsess over technical stuff and sneer at 'creative' work tend to produce images that are certainly correctly exposed as well as being nice and sharp, but that's about it regarding the content of the image. I don't know if you are typical of such folk or the first exception to the rule, because you have no links to your work.
Here's a photo for your viewing pleasure,. It's grainy/noisey, the subject isn't even sharp and yet lots people seem to like it.

(http://fotografiamagazine.com/wp-content/uploads/sites/6/2014/04/decisive-moment-henri-cartier-bresson-1.jpg)

Title: Re: The Optimum Digital Exposure
Post by: jjj on October 31, 2014, 08:11:36 am
You should know by now, Jeremy, that all appearances such as 'good-looking', 'ugly', 'beautiful', and so on, exist only in the mind of the beholder.

Technically better is unequivocally technically better, provided the science is sound.
But as the end result is usually  to produce images for people's viewing pleasure and not to simply duplicate reality, technically better may be irrelevant, which was my point. The whole reason why say film images looked good was because they were an interpretation of reality. The medium's limitations is what usually made photos look good.
Title: Re: The Optimum Digital Exposure
Post by: Ray on October 31, 2014, 08:52:39 am
It's grainy/noisey, the subject isn't even sharp and yet lots people seem to like it.

(http://fotografiamagazine.com/wp-content/uploads/sites/6/2014/04/decisive-moment-henri-cartier-bresson-1.jpg)



The moment has been captured so well that lots of people like it despite the fact it's grainy, noisy and lacking sharpness.  ;)
Title: Re: Re: Re: Re: Re: Re: Re:
Post by: Ray on October 31, 2014, 08:58:31 am
Ray,

I think that is not quite correct regarding the histogram to the right. Let's assume that you are shooting with a so called ISOless camera like the D800 in dim light. You would select an aperture small enough for adequate depth of field and a high enough shutter speed to freeze action. Once you have selected the exposure as determined by f/stop and shutter speed, you could take the shot at base ISO. In this case, the histogram will be far to the left. You then increase "exposure" in the raw converter. Its called exposure in ACR or LR, but it is really increasing the gain. Or you could increase the ISO on the camera to move the histogram to the right and take the shot with the same shutter speed and f/stop. The final result would be the same, except that with the higher ISO you have less highlight headroom.

Bill

Hi Bill,
To have a truly ISO-less camera with a wide dynamic range, would be a tremendous feature. However, the D800 and D800E are not really ISO-less, as you mentioned to me over a year ago.

If one underexposes by 6 stops at ISO 100, with the D800E, the DR will be reduced by 6 stops (or 6 EV), which would be equivalent to an ISO 6400 shot with an ISO-less camera, after exposure compensation in ACR.

However, according to DXOMark, if one uses the ISO 6400 setting with the D800E, and the same exposure as a 6 stop underexposure at ISO 100, the exposure will be an ETTR and the DR will be only 5.1 EV lower, instead of 6 EV lower. One gains a significant benefit of 0.9 EV in dynamic range by choosing to use an ETTR exposure at ISO 6400 instead of underexposing 6 stops at ISO 100.

This advantage is not as great as it would be with Canon DSLRs, but is still an advantage.
Title: Re: The Optimum Digital Exposure
Post by: Bart_van_der_Wolf on October 31, 2014, 09:11:26 am
The reason being was that to my mind the end result looked good, because of the grain or what would now be called noise.

Which would be a nice thread topic by itself, Jeremy. Personally, I loath noise when there are better options available to drive the creative intent of an image home. When I look at the real world, I rarely see noise ... Adding noise, by choice of capture modality (if there is a choice to begin with, which I doubt in the case of HCB), can have it's use (e.g. by underlining the grungy feel of an image), but successful use is more exception than rule (more often it is a sign of sloppy technique, and a distraction).

Quote
Something I've noticed over the years that those who obsess over technical stuff and sneer at 'creative' work tend to produce images that are certainly correctly exposed as well as being nice and sharp, but that's about it regarding the content of the image.

That's funny, I just got of the phone with a fellow photographer who mentioned his astonishment about a recent museum exhibition he visited, showing (amongst others) both an original excellent platinum print of an image, and the same thing blown-up to cover a wall, but with purple streaky banding in the shadows, clearly poorly executed technique, no real excuse can be made, especially for a photography museum. Sloppy technique seems to spread, and is rarely used as an intentional creative element.

Quote
I don't know if you are typical of such folk or the first exception to the rule, because you have no links to your work.


Rest assured, I don't aspire to be like any other person. I hope I'm in some sense unique, or at least know my craft skills and my limitations, whatever they are. I also strive to improve my skills to avoid them becoming a distraction.

Quote
Here's a photo for your viewing pleasure,. It's grainy/noisey, the subject isn't even sharp and yet lots people seem to like it.

Are you suggesting it was an intentional creative choice of Henri to have it be noisy, or could the lack of available alternative materials be a part of it? If I'm not mistaken, Henri is also known to have sucked at printing his photo's which is why he let someone else do it for him. He knew his limitations, and didn't (ab)use them as a creative label. He knew he wanted to get the timing of events right. The image is sharp, but there was motion blur which was effective (recognition of the subject would also not have helped), although I do not know how intentional it was recorded by choosing a different shutterspeed.

There's a difference between a successful image despite of technical shortcomings, and one because of the shortcomings ...

Cheers,
Bart
Title: Re: Re: Re: Re: Re: Re: Re:
Post by: bjanes on October 31, 2014, 09:31:20 am
However, according to DXOMark, if one uses the ISO 6400 setting with the D800E, and the same exposure as a 6 stop underexposure at ISO 100, the exposure will be an ETTR and the DR will be only 5.1 EV lower, instead of 6 EV lower. One gains a significant benefit of 0.9 EV in dynamic range by choosing to use an ETTR exposure at ISO 6400 instead of underexposing 6 stops at ISO 100.

This advantage is not as great as it would be with Canon DSLRs, but is still an advantage.

Ray,

Thanks for reminding me of that. The D800 is not truly ISO-less. I stand corrected.

Bill
Title: Re: The Optimum Digital Exposure
Post by: jjj on October 31, 2014, 01:08:48 pm
Which would be a nice thread topic by itself, Jeremy. Personally, I loath noise when there are better options available to drive the creative intent of an image home. When I look at the real world, I rarely see noise ... Adding noise, by choice of capture modality (if there is a choice to begin with, which I doubt in the case of HCB), can have it's use (e.g. by underlining the grungy feel of an image), but successful use is more exception than rule (more often it is a sign of sloppy technique, and a distraction).
Nearly all photos are rubbish, usually through lack of any photographic skills and I'm not talking technical ability here, particularly as getting a sharp in focus image is so very easy nowadays. Successful imagery is the exception, full stop.
Personally I really like lo-fi imagery, but it is no easier to make look good than technically perfect shots. Possibly harder if anything.
I actually like grain or noise in images, to me it's in fact more real. Why? Because the world is full of texture, unlike than plastic smooth, digital images that have a video quality.
However banding in shadows or colour noise is something I cannot stand. LR/ACR remove colour noise really well leaving just the 'grain' which is fine by me. As for noise removal software, I've never seen any that actually removes noise and retains detail. Usually it simply looks like the image is softer/not sharp. BTW- adding grain to an image is something you can do to increase apparent sharpness/detail at times

Quote
Are you suggesting it was an intentional creative choice of Henri to have it be noisy, or could the lack of available alternative materials be a part of it? If I'm not mistaken, Henri is also known to have sucked at printing his photo's which is why he let someone else do it for him. He knew his limitations, and didn't (ab)use them as a creative label. He knew he wanted to get the timing of events right. The image is sharp, but there was motion blur which was effective (recognition of the subject would also not have helped), although I do not know how intentional it was recorded by choosing a different shutterspeed.
No I'm saying that noise doesn't necessarily matter. What does matter is the content of the photo.

Quote
There's a difference between a successful image despite of technical shortcomings, and one because of the shortcomings ...
As there is between a technically competent image and one that is actually interesting.  :P
Title: Re: The Optimum Digital Exposure
Post by: Fine_Art on October 31, 2014, 01:12:09 pm
The moment has been captured so well that lots of people like it despite the fact it's grainy, noisy and lacking sharpness.  ;)

Nailed it.
Title: Re: The Optimum Digital Exposure
Post by: NancyP on October 31, 2014, 03:19:53 pm
Tiny point concerning the following exchange:
"There are times, however,  e.g. photographing wild life or 'grabbing the moment' when ETTR is difficult to employ."
This is the beauty the OneZone method #2. You set your exposure for EV +1-1/3 then bracket +/-2/3 of the stop!. If you put these settings in one of your camera presets, when the time comes for "grabbing the moment" you just simply hold your finger on the shutter button (capturing 3 exposures) then go on to the next scene.

I shoot action sequences of wildlife in which I have a second or two of capture time, and I put more priority on catching the most interesting posture, wing position, etc than on perfect ETTR. Basically, I put the camera on continuous high speed drive, hit the shutter when the action commences (bird in flight comes near, heron dives at fish), and capture 2 seconds of action. I wouldn't bracket.
Title: Re: The Optimum Digital Exposure
Post by: telyt on October 31, 2014, 03:27:10 pm
Tiny point concerning the following exchange:
I shoot action sequences of wildlife in which I have a second or two of capture time, and I put more priority on catching the most interesting posture, wing position, etc than on perfect ETTR. Basically, I put the camera on continuous high speed drive, hit the shutter when the action commences (bird in flight comes near, heron dives at fish), and capture 2 seconds of action. I wouldn't bracket.

Agreed.  Bracketing increases the chances that the 'right' posture or activity will coincide with the wrong exposure.
Title: Re: The Optimum Digital Exposure
Post by: jjj on October 31, 2014, 03:32:07 pm
The moment has been captured so well that lots of people like it despite the fact it's grainy, noisy and lacking sharpness.  ;)
My point entirely.
Title: Re: The Optimum Digital Exposure
Post by: jpegman on October 31, 2014, 03:36:39 pm
Agreed.  Bracketing increases the chances that the 'right' posture or activity will coincide with the wrong exposure.

I agree 100% - No one said this is for 100% of your shooting, so if you worry about critical timing then this process is not for you. Fall back on whatever works for your shooting style-requirements.
However, if you want the best quality you can extract from your sensor and you have the time for at least 3 bracketed shots (<1sec on most DSLRS!), then this is one process to apply!
Title: Re: The Optimum Digital Exposure
Post by: jjj on October 31, 2014, 03:44:48 pm
I shoot action sequences of wildlife in which I have a second or two of capture time, and I put more priority on catching the most interesting posture, wing position, etc than on perfect ETTR. Basically, I put the camera on continuous high speed drive, hit the shutter when the action commences (bird in flight comes near, heron dives at fish), and capture 2 seconds of action. I wouldn't bracket.
I'd use manual exposure for that sort of thing and set it to ETTR - if that was my preferred method.
Title: Re: The Optimum Digital Exposure
Post by: Bart_van_der_Wolf on October 31, 2014, 03:53:59 pm
Quote
There's a difference between a successful image despite of technical shortcomings, and one because of the shortcomings ...

As there is between a technically competent image and one that is actually interesting.  :P

I prefer an image that is both. Are you suggesting they are exclusive qualities?

Cheers,
Bart
Title: Re: The Optimum Digital Exposure
Post by: jjj on October 31, 2014, 04:01:03 pm
"As there is between a technically competent image and one that is actually interesting.  :P"

I prefer an image that is both. Are you suggesting they are exclusive qualities?
Not at all.
But I will say that those who obsess over tech stuff do tend to produce competent, but uninteresting photos
Title: Re: The Optimum Digital Exposure
Post by: Telecaster on October 31, 2014, 04:04:43 pm
I actually like grain or noise in images, to me it's in fact more real. Why? Because the world is full of texture, unlike than plastic smooth, digital images that have a video quality.
However banding in shadows or colour noise is something I cannot stand. LR/ACR remove colour noise really well leaving just the 'grain' which is fine by me. As for noise removal software, I've never seen any that actually removes noise and retains detail. Usually it simply looks like the image is softer/not sharp. BTW- adding grain to an image is something you can do to increase apparent sharpness/detail at times

Yep, I agree with this. Chroma & patterned luma noise I don't like. Grain-like luma noise not only doesn't bother me, I often accentuate it for effect. Given that photography is for me a creative enterprise, whatever helps me get a look I like or find compelling or even provocative is the optimum thing to do. Attempts to impose any particular æsthetic or approach as the "right" one should be disregarded.

-Dave-
Title: Re: The Optimum Digital Exposure
Post by: Bart_van_der_Wolf on October 31, 2014, 04:13:31 pm
Not at all.
But I will say that those who obsess over tech stuff do tend to produce competent, but uninteresting photos

LOL, that's like suggesting that people like Rembrandt van Rijn, or Johannes Vermeer, or even Leonardo DaVinci, to just name a few commonly known ones didn't care (or even obsess) about the process as much as the composition and the politics of those times.

Cheers,
Bart
Title: Re: The Optimum Digital Exposure
Post by: jjj on October 31, 2014, 08:05:44 pm
LOL, that's like suggesting that people like Rembrandt van Rijn, or Johannes Vermeer, or even Leonardo DaVinci, to just name a few commonly known ones didn't care (or even obsess) about the process as much as the composition and the politics of those times.
No it isn't. People with talent who learn the technicalities of painting are very different from those who obsess of the technicalities of photography. The major difference between photography and painting is that no matter how much you know about technicalities, if you have no innate talent, you very obviously still cannot paint. With photography, if you know the technicalities [which are not exactly tricky] you can produce photos that are correctly exposed, in focus etc, even if you have no eye for composition or any creative ability at all.

Title: Re: The Optimum Digital Exposure
Post by: jjj on October 31, 2014, 08:10:09 pm
Given that photography is for me a creative enterprise, whatever helps me get a look I like or find compelling or even provocative is the optimum thing to do. Attempts to impose any particular æsthetic or approach as the "right" one should be disregarded.
Yup.
Title: Re: The Optimum Digital Exposure
Post by: Rand47 on October 31, 2014, 08:19:39 pm
Nailed it.

I respectfully disagree.  If this image were grainless and tack sharp it would be reduced to sterile photo reportage and the sense of tension and mystery would evaporate to a large extent.  The murky quality actually adds to the image, I think.

Rand
Title: Re: The Optimum Digital Exposure
Post by: Iliah on October 31, 2014, 10:26:23 pm
> The major difference between photography and painting is that no matter how much you know about technicalities, if you have no innate talent, you very obviously still cannot paint

You may want to dwell a little into the history of Vatican to see how it is not so. Does the name Piero Dei ring a bell?
Title: Re: The Optimum Digital Exposure
Post by: Ray on October 31, 2014, 10:43:47 pm
I respectfully disagree.  If this image were grainless and tack sharp it would be reduced to sterile photo reportage and the sense of tension and mystery would evaporate to a large extent.  The murky quality actually adds to the image, I think.

Rand

As I mentioned before, there's no accounting for taste.  ;D

However, a relevant point I would make is that a technically sharp and noise-free image can be degraded as much as you like during post processing, to accord with your taste. The reverse is much more difficult.

Whilst a noisy and unsharp image can certainly be improved, for the benefit of those who do not find noise and blurriness a big attraction, most of us would agree that it's difficult to create detail that was never captured in the first instance.

A few years ago I visited an Henri Cartier Bresson exhibition in Australia. I certainly found the photos interesting, but I could not restrain myself, as I walked around the exhibition hall, from making a mental comparison with what I imagined each photo could have been like if it had been taken with a modern DSLR, and processed by me.  ;)  How much better each photo would then have been.  ;)

When I came to this shot of a man jumping over a puddle, what struck me was not any appeal due to murkiness, but the symmetry and the relationships between various elements in the composition.

Previously, I'd seen only rather small images of this shot on my computer screen, no bigger than the current image from Jeremy. On the fairly large print presented in the exhibition (maybe 20" x 25" - can't remember precisely), the background advertisement of the Railowsky Circus, depicting another person in the act of 'jumping',  was much clearer, and therefore the connection or relationship, in the composition, between the picture of the jumping man in the advertisement and the real man jumping the puddle, both with their own reflections in the water, was stronger and more obvious, and as a consequence that increased the appeal of the photo.

This image is truly a great example of 'capturing the moment'. Grain, noise and general lack of clarity do not add to its appeal one whit, in my view.
Title: Re: The Optimum Digital Exposure
Post by: Tony Jay on October 31, 2014, 10:55:37 pm
As a corollary to Ray's point, having artistic latitude inherent in a camera's ability to produce an image is helpful.
Not having certain characteristics in an image means having to introduce it later in post-processing.
Many of these edits are best done from a position of strength as it were since adding grain to a clean image (with respect to noise) is better than trying to clean up a noisy image when clean shadows and fine detail are required or desired.

Tony Jay
Title: Re: The Optimum Digital Exposure
Post by: Fine_Art on October 31, 2014, 11:02:11 pm
I respectfully disagree.  If this image were grainless and tack sharp it would be reduced to sterile photo reportage and the sense of tension and mystery would evaporate to a large extent.  The murky quality actually adds to the image, I think.

Rand

You are romanticizing, filling in the low detail of the shot with your imagined wonders. We all do it when we watch old B/W movies, etc. It becomes interesting because it lacks so much quality, the ancient times become magical. If you objectively look at a lot  of old stuff it is really very primitive. Compare the first running horse movie, which was a technical marvel at the time, to a modern blockbuster. I think it was Arthur C Clark that said "Any technology sufficiently advanced from the norm, becomes indistinguishable from magic." [Probably not his exact words] Well, when we look back, romanticizing about a lost time when things were simpler and nicer, it is all in our heads. Despite the manners of the Victorian age, life was quite rough. Despite the chivalry of the middle ages, their wars were absolute bloodbaths of masses hacking apart masses.

The HCB image is great given the limitations of the time. If someone recreated the shot in 100MP detail, and you think the magic has disappeared, it is because the magic was in your imagination. Hitchcock was the master of playing on that.

edit: here is the first running horse video
https://www.youtube.com/watch?v=UrRUDS1xbNs

Is it more wonderful due to the low image quality? Probably not.
Title: Re: The Optimum Digital Exposure
Post by: Ray on November 01, 2014, 12:16:41 am
You are romanticizing, filling in the low detail of the shot with your imagined wonders. We all do it when we watch old B/W movies, etc. It becomes interesting because it lacks so much quality, the ancient times become magical. If you objectively look at a lot  of old stuff it is really very primitive. Compare the first running horse movie, which was a technical marvel at the time, to a modern blockbuster. I think it was Arthur C Clark that said "Any technology sufficiently advanced from the norm, becomes indistinguishable from magic." [Probably not his exact words] Well, when we look back, romanticizing about a lost time when things were simpler and nicer, it is all in our heads. Despite the manners of the Victorian age, life was quite rough. Despite the chivalry of the middle ages, their wars were absolute bloodbaths of masses hacking apart masses.

The HCB image is great given the limitations of the time. If someone recreated the shot in 100MP detail, and you think the magic has disappeared, it is because the magic was in your imagination. Hitchcock was the master of playing on that.

edit: here is the first running horse video
https://www.youtube.com/watch?v=UrRUDS1xbNs

Is it more wonderful due to the low image quality? Probably not.


Ah! I see! There is accounting for taste.  ;D  Thanks for that, Fine_Art.
Title: Re: The Optimum Digital Exposure
Post by: Eyeball on November 01, 2014, 10:41:07 am
I'm a little late to the party here but I have read through most of the thread and I still have some questions regarding the article:

- What makes this article different from the normal ETTR article, including Michael's?  I don't really see any new ground being covered here other than trying to wrap it up in a trademarked phrase and sell a book.

- The only part that was a little new for me was the assertion that there is even more headroom available than we have been led to believe.  But that assertion appears to be based on George Jardine's experiments back in 2012 and after looking over THAT article, it seems questionable as to how accurately it was done and just what conclusions can be drawn from it.  Even George starts to add disclaimers in the comments on that page.  Since apparently nobody from Adobe was consulted, I am extremely skeptical regarding the "97% in LR still leaves one full stop" assertion, given all the behind-the-scenes "magic" that happens in LR regarding highlight recovery.

- The sample pics in the article seem over-the-top to me - the black cat on the coal pile, in particular.  I guess maybe exaggeration to make a point?

- The other thing that occurred to me while reading it was how ETTR is now starting to show a little wear where the new sensors are concerned (Guillermo is talking about this and showing examples in the thread) so it gives me the impression that the author is trying to make a lot of noise about something that has already been extensively covered AND is beginning to be less important than it was in the past.  Seems like poor timing.

What am I missing?  It just came off to me as really light-weight and "gimmicky" compared to what I am used to seeing on LuLa.
Title: Re: The Optimum Digital Exposure
Post by: deejjjaaaa on November 01, 2014, 11:25:36 am
What am I missing?
the big picture... the front page of LuLa has to be filled with something when there is nothing of real value, it might need to be filled with such articles
Title: Re: The Optimum Digital Exposure
Post by: bjanes on November 01, 2014, 11:30:40 am
- The only part that was a little new for me was the assertion that there is even more headroom available than we have been led to believe.  But that assertion appears to be based on George Jardine's experiments back in 2012 and after looking over THAT article, it seems questionable as to how accurately it was done and just what conclusions can be drawn from it.  Even George starts to add disclaimers in the comments on that page.  Since apparently nobody from Adobe was consulted, I am extremely skeptical regarding the "97% in LR still leaves one full stop" assertion, given all the behind-the-scenes "magic" that happens in LR regarding highlight recovery.

A major flaw of the article is that the author uses LR percentages to judge the status of the raw data. This is hazardous since LR and ACR in PV2012 use automatic highlight recovery and blown highlights may not be recognized. Furthermore, LR and ACR use baseline exposure offsets that affect the histograms and pixel value readouts. RawDigger is the proper tool with which to evaluate the raw file.

- The other thing that occurred to me while reading it was how ETTR is now starting to show a little wear where the new sensors are concerned (Guillermo is talking about this and showing examples in the thread) so it gives me the impression that the author is trying to make a lot of noise about something that has already been extensively covered AND is beginning to be less important than it was in the past.  Seems like poor timing.

Quite true. With current high performance sensors, one may get excellent results without optimum exposure and without the risk of blown highlights with resultant loss of data.

Bill
Title: Re: The Optimum Digital Exposure
Post by: Iliah on November 01, 2014, 01:21:40 pm
> With current high performance sensors, one may get excellent results without optimum exposure and without the risk of blown highlights with resultant loss of data

More, with some cameras at base ISO, where close to full well is used, standard exposure advices like keep right are resulting in linearity limit being exceeded by close to a stop - like it is with some Panasonic models and Sigmas. A study based on 1 or 2 camera models resulting in general advice is simply wrong.
Title: Re: The Optimum Digital Exposure
Post by: Bart_van_der_Wolf on November 01, 2014, 01:30:39 pm
A major flaw of the article is that the author uses LR percentages to judge the status of the raw data. This is hazardous since LR and ACR in PV2012 use automatic highlight recovery and blown highlights may not be recognized. Furthermore, LR and ACR use baseline exposure offsets that affect the histograms and pixel value readouts. RawDigger is the proper tool with which to evaluate the raw file.

Spot on, Bill. With Lightroom there is too much going on to allow (in particular quantitative and) qualitative assessments. It's also not designed for that purpose, so it's fine for what it does do.

Quote
With current high performance sensors, one may get excellent results without optimum exposure and without the risk of blown highlights with resultant loss of data.

Indeed, but still no license for sloppy technique. Better exposure (more photons) still gives better technical quality, but it is nice to have room to play, in case we cannot avoid to underexpose.

Cheers,
Bart
Title: Re: The Optimum Digital Exposure
Post by: deejjjaaaa on November 01, 2014, 01:31:01 pm
RawDigger is the proper tool with which to evaluate the raw file.
FRV (http://www.fastrawviewer.com/ ) from the same sources as RawDigger works nicely too (when you do not need a precision greater than say 1/6-1/12 EV ? or not doing a precise research), now that it has much improved, user friendly (= mouse friendly for me), UI I find myself starting FRV from my image browser almost always instead of RawDigger for the purpose of evaluating regular shots/bracketed shots/etc.
Title: Re: The Optimum Digital Exposure
Post by: Telecaster on November 01, 2014, 05:50:06 pm
…having artistic latitude inherent in a camera's ability to produce an image is helpful. Not having certain characteristics in an image means having to introduce it later in post-processing. Many of these edits are best done from a position of strength as it were since adding grain to a clean image (with respect to noise) is better than trying to clean up a noisy image when clean shadows and fine detail are required or desired.

This is a variant on the recording musician's dilemma: Track with effects (phasing, delay, reverb, etc.) inline with your instrument or add the effects later? I find with photography as well as music that the "right" way to do it depends. If I'm taking photos with the intent of including noise/grain and I'm using a camera that has a pleasing (as defined by me) noise profile, then I'll set the ISO to give me the amount of noise I want. No "fixing it in the mix." Different camera, different circumstances, different look desired…different choices. There is no right or wrong involved, just a combination of impulse & experience plus subjective taste. Do what works for you.

Also note that these days there's as much technique involved in achieving a deliberately grainy/noisy look as in a clean one. It's a deliberate process, not the result of ignorance or sloppiness.

-Dave-
Title: Re: The Optimum Digital Exposure
Post by: Bart_van_der_Wolf on November 01, 2014, 06:58:35 pm
Also note that these days there's as much technique involved in achieving a deliberately grainy/noisy look as in a clean one. It's a deliberate process, not the result of ignorance or sloppiness.

Totally agree Dave. Too bad there are apparently only relatively few who master that craft...

Cheers,
Bart
Title: Re: The Optimum Digital Exposure
Post by: Iliah on November 01, 2014, 07:56:06 pm
Totally agree Dave. Too bad there are apparently only relatively few who master that craft...

Cheers,
Bart
It was (and is) the same with the film. A lot of time needs to be spent with the film choosing the right emulsion, right exposure, right development, and right magnification of the shots of sandpaper to be overlaid to get the grain you want :)
Title: Re: Re: Re: The Optimum Digital Exposure
Post by: Guillermo Luijk on November 01, 2014, 08:30:15 pm
If I'm taking photos with the intent of including noise/grain and I'm using a camera that has a pleasing (as defined by me) noise profile, then I'll set the ISO to give me the amount of noise I want.

I doubt there is such camera. RAW noise has a gaussian/Poisson distribution on all 4 channels in any Bayer sensor, and how this noise translates to the final image depends mainly on the RAW development software and processing applied (noise reduction, etc...). Same software and processing, equal noise appearance for the same level of input noise.

The best way to have nice noise in the final image on a digital camera is to capture as noise free as possible, then add noise. Whether these manoeuvres make sense or not is subjective.

Regards
Title: Re: Re: Re: The Optimum Digital Exposure
Post by: Bart_van_der_Wolf on November 01, 2014, 08:42:27 pm
RAW noise has a gaussian/Poisson distribution on all 4 channels in any Bayer sensor, and how this noise translates to the final image depends mainly on the RAW development software and processing applied (noise reduction, etc...). Same software and processing, equal noise appearance for the same level of input noise.

The best way to have nice noise in the final image on a digital camera is to capture as noise free as possible, then add noise. Whether these manoeuvres make sense or not is subjective.

Hi Guillermo,

Couldn't agree more. The Raw conversion process will dictate the 'look' of the mostly Poisson/Gaussian noise, but demosaicing will determine what it's going to look like.

The best approach, if (!) one wants to use noise as a creative option, adding it to a relatively noise free image is the best approach to have creative control.

Cheers,
Bart
Title: Re: The Optimum Digital Exposure
Post by: Iliah on November 02, 2014, 10:19:30 am
Well, officially version 0.9.4 was published this morning only, that includes colour management, EXIF,... Changelog is on the download page, http://www.fastrawviewer.com/download
Title: Re: Re: Re: The Optimum Digital Exposure
Post by: Telecaster on November 02, 2014, 03:20:59 pm
I doubt there is such camera. RAW noise has a gaussian/Poisson distribution on all 4 channels in any Bayer sensor, and how this noise translates to the final image depends mainly on the RAW development software and processing applied (noise reduction, etc...). Same software and processing, equal noise appearance for the same level of input noise.

I should've noted that when I gain up the sensor with the intent of producing noise it's with JPEG use in mind. Thus the only RAW converter in play is the one on-board the camera.

-Dave-
Title: Re: Re: Re: The Optimum Digital Exposure
Post by: Ray on November 02, 2014, 06:33:55 pm
I should've noted that when I gain up the sensor with the intent of producing noise it's with JPEG use in mind. Thus the only RAW converter in play is the one on-board the camera.

-Dave-

I'm not so sure I'd want to hire you for a photographic project, Dave.  ;D

Consider the following example. You get a call from a client who wants a noisy, murky, blurry shot for a specific customer who has requested an image that produces a nostaligic effect harking back to the early days of photography, for the purpose of selling a specific product.

You go to some trouble and expense setting up the conditions and hiring suitable models. The action begins and you take numerous shots, deciding to use the camera in jpeg mode and set to ISO 6400, perhaps also using a neutral density filter to slow down the shutter speed.

From the numerous shots you've taken, one is just right. The expression on the model's face is the most appropriate and you've captured the action well. The customer is delighted with the shot.

Some time later you get another call from the same client who has great news for you and excitedly tells you that another very important and wealthy customer has seen your blurry and noisy shot and likes the general theme and composition so much that he would like to have a huge, tack-sharp, noise-free print of the image. He'll pay big  money.

Alas! You inform your client that you only took noisy jpegs on that occasion and that none of the images would be suitable to produce a large, normally sharp, modern-looking print with smooth tones.  >:(
Title: Re: The Optimum Digital Exposure
Post by: Wayne Fox on November 02, 2014, 11:55:36 pm
Just finished glancing through this thread, curious if there is a way to see the image that started all of this?  I must of have missed it.

whoops, sorry wrong thread.
Title: Re: The Optimum Digital Exposure
Post by: stamper on November 03, 2014, 03:30:22 am
It is an article from the LULA main site.
Title: Re: The Optimum Digital Exposure
Post by: Ray on November 03, 2014, 04:11:09 am
Just finished glancing through this thread, curious if there is a way to see the image that started all of this?  I must of have missed it.

Hi Wayne,
The image that started it is at Reply #102 where Jeremy posted the famous Henri Cartier Bresson shot of a man jumping a puddle, claiming it was grainy, noisy and blurry, yet lots of people liked it.

Also, in Reply #121 Rand expressed the opinion if the image were grain-free and tack sharp it would lose its tension and mystery and just become another sterile reportage photo. He likes the murky quality.
Title: Re: The Optimum Digital Exposure
Post by: stamper on November 03, 2014, 04:15:09 am
Wayne I think Ray is getting confused.

http://www.luminous-landscape.com/essays/the_optimum_digital_exposure.shtml
Title: Re: The Optimum Digital Exposure
Post by: jjj on November 03, 2014, 04:34:33 am
I doubt there is such camera. RAW noise has a gaussian/Poisson distribution on all 4 channels in any Bayer sensor, and how this noise translates to the final image depends mainly on the RAW development software and processing applied (noise reduction, etc...). Same software and processing, equal noise appearance for the same level of input noise.

The best way to have nice noise in the final image on a digital camera is to capture as noise free as possible, then add noise. Whether these manoeuvres make sense or not is subjective.
Added noise is definitely not the same as noise produced during capture. So if you prefer the noise produced by the camera, then that is the way to go.
Also the lack of detail caused by using a low quality/high ISO image is different from a tack sharp image with added noise. Doing nice lo-fi imagery is as tricky, possibly even harder than high quality photos.
Title: Re: Re: Re: The Optimum Digital Exposure
Post by: Guillermo Luijk on November 03, 2014, 04:39:48 am
Added noise is definitely not the same as noise produced during capture. So if you prefer the noise produced by the camera, then that is the way to go.
Also the lack of detail caused by using a low quality/high ISO image is different from a tack sharp image with added noise. Doing nice lo-fi imagery is as tricky, possibly even harder than high quality photos.

Of course is not the same, added noise can be MUCH better than camera noise because it can be added to your liking in amount, appearance and even location. That is why I said the best way to have beautiful noisy images starts from a clean RAW capture.
Title: Re: The Optimum Digital Exposure
Post by: jjj on November 03, 2014, 04:55:37 am
Of course is not the same, added noise can be MUCH better than camera noise because it can be added to your liking in amount, appearance and even location. That is why I said the best way to have beautiful noisy images starts from a clean RAW capture.
Not necessarily the best way for the reasons I pointed out in my previous reply. I don't necessarily want 'beautiful' noisy images, I may want crappy noisy images.  :P 
Post production cannot always replicate camera or film flaws as you may want them.
Title: Re:
Post by: Guillermo Luijk on November 03, 2014, 05:08:15 am
The point is camera noise is not pleasant, but some statistical knowkedge is needed to understand why.

Film is a totally different story and can be very beautiful because of its analogue nature, don't mix it with digtal noise.
Title: Re: The Optimum Digital Exposure
Post by: jjj on November 03, 2014, 05:24:26 am
The point is camera noise is not pleasant, but some statistical knowkedge is needed to understand why.
Don't confuse your preferences with other people's likes/dislikes and knowledge of statistics is definitely not necessary to decide what you like.

Quote
Film is a totally different story and can be very beautiful because of its analogue nature, don't mix it with digtal noise.
I wasn't, but digital camera noise can also look good in my view.
Colour noise however I completely loathe, but as that is very easily removed [and part of my default settings] it never bothers me. At times this luminance noise can can be very film like and therefore pleasing to me, though it all depends on your particular sensor, ISO/exposure combination.
Title: Re: The Optimum Digital Exposure
Post by: Bart_van_der_Wolf on November 03, 2014, 05:47:40 am
I don't necessarily want 'beautiful' noisy images, I may want crappy noisy images.  :P

Practice makes perfect crap.

Quote
Post production cannot always replicate camera or film flaws as you may want them.

There are tools available that (besides improving images) can also be used to assist (simulate Lomography and light leaks, add grain, etc.) in the process of crapifying ...

Cheers,
Bart
Title: Re: Re: Re: The Optimum Digital Exposure
Post by: Guillermo Luijk on November 03, 2014, 05:49:52 am
digital camera noise can also look good in my view.

You are right, a matter of having good taste :P
Title: Re: The Optimum Digital Exposure
Post by: jjj on November 03, 2014, 07:35:50 am
You are right, a matter of having good taste :P
Well let me know if you get some then. ;)
Title: Re: Re: Re: The Optimum Digital Exposure
Post by: Guillermo Luijk on November 03, 2014, 08:52:43 am
Colour noise however I completely loathe, but as that is very easily removed [and part of my default settings] it never bothers me. At times this luminance noise can can be very film like and therefore pleasing to me, though it all depends on your particular sensor, ISO/exposure combination.

It does not depend on the sensor, all sensors will provide the same noise appearance as long the same software is used to develop and process the RAW files. Some statistical knowledge is needed to understand why though.

In fact it doesn't make sense to talk about colour noise when talking about sensors since there is no colour noise in a RAW file. What you call colour noise is the result of a particular RAW demosaicing algorithm (software) propagating the gaussian/Poisson noise in captured RAW data and giving it a shape in the RGB rendered image.
Title: Re: The Optimum Digital Exposure
Post by: williamchutton on November 03, 2014, 10:14:28 am
Added noise is definitely not the same as noise produced during capture. So if you prefer the noise produced by the camera, then that is the way to go.
Also the lack of detail caused by using a low quality/high ISO image is different from a tack sharp image with added noise. Doing nice lo-fi imagery is as tricky, possibly even harder than high quality photos.

Photon-noise statistics are well-predicted by Q.M. Adding noise in post-production to simulate increased photon noise would be straightforward. Several camera brands offer an option to record lossy, compressed, raw data.  The more clever compression algorithms act as noise filters that selectively affect the brightest regions in the image. While irreversably manipulating original data is always a bad idea, in this case most of the information loss (photon noise) can be added back to the image during post production.

Read noise is different matter. But modeling read noise is possible. People who are cursed with making parameter estimated from data with oppressive signal-to-nose ratios (astronomers for instance) are forced to include a model for the noise in their parameter-estimation calculations. It is common to include several terms to the properly model the noise. Including a model for the noise reduces the uncertainty in the signal parameter estimates of interest. Sometimes adding noise computed from candidate models to high SNR data is an efficient way to empirically refine the noise model.
Title: Re: The Optimum Digital Exposure
Post by: jjj on November 03, 2014, 11:04:41 am
It does not depend on the sensor, all sensors will provide the same noise appearance as long the same software is used to develop and process the RAW files. Some statistical knowledge is needed to understand why though.
Funny that as different cameras/sensors I've used give different noise characteristics. Some were quite crappy, others OK - with the same software.

Quote
In fact it doesn't make sense to talk about colour noise when talking about sensors since there is no colour noise in a RAW file. What you call colour noise is the result of a particular RAW demosaicing algorithm (software) propagating the gaussian/Poisson noise in captured RAW data and giving it a shape in the RGB rendered image.
I understand how raw files are made usable, but it still makes perfect sense to refer to colour noise as colour noise, because that is exactly how it looks. We don't look at the raw data, we look at images. after demosaicing. You may as well argue the entire raw file is B+W despite the fact it looks pretty colourful when viewed by humans or that say a camera isn't really a camera, it's actually metal, glass and plastic.

What you appear to be doing is arguing that the word 'colour' is not actually colour or the concept [English speaking] people understand by colour, but is six individual letters that have no meaning.
The phrase reductio ad absurdum was invented for such ways of arguing.

BTW I studied stats at uni and school and do not really care about what distributions are used in the data, I look at the images not the physics and maths that produced them.
Title: Re: The Optimum Digital Exposure
Post by: jjj on November 03, 2014, 11:12:44 am
Photon-noise statistics are well-predicted by Q.M. Adding noise in post-production to simulate increased photon noise would be straightforward. Several camera brands offer an option to record lossy, compressed, raw data.  The more clever compression algorithms act as noise filters that selectively affect the brightest regions in the image. While irreversably manipulating original data is always a bad idea, in this case most of the information loss (photon noise) can be added back to the image during post production.

Read noise is different matter. But modeling read noise is possible. People who are cursed with making parameter estimated from data with oppressive signal-to-nose ratios (astronomers for instance) are forced to include a model for the noise in their parameter-estimation calculations. It is common to include several terms to the properly model the noise. Including a model for the noise reduces the uncertainty in the signal parameter estimates of interest. Sometimes adding noise computed from candidate models to high SNR data is an efficient way to empirically refine the noise model.
There seems to be a lot of not seeing the wood for the trees going on here and with Guillermo.
Rather than faffing around in post trying to simulate an effect, why not just do it? Sometimes that is the better, simpler and easier solution.
Title: Re:
Post by: Guillermo Luijk on November 03, 2014, 12:01:47 pm
I never said it is not simpler or easier, of course it is. I simply said camera digital noise is ugly, and if you like it is only because of a lack of good taste.
Title: Re: The Optimum Digital Exposure
Post by: Telecaster on November 03, 2014, 04:33:39 pm
Consider the following example… (see above for full post)

It's all about context. As I'm pic-taking only for my own enjoyment your example has no relevance. If I were a pro, OTOH, I'd be doing everything I could to satisfy my clients…and that would include trying to anticipate their future wants & needs. But I'm not a pro and have zero interest in emulating that approach.

I'm not precious about my own photos. They remind me of experiences I've had & things I've seen but otherwise aren't that important in-and-of themselves. My favorites end up on a wall for awhile, then get replaced by others, which will in turn be replaced, etc. I get far more pleasure out of continually experimenting than from sticking with any particular photographing style, processing approach…or subject matter.

-Dave-
Title: Re:
Post by: Telecaster on November 03, 2014, 04:58:14 pm
I simply said camera digital noise is ugly, and if you like it is only because of a lack of good taste.

Now there's an example of thoughtful humility. ::)

-Dave-
Title: Re: The Optimum Digital Exposure
Post by: jjj on November 03, 2014, 06:03:02 pm
I never said it is not simpler or easier, of course it is. I simply said camera digital noise is ugly, and if you like it is only because of a lack of good taste.
That's a pretentious and sneery thing thing to say to be honest. You are claiming your taste is superior because I like something you do not - that's a sad place to be.
Title: Re: The Optimum Digital Exposure
Post by: jjj on November 03, 2014, 06:05:32 pm
It's all about context. As I'm pic-taking only for my own enjoyment your example has no relevance. If I were a pro, OTOH, I'd be doing everything I could to satisfy my clients…and that would include trying to anticipate their future wants & needs. But I'm not a pro and have zero interest in emulating that approach.

I'm not precious about my own photos. They remind me of experiences I've had & things I've seen but otherwise aren't that important in-and-of themselves. My favorites end up on a wall for awhile, then get replaced by others, which will in turn be replaced, etc. I get far more pleasure out of continually experimenting than from sticking with any particular photographing style, processing approach…or subject matter.
Seems like a very sensible approach to photography.
Title: Re: The Optimum Digital Exposure
Post by: Ray on November 03, 2014, 10:03:13 pm
It's all about context. As I'm pic-taking only for my own enjoyment your example has no relevance. If I were a pro, OTOH, I'd be doing everything I could to satisfy my clients…and that would include trying to anticipate their future wants & needs. But I'm not a pro and have zero interest in emulating that approach.

I'm not precious about my own photos. They remind me of experiences I've had & things I've seen but otherwise aren't that important in-and-of themselves. My favorites end up on a wall for awhile, then get replaced by others, which will in turn be replaced, etc. I get far more pleasure out of continually experimenting than from sticking with any particular photographing style, processing approach…or subject matter.

-Dave-

Well, you'll have to forgive me for not realising this, Dave. You seems to be in a most unusual situation, from my perspective.

Whilst I also consider myself to be an amateur whose main motivation for engaging in photography is the creative pleasure I get from the taking and processing of the shots, as well as the pleasure I get from a sense of reliving past experiences when I view old shots that I've taken, sometimes many years later, I would not go so far as to claim that such enjoyment excludes other people's opinions and appreciation of my work, as your term 'only for my own enjoyment' implies.

Occasionally, I've sold prints in the past for a tidy sum, and occasionally I'll make a large print for a friend or neighbour, charging just the cost of the paper and ink. Sometimes I'll produce smaller prints for friends or relatives without charge, and quite often I'll entertain guests with a slide show of selected images on a large HDTV screen.

As an amateur, I consider myself to be my own client,and just as all clients can differ in their tastes and requirements, I, as my own client, accept that my tastes and requirements may gradually change over time. I frequently revisit old shots with new ideas for processing.

One of the great attractions of the DSLR for me, with its RAW image capability, is the concept that having a RAW file is like having an undeveloped film that can be developed again and again in as many different ways as one likes, without affecting the original undeveloped film. Imagine if Kodak had developed such a process before the digital age.   ;)

Experimenting with a noisy jpeg deliberately taken at an unecessarily high ISO, or severely underexposed, in order to compare how much crappier, or how different such an image appears, compared with a clean RAW file of the same scene deliberately processed to appear equally crappy, is fine as an experiment.

However, to exclude also taking a RAW image, or at least a correctly exposed ETTR of the same scene, seems very shortsighted to me, unless your motivation is to take an otherwise meaningless shot of no consequence, apart from its abstract nature of general crappiness.

For example, I would understand perfectly if someone wanted to exploit the banding problem of a Canon 5D by underexposing 10 stops at base ISO in order to produce a tapestry effect of coarsely woven threads.

The subject might be just a plain wall, in which case one might skip taking an ETTR shot, realising that without the extreme banding, the correctly exposed image would be just plain boring and that there would be almost no possibility of one ever wanting to produce a tack-sharp, noise-free image of such a boring subject, for any imaginable reason. Is that your concept here, Dave?  ;D

Title: Re: The Optimum Digital Exposure
Post by: Telecaster on November 03, 2014, 10:43:20 pm
Whilst I also consider myself to be an amateur whose main motivation for engaging in photography is the creative pleasure I get from the taking and processing of the shots, as well as the pleasure I get from a sense of reliving past experiences when I view old shots that I've taken, sometimes many years later, I would not go so far as to claim that such enjoyment excludes other people's opinions and appreciation of my work, as your term 'only for my own enjoyment' implies.

You misread. I don't exclude anyone from enjoying photos I've taken. There are prints of mine at friends' homes and JPEGs on their gizmos. It pleases me when someone likes a photo I've taken. But I don't do the actual photography & editing with the intent of satisfying anyone other than myself. That's how I go about it. Other people have all sorts of different motivations and processes. These aren't challenges to anyone, just differences. Can you not accept this? Judging by your tone here the answer appears to be no.

And, as this is going nowhere fruitful or productive, that's all I have to say on the matter.

-Dave-
Title: Re: The Optimum Digital Exposure
Post by: Ray on November 04, 2014, 03:57:21 am
Can you not accept this? Judging by your tone here the answer appears to be no.

I never accept what I consider to be poor advice. However, I have an open mind, which is why I'm prepared to consider and discuss alternative views.
Title: Re: The Optimum Digital Exposure
Post by: RogerW on November 04, 2014, 06:54:36 am
To misquote the great Winston Churchill:

never in the field of digital photography has so much been posted to very little effect. 

If you don't want to use ETR, or Bob's OneZone, don't bother! 

As for me, I don't see any conflict between wanting to make creative, interesting photos and wanting to get the absolute most out of the equipment I've scrimped and saved to buy!

The ENT specialist I saw recently spent years learning the science and the result is that my ears are 100% better!
Title: Re: The Optimum Digital Exposure
Post by: jjj on November 04, 2014, 07:11:03 am
To misquote the great Winston Churchill:

never in the field of digital photography has so much been posted to very little effect.
You obviously missed the thread with Ray's curious thoughts on perspective and lens then.  ;D
Title: Re: The Optimum Digital Exposure
Post by: dwswager on November 04, 2014, 02:38:47 pm
John. You can't "overexpose" a sensor. Once the wells are full the photons fall on the floor. It's a brick wall.


If a sensor can't be overexposed, we could have cameras with infinite dynamic range and usable ISOs approaching zero!  Allah be praised!

In reality, sensors have both a lower threshold and upper saturation limit on how much photon energy can be converted to signal.  Below the lower threshold, signal is indistinguishable from the spurious noise.  Above the saturation limit, the signal generated basically tops out irregardless how much light hits it.

The fundamental issue is the way darker tones are expressed in a digital file as encoded in binary format.  Assuming an 8 bit channel, there are 8 different bits of data that can have values 1 or 0.  In a linear system then notice how little data is expressing the bottom tones and how little variation there is in the amount of data between them.  Add rounding errors from pushing and pulling the data around and pretty soon you have no data at all!

Linear Data
White  =  (2^8) 256,    100%
-1 stop = (2^7) 128,    50%
-2 stops = (2^6) 64,    25%
-3 stops = (2^5) 32,    12.5%
-4 stops = (2^4) 16,    6%
-5 stops = (2^3) 8,    3.1%
-6 stops = (2^2) 4,    1.56%
-7 stops = (2^1) 2,    0.78%
-8 stops = (2^0) 1,    0.39%
 
Title: Re: The Optimum Digital Exposure
Post by: Wayne Fox on November 04, 2014, 04:04:26 pm
I'm a little late to the party here but I have read through most of the thread and I still have some questions regarding the article:

- What makes this article different from the normal ETTR article, including Michael's?  


Overall this article is a good summary, but I am a bit unclear what is new relative to the original ETTR concept?



After wading through the article again, and through this thread again, I must say these two comments sum it up for me as well ... I just don’t see any new ground from the article in regards to ETTR.  Even the concept of bracketing to insure a single correct ETTR exposure has been practiced by many for years.
Title: Re: The Optimum Digital Exposure
Post by: bjanes on November 04, 2014, 04:26:58 pm
If a sensor can't be overexposed, we could have cameras with infinite dynamic range and usable ISOs approaching zero!  Allah be praised!

In reality, sensors have both a lower threshold and upper saturation limit on how much photon energy can be converted to signal.  Below the lower threshold, signal is indistinguishable from the spurious noise.  Above the saturation limit, the signal generated basically tops out irregardless how much light hits it.

The fundamental issue is the way darker tones are expressed in a digital file as encoded in binary format.  Assuming an 8 bit channel, there are 8 different bits of data that can have values 1 or 0.  In a linear system then notice how little data is expressing the bottom tones and how little variation there is in the amount of data between them.  Add rounding errors from pushing and pulling the data around and pretty soon you have no data at all!

Linear Data
White  =  (2^8) 256,    100%
-1 stop = (2^7) 128,    50%
-2 stops = (2^6) 64,    25%
-3 stops = (2^5) 32,    12.5%
-4 stops = (2^4) 16,    6%
-5 stops = (2^3) 8,    3.1%
-6 stops = (2^2) 4,    1.56%
-7 stops = (2^1) 2,    0.78%
-8 stops = (2^0) 1,    0.39%
 

That is a useful calculation for the encoding dynamic range of a linear space, where the DR in stops is equal to the number of bits used for the encoding. However, for a useful photographic DR, two factors are involved. The first is how much noise is acceptable in the darkest f/stop. A SNR of 10:1 is often used, but this is a judgement call. The second factor is how many levels are needed in the darkest f/stop to avoid posterization. Norman Koren has a useful table here (http://www.normankoren.com/digital_tonality.html). Look at the paragraph human vision and tonal levels. He requires 8 levels in the darkest f/stop. His table doesn't include values for a bit depth of 8. I recalculated the table for 8 bits and the 8 level cutoff is highlighted. With this criterion, the possible DR for 8, 10, 12, and 14 bits is 5, 7, 8, and 11 stops respectively.

For the DR of individual cameras, sensor data such as found on DXO are needed.

A bit of noise can dither the levels and reduce visible posterization as Emil discusses here (http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p3.html#bitdepth).

Bill

Title: Re: The Optimum Digital Exposure
Post by: jmlphotography on November 04, 2014, 04:58:05 pm
I'm confused.  How did I end up on DPReview?  (My reference is to the personal tone of some of the replies that I thought would not be acceptable here.)
Title: Re:
Post by: Herb on November 04, 2014, 05:33:42 pm
... camera digital noise is ugly, and if you like it is only because of a lack of good taste.
Maybe digital noise 'was' always ugly, but that could soon be history. Have you see the noise from the Olympus OMD E-M5?

http://robinwong.blogspot.co.uk/2014/10/shoot-model-portrait.html

Most times I'd pass on ETTR to get that noise. Bad taste? Everyone can decide for themselves. But many users are happy about the grain-like effect and I expect all manufacturers are working to copy it.

A lot of things are being tweaked in software that didn't happen when Michael first wrote the article. Some smart guys think ETTR conflicts with such software tweaks:

http://chromasoft.blogspot.co.uk/2009/09/why-expose-to-right-is-just-plain-wrong.html

http://theonlinephotographer.typepad.com/the_online_photographer/2011/10/expose-to-the-right-is-a-bunch-of-bull.html
Title: Re:
Post by: Jim Kasson on November 04, 2014, 06:01:42 pm

A lot of things are being tweaked in software that didn't happen when Michael first wrote the article. Some smart guys think ETTR conflicts with such software tweaks:

http://chromasoft.blogspot.co.uk/2009/09/why-expose-to-right-is-just-plain-wrong.html

This guy says this in the article linked to above: "But there is one situation where ETTR can help - when you're already at the lowest ISO setting you camera offers."  I don't think he really means that; I think he means at base ISO.

So, the way I think about it, he's saying that "real" ETTR works, but that you can't just crank up the ISO. Not a surprise.

Just for the record, I view "real" ETTR as minimizing photon noise. Moving the histogram to the right with the ISO control can mitigate things if you just can't get enough light on the sensor, but what you're doing there is managing read noise, because you've already decided what your exposure has to be.

If you don't think that ETTR is useful in modern cameras, and you can just throw away any improvements in photon noise because they're just so doggone good, you could save yourself some money and weight by using ETTR on a smaller sensor:

http://blog.kasson.com/?p=6871

Jim
Title: Re: The Optimum Digital Exposure
Post by: AlfSollund on November 05, 2014, 09:04:42 am
Thanks a lot for sharing this. Most useful aproximation.

I assume by having a much larger, heavier camera (larger sendor) I can use ETTR and keep the advantage versus a smaller sensor  ;)? Example 2 steps advantage from Nikon D810 to m43. Still ignoring many things, among them lens quality, sensor technology, read noise lar, PRNU, diffraction, depth of field…
Title: Re:
Post by: williamchutton on November 05, 2014, 09:34:53 am
This guy says this in the article linked to above: "But there is one situation where ETTR can help - when you're already at the lowest ISO setting you camera offers."  I don't think he really means that; I think he means at base ISO.

So, the way I think about it, he's saying that "real" ETTR works, but that you can't just crank up the ISO. Not a surprise.

Just for the record, I view "real" ETTR as minimizing photon noise. Moving the histogram to the right with the ISO control can mitigate things if you just can't get enough light on the sensor, but what you're doing there is managing read noise, because you've already decided what your exposure has to be.

If you don't think that ETTR is useful in modern cameras, and you can just throw away any improvements in photon noise because they're just so doggone good, you could save yourself some money and weight by using ETTR on a smaller sensor:

http://blog.kasson.com/?p=6871

Jim

How is it possible to alter photon noise?
How do you make improvements in photon noise?
Isn't the photon noise level a constant ratio in terms of the integer raw data?

Perhaps this Stanford Computer Optics article is incorrect or misleading?

http://www.stanfordcomputeroptics.com/technology/dynamic-range/photon-noise.html
Title: Re: The Optimum Digital Exposure
Post by: dwswager on November 05, 2014, 09:41:20 am
That is a useful calculation for the encoding dynamic range of a linear space, where the DR in stops is equal to the number of bits used for the encoding. However, for a useful photographic DR, two factors are involved. The first is how much noise is acceptable in the darkest f/stop. A SNR of 10:1 is often used, but this is a judgement call. The second factor is how many levels are needed in the darkest f/stop to avoid posterization. Norman Koren has a useful table here (http://www.normankoren.com/digital_tonality.html). Look at the paragraph human vision and tonal levels. He requires 8 levels in the darkest f/stop. His table doesn't include values for a bit depth of 8. I recalculated the table for 8 bits and the 8 level cutoff is highlighted. With this criterion, the possible DR for 8, 10, 12, and 14 bits is 5, 7, 8, and 11 stops respectively.

For the DR of individual cameras, sensor data such as found on DXO are needed.

A bit of noise can dither the levels and reduce visible posterization as Emil discusses here (http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p3.html#bitdepth).

Bill

Concur, but in all cases the darker tones are expressed with less data which is the point of ETTR: to move the darker tones up the scale which also helps with the S/N issue.  And, of course, because the sensor does not match our eye response, a gamma is applied.  

My point is there is no "optimal" exposure, at least in all cases.  A scene can contain DR less than, equal to or greater than that of our medium of capture.  If it is less than the DR of our medium, then yes, moving toward more exposure up to the saturation point of the sensor can result in better overall data capture.  But what of the other interesting cases.  What is optimal?

Can an a camera properly predict the saturation point of it's sensor based on the exposure meter data and other settings?  I don't know.  But that seems to be what we are asking for.  An exposure meter setting that set exposure to that 99.9% saturation point for the brightest pixel.  We will still have situations where the DR would be too large to capture.  Then a choice is required.
Title: Re: The Optimum Digital Exposure
Post by: bjanes on November 05, 2014, 10:40:22 am
Concur, but in all cases the darker tones are expressed with less data which is the point of ETTR: to move the darker tones up the scale which also helps with the S/N issue.  And, of course, because the sensor does not match our eye response, a gamma is applied.  

My point is there is no "optimal" exposure, at least in all cases.  A scene can contain DR less than, equal to or greater than that of our medium of capture.  If it is less than the DR of our medium, then yes, moving toward more exposure up to the saturation point of the sensor can result in better overall data capture.  But what of the other interesting cases.  What is optimal?

Can an a camera properly predict the saturation point of it's sensor based on the exposure meter data and other settings?  I don't know.  But that seems to be what we are asking for.  An exposure meter setting that set exposure to that 99.9% saturation point for the brightest pixel.  We will still have situations where the DR would be too large to capture.  Then a choice is required.


Automatic ETTR exposure would be nice, but I favor giving the photographer the tools to take charge and set proper exposure manually. A raw histogram is often asked for, but is not forthcoming. Also, the widest color space available with most cameras is AdobeRGB, whose gamut is too narrow. To illustrate, here is an exposure of a red flower made according the the camera meter reading. The red channel appears severely blown on the camera histogram.

(http://bjanes.smugmug.com/Photography/Red-Flower-WB/i-gr7L5WN/1/M/Prev-1-M.png)

However, the raw histogram shows the red channel to lie nearly 2 EV below clipping.

(http://bjanes.smugmug.com/Photography/Red-Flower-WB/i-6ZNkkkX/0/L/Img-1-RD-L.png)

Two factors are involved in the misleading camera histogram. The first is white balance. For daylight, the red multiplier is close to 2x. The raw channel can be intact before white balance but blown after WB. The other factor is that the color space of AdobeRGB can not contain the gamut of the flower. If the camera makers don't want to change their approach to give a raw histogram, they could at least offer ProPhotoRGB as a color space for the JPEG preview. To eliminate red channel clipping in the camera histogram required an exposure compensation of -1 2/3 EV.

Bill
Title: Re:
Post by: deejjjaaaa on November 05, 2014, 11:20:25 am
How is it possible to alter photon noise?
mr Kasson naturally means not absolute reduction in photon noise, but reduction in S/N ratio, which naturally involves increase in absolute numbers, but reduction in ratio.
Title: Re: The Optimum Digital Exposure
Post by: Jim Kasson on November 05, 2014, 11:20:59 am
Two factors are involved in the misleading camera histogram. The first is white balance. For daylight, the red multiplier is close to 2x. The raw channel can be intact before white balance but blown after WB.

Nice example, Bill. A custom WB can bring the in-camera histogram more into line (but not perfectly so) with the actual raw histogram, but then it becomes useless for judging color. You probably know that well, but it should be mentioned.

The other factor is that the color space of AdobeRGB can not contain the gamut of the flower. If the camera makers don't want to change their approach to give a raw histogram, they could at least offer ProPhotoRGB as a color space for the JPEG preview.

That's an interesting idea, but then we could have the opposite problem: the real raw histogram clipping while the PPRGB-derived histogram looks fine.

Jim
Title: Re:
Post by: Jim Kasson on November 05, 2014, 11:31:58 am
mr Kasson naturally means not absolute reduction in photon noise, but reduction in S/N ratio, which naturally involves increase in absolute numbers, but reduction in ratio.

Correct in every respect, except it's an increase in the SNR, which is I imagine what you meant to say..

Let me elaborate a bit. In a counterintuitive way, ETTR (real, ETTR, at base ISO) maximizes the photon noise, but maximizes the signal-to-noise ratio (SNR). How is that possible?

Photon noise varies as the square root of the exposure, exposure in this case being defined as number of photons per unit area impinging on the sensor. More light, more photon noise. Let's call the amount of photon noise sigma, and the light level mu.

SNR = mu / sigma.

Since sigma = sqrt(mu),

SNR = mu/sqrt(mu) = sqrt(mu)

So, every time we add a stop more light, the SNR goes up by 1.414; that's the square root of two.

That's the reason that ETTR (at base ISO) works.

Jim
Title: Re: The Optimum Digital Exposure
Post by: Eyeball on November 05, 2014, 11:33:33 am
If the camera makers don't want to change their approach to give a raw histogram, they could at least offer ProPhotoRGB as a color space for the JPEG preview. To eliminate red channel clipping in the camera histogram required an exposure compensation of -1 2/3 EV.

I agree with everything you said, Bill, and the ProPhoto alternatives does sound like a decent compromise.
I suspect though that the camera manufacturers don't provide raw or ProPhoto alternatives for usability concerns for the more general customer base.  They have to weigh the benefits to a few vs. the possible confusion/problems for the many.
ProPhoto is going to look even worse in a Jpeg that somebody assumes is in sRGB (consciously or unconsciously) and it may even begin to stress those measly 8-bits in a Jpeg, even for someone who understands color management.
Maybe they could bury the option about 3 levels down in the menu with a warning.  :)
Title: Re: The Optimum Digital Exposure
Post by: bjanes on November 05, 2014, 11:35:34 am
Nice example, Bill. A custom WB can bring the in-camera histogram more into line (but not perfectly so) with the actual raw histogram, but then it becomes useless for judging color. You probably know that well, but it should be mentioned.

That's an interesting idea, but then we could have the opposite problem: the real raw histogram clipping while the PPRGB-derived histogram looks fine.

Jim

Jim,

I do have UNIWB loaded into a custom bank on my camera, but I rarely use it. Too much trouble. In my example, the metered exposure is not optimum, but it does give good results at base ISO.

I don't quite understand your second statement. If the raw histogram is clipped, channels are blown and wouldn't the PPRGB histogram also be clipped? I don't doubt that you are correct, but my understanding is lacking.

Bill
Title: Re: The Optimum Digital Exposure
Post by: Eyeball on November 05, 2014, 11:37:04 am
That's an interesting idea, but then we could have the opposite problem: the real raw histogram clipping while the PPRGB-derived histogram looks fine.

But does that ever happen?  I can't think of any time I have seen an image unclipped in Lightroom, for example, but that showed up clipped in RawDigger (at least before moving the LR sliders around forcing LR to do more highlight recovery).
Title: Re: The Optimum Digital Exposure
Post by: bjanes on November 05, 2014, 11:40:25 am
But does that ever happen?  I can't think of any time I have seen an image unclipped in Lightroom, for example, but that showed up clipped in RawDigger (at least before moving the LR sliders around forcing LR to do more highlight recovery).

LR with PV 2012 does automatic highlight recovery and this may hide channel clipping. If you use PV2010 with a linear tone curve (point curve = linear, and sliders zeroed in the main panel), you get a better approximation of what RawDigger shows.

Bill
Title: Re: The Optimum Digital Exposure
Post by: Jim Kasson on November 05, 2014, 11:54:24 am
I don't quite understand your second statement. If the raw histogram is clipped, channels are blown and wouldn't the PPRGB histogram also be clipped? I don't doubt that you are correct, but my understanding is lacking.

Bill, I said that without a lot of thought. The basis for my thinking was that there are non-visible colors in the PPRGB gamut, including the blue primary (and, kinda, the green one, although that's a niggle). But camera native capture color spaces aren't linear transforms of XYZ, and what passes for a gamut in these spaces isn't what we think of as a gamut in a tristimulus color space traceable to 1931 XYZ.

So, you may be right. I'll think on it.

Jim
Title: Re: The Optimum Digital Exposure
Post by: Eyeball on November 05, 2014, 12:02:08 pm
LR with PV 2012 does automatic highlight recovery and this may hide channel clipping. If you use PV2010 with a linear tone curve (point curve = linear, and sliders zeroed in the main panel), you get a better approximation of what RawDigger shows.

Yep, I realize that and that's why I mentioned it as a possible caveat.  I normally use a linear tone curve but always use PC2012 now.

I guess it's theoretically possible for a camera to produce colors that are out of ProPhoto gamut but it seems like with today's cameras we would be talking very small fractions of a stop if it would happen at all, not the 1 or more stops of deviation that we can sometimes see with the current histograms.
Title: Re:
Post by: bjanes on November 05, 2014, 12:34:37 pm
How is it possible to alter photon noise?
How do you make improvements in photon noise?
Isn't the photon noise level a constant ratio in terms of the integer raw data?

Perhaps this Stanford Computer Optics article is incorrect or misleading?

http://www.stanfordcomputeroptics.com/technology/dynamic-range/photon-noise.html

The article could be misleading if not properly interpreted. It confirms a basic tenet of ETTR in that one should maximize the exposure. However, is discredits the idea that ETTR has to do with the number of levels in brighter f/stops. Those extra levels do a good job of quantizing noise but are not needed to record the information in the image. The presumed MFDB advantage in having a 16 bit file is a myth.  However, the article does not address read noise. With many cameras, increasing the ISO will reduce read noise and incidentally move the histogram to the right.

Bill
Title: Re: The Optimum Digital Exposure
Post by: Jim Kasson on November 05, 2014, 12:52:59 pm
I don't quite understand your second statement. If the raw histogram is clipped, channels are blown and wouldn't the PPRGB histogram also be clipped? I don't doubt that you are correct, but my understanding is lacking.

Bill, I've thought about it a little more, and I've come up with one significant problem. The white point of PPRGB is a linear transform of the CIE 1931 Standard Observer's response to the D50 illuminant. What if the white point of the native sensor is different from that? Specifically, most sensor's white points appear to be magenta-ish (the corrected JPEG looks green). If the sensor is exposed to a color with the chromaticity of its native white point (eliding capture metameric errors), as that color gets more illuminated, the PPRGB histogram will indicate clipping in its red and blue channels before the those channels of the sensor actually clip. Conversely, If the sensor is exposed to a color with the chromaticity of D50 (eliding capture metameric errors), as that color gets more illuminated, the PPRGB histogram will not indicate clipping when the green channel(s) of the sensor start to clip.

I've used a little shorthand in the above explanation, and will expand it to the point of tediousness if it's not clear.

I think that's right, anyway.  Does that make sense to you?

Jim
Title: Re:
Post by: deejjjaaaa on November 05, 2014, 02:26:26 pm
Correct in every respect, except it's an increase in the SNR, which is I imagine what you meant to say..
yes, typing too fast.
Title: Re: The Optimum Digital Exposure
Post by: deejjjaaaa on November 05, 2014, 02:40:39 pm
If the camera makers don't want to change their approach to give a raw histogram, they could at least offer ProPhotoRGB as a color space for the JPEG preview.
then the issue is not with a colorspace for OOC JPG, the issue is with the color transform... you can very well do a proper color transform into sRGB to reflect raw clipping properly using camera's blinkies/zebra and then you do an improper transform into ppRGB to show clipping when all raw channels are safe and sound

PS: you are not getting proper display on your LCD/EVF anyways with ppRGB or similar color spaces spaces for OOC JPG and that is naturally a concern for a camera manufacturers... and if you do not care about proper display then you might as well just tune your OOC JPG to use UniWB, proper contrast, etc... a lot of cameras are quite tunable to show real clipping in raw data... framing does not suffer and proper raw exposure still damage your OOC JPG in any color space, so having a green tint does not change a lot...  so for as long as you not getting true raw histogram (or blinkies/zebra based on raw data) it really does not matter if you have ppRGB... sRGB or AdobeRGB is enough
Title: Re: The Optimum Digital Exposure
Post by: Rand47 on November 05, 2014, 10:03:51 pm
Ray, FineArt...

Thanks for your feedback.  I find your perspectives on this fascinating.

Rand
Title: Re:
Post by: williamchutton on November 05, 2014, 11:50:00 pm
mr Kasson naturally means not absolute reduction in photon noise, but reduction in S/N ratio, which naturally involves increase in absolute numbers, but reduction in ratio.

Unless the article in the link I posted flawed, the ratio of bits representing photon fluctuations to the bits representing the total signal amplitude is constant for all exposures, i.e. for all signal-to-noise ratios. Reducing the S/N does not change the bit ratio.

Is the article flawed?

Is it true the ratio of bits representing photon fluctuations to the bits used for the total signal remain constant?
Title: Re:
Post by: Hans Kruse on November 06, 2014, 05:47:51 am
Maybe digital noise 'was' always ugly, but that could soon be history. Have you see the noise from the Olympus OMD E-M5?

http://robinwong.blogspot.co.uk/2014/10/shoot-model-portrait.html

Most times I'd pass on ETTR to get that noise. Bad taste? Everyone can decide for themselves. But many users are happy about the grain-like effect and I expect all manufacturers are working to copy it.

A lot of things are being tweaked in software that didn't happen when Michael first wrote the article. Some smart guys think ETTR conflicts with such software tweaks:

http://chromasoft.blogspot.co.uk/2009/09/why-expose-to-right-is-just-plain-wrong.html

http://theonlinephotographer.typepad.com/the_online_photographer/2011/10/expose-to-the-right-is-a-bunch-of-bull.html

I bracket my shots always to avoid clipping essential highlights and in severe cases also to be able to to an HDR merge of several exposures. I have not seen color shifts that I could see with my eyes. I have often edited on shot in the bracket sequence and copied the edit to all others in Lightroom and adjusted the exposure for the difference in camera exposure and there are very very close in look except for possible blown highlights and noise in shadows. I have nor done any "scientific" tests but for landscape photography I have not seen color shifts using this method that I could see on my wide gamut display.

The other reason that I bracket for landscape photography is to not chimp all the time which takes the attention away from what I really want do: Shoot different compositions and sometimes very fast when the light changes very quickly. In such cases there is no time to chimp, check histograms and do exposure compensations. I could alternatively always underexposure enough that I would not get blown highlights but that would compromise IQ. Using this method I always get the shots where I see others don't get them since they spend too much time fiddling with the camera ;)

In some years I'm pretty sure these problems will have disappeared by sensors that automatically will avoid blown highlights and can be exposed for shadows as much as desired by using a non-global shutter. Basically a shutter per pixel.

Title: Re:
Post by: Herb on November 06, 2014, 06:09:24 am
Unless the article in the link I posted flawed, the ratio of bits representing photon fluctuations to the bits representing the total signal amplitude is constant for all exposures, i.e. for all signal-to-noise ratios. Reducing the S/N does not change the bit ratio.

Is the article flawed?

Is it true the ratio of bits representing photon fluctuations to the bits used for the total signal remain constant?

These articles are too technical for me. My best understanding is the diagram posted by Guillermo Luijk here:

http://www.luminous-landscape.com/forum/index.php?topic=56906.40

The current thinking seems to be that Canons needlessly add noise after ISO amplification whereas Sony and others do not.  Consequently, ETTR can improve the S/N for Canon; much less so for Sony. (John Sheehy was preaching the problem with Canon a decade ago on dpreview.)

My impression is that, as time goes on, manufacturers will do more re-work with software before even the raw is delivered. Those purists who hope to understand the workings of hardware so that they can optimize their quality may be increasingly frustrated. Engineers are constrained by the laws of nature; software writers are not. In their world, anything goes. Whether the results are good or bad for you may depend on your outlook.

James Russell, posting as bcooter:

'A lot of photographers raised in the digital age don't know how film looked.   To them the standard look is Canon on nikon dslrs which are somewhat overly smooth and somewhat global in color.  Also digital tends to pick up a great deal of ambient color.  In other words a brown room makes for a brown photo, even with specific lighting.   The olympus look more like film, in the fact film was kind of dumb.  It saw what it saw and didn't usually pick up ambient color.  Also with film, once you learned a specific film you knew how it would react regardless of setting.  (That one is hard to explain but you know it when you see it).....

Now this one blows me away.  i'm usually not one to say this camera costs less than that camera, but for a $1,200 camera (the em-5 I bought new) vs. a $6,000 Canon there should be a difference titled towards the higher price and in image quality there isn't, it's the other way.   maybe the Canon has a fraction more noise reduction and a fraction more detail (on this I'm really not sure), but the look is not near as nice or specific . . . or film like.

Obviously olympus is on to something good, because Sony and Fuji have tried to emulate the look.   I don't think they'll hit the built quality, but the looks like a camera style seems to have caught on.'

http://www.luminous-landscape.com/forum/index.php?topic=86337.msg701314#msg701314

I'm guessing the film-like look and grain-like noise that James Russell and others admire is somewhat dependent on Olympus working software magic on a file that has NOT been ETTR'd. Am I mistaken? If the only downside of not using ETTR is to live with the grain-like noise of the E-M5, I'm OK. I'd even choose ISO 800 to GET that look most of the time. Just my outlook; YMMV. Everyone comes at it from a different angle. Living in the UK, we look with envy on the beautiful reflected sunlight that those in the US seem to have on tap. Those folk need ND filters; here we're struggling to achieve usable speeds. For anyone shooting a wedding here today - wet and windy - ETTR will be the least of their worries.
Title: Re:
Post by: Herb on November 06, 2014, 06:14:51 am
... I have nor done any "scientific" tests but for landscape photography I have not seen color shifts using this method that I could see on my wide gamut display.


That's interesting. You use Canon which is a conservative manufacturer; I wonder would it be any different for Sony, Olympus or Fuji?

Anyway; I had a look at your site. I think you should keep doing whatever it is you're doing! Beautiful work.
Title: Re:
Post by: Hans Kruse on November 06, 2014, 06:37:05 am
That's interesting. You use Canon which is a conservative manufacturer; I wonder would it be any different for Sony, Olympus or Fuji?

Anyway; I had a look at your site. I think you should keep doing whatever it is you're doing! Beautiful work.

Thanks :) I use both Canon 5D mkIII and Nikon D810 (D800E before). Especially with the D810 I can now make pictures from a single RAW file that I had to HDR merge with the Canon. By HDR merge I mean merging a bracket sequence using the Photomatix 32 bit plugin and edit the resulting 32 bit TIFF file in Lightroom.

The articles linked were written before Lightroom 4 which changed everything. Before that with LR3 I often struggled getting the results I liked in Lightroom. Since LR4 this has totally changed. Funny to think back on those threads here in the LuLa forum where people speculated if it was even worth while upgrading to LR4  :D With LR4.1 we also got support for 32 bit TIFF files which saved the day for my Canon.

In more challenging lighting situations I reach for the Nikon more and more. I shoot both systems since I have mostly Canon and Nikon shooters on my workshops.

I have a Sony RX100 III but have not checked for color shifts. As from what I see LR4 and LR5 does an amazing job of moving exposure and keep the look with only very minor differences (except for blown highlights and noise). If I take a bracket sequence -1EV, 0EV and +1 EV in the camera and in Lightroom edit the 0EV and copy the edits to the -1EV and +1EV and adjust the -1EV with +1EV  and the +1EV with -1EV in LR and compare they look really very very similar (except for blown highlights and noise). I use the highlights and shadows sliders a lot in LR and noise does come out if the exposure is not optimal: The most exposed from the bracket sequence that does not have essential highlights blown and judged in Lightroom. The automatic highlight recovery does on a few occasions cause artifacts in small areas where one or more channels are blown. I such cases I check with Rawdigger to see where the blown out areas are and then check to exposures against each other. Often I realize that is not a problem where I thought there was. Having the optimal exposure and especially for landscape photography I can edit a photo to where I want without any problems with noise and in probably 95%+ of the cases also with the Canon!

A little side note: I wish that Lightroom would implement a RAW histogram and clipping indicators from the RAW file. It would save the round trip to Rawdigger in cases where I suspect there could be as issue with clipping.

Basically my shooting philosophy has been for years now for landscapes to postpone the choice of the optimal exposure to review Lightroom. Memory cards are large and computers are fast so it is a quick process in Lightroom.
Title: Re:
Post by: Eyeball on November 06, 2014, 07:24:48 am
The olympus look more like film, in the fact film was kind of dumb.  It saw what it saw and didn't usually pick up ambient color.

Can someone explain to me how this is/was possible?

I understand how different films had different color responses but how does film NOT pick up ambient color in subjects (the brown room mentioned or I suppose green reflected off of human subjects in an outdoor shot).
Title: Re: The Optimum Digital Exposure
Post by: Ray on November 06, 2014, 07:56:29 am
Ray, FineArt...

Thanks for your feedback.  I find your perspectives on this fascinating.

Rand

Your welcome! Thanks for the courtesy.
Title: Re:
Post by: Bart_van_der_Wolf on November 06, 2014, 07:59:29 am
Can someone explain to me how this is/was possible?

I understand how different films had different color responses but how does film NOT pick up ambient color in subjects (the brown room mentioned or I suppose green reflected off of human subjects in an outdoor shot).

Hi,

It's an interesting observation, but there is no physical explanation possible (based on the information that was provided). Most of such observations are based on totally different color rendering workflows, which could introduce all sorts of color differences.

Cheers,
Bart
Title: Re: The Optimum Digital Exposure
Post by: bjanes on November 06, 2014, 08:55:07 am
Bill, I've thought about it a little more, and I've come up with one significant problem. The white point of PPRGB is a linear transform of the CIE 1931 Standard Observer's response to the D50 illuminant. What if the white point of the native sensor is different from that? Specifically, most sensor's white points appear to be magenta-ish (the corrected JPEG looks green). If the sensor is exposed to a color with the chromaticity of its native white point (eliding capture metameric errors), as that color gets more illuminated, the PPRGB histogram will indicate clipping in its red and blue channels before the those channels of the sensor actually clip. Conversely, If the sensor is exposed to a color with the chromaticity of D50 (eliding capture metameric errors), as that color gets more illuminated, the PPRGB histogram will not indicate clipping when the green channel(s) of the sensor start to clip.

I've used a little shorthand in the above explanation, and will expand it to the point of tediousness if it's not clear.

I think that's right, anyway.  Does that make sense to you?

Jim,

Your explanation does make sense. White balance does complicate things. I remember a recent thread when it was demonstrated that AdobeRGB contains colors that are not included in ProPhotoRGB.

However, wouldn't the same problem occur with the D65 white point of AdobeRGB? Another wider colorspace for the camera rendering of the preview JPEG would be Wide-GamutRGB, which does not include any imaginary colors. I understand that this space is available in some Canon cameras.

Bill
Title: Choice of raw converter
Post by: Herb on November 06, 2014, 09:36:48 am
... with LR3 I often struggled getting the results I liked in Lightroom. Since LR4 this has totally changed.

IIRC, Michael's original article acknowledged Thomas Knoll as the originator of the ETTR approach. Maybe it's not surprising that Adobe software handles it well. Perhaps the developers are still aware that people will be exposing this way.

OTOH, Japanese developers may be less accommodating.

The Olympus cameras are widely praised for good color. 'Looks like film'. Portra is often cited. 'Long shoulder'. I wonder, though, if getting this good color requires leaving the camera to do its own thing with exposure. If thousands of hours of development work has gone into building a low exposure system that achieves a long shoulder, but the users consistently ETTR..... won't that dilute the benefits?

Ctein has a couple of good articles on The Online Photographer basically implying that Olympus does whatever gives good results rather than ensuring blind compliance with ISO or other standards:

http://theonlinephotographer.typepad.com/the_online_photographer/2012/09/why-iso-isnt-iso.html

http://theonlinephotographer.typepad.com/the_online_photographer/2012/10/raw-is-not-raw.html

Just about everyone says that, for all its shortcomings, Olympus software gives the best color. Maybe that's not surprising... and maybe getting that color necessitates passing on ETTR?
Title: Re:
Post by: Herb on November 06, 2014, 10:05:29 am
Can someone explain to me how this is/was possible?

I understand how different films had different color responses but how does film NOT pick up ambient color in subjects (the brown room mentioned or I suppose green reflected off of human subjects in an outdoor shot).

In the same vein:

James Russell:
'I love the fact that with film, that film was kind of stupid.  Once you zoned in on a film stock, you knew how it would react regardless of ambient colour bounce, but with digital, it just seems like it's always hit and miss and like T says, the back end on digital is a monster.

I'm amazed I can shoot 10 subjects on white and have to adjust skin tones (usually a lot)  to stop casting, red in shadows, yellow in transitional areas, etc. etc.

You see it in cinema and television also.  Watch someone set at a desk and drop their head.  The brown (red) of the desk just throws their faces red in the medium shadows) and then they go back up to the key and they go yellow, because the colorist probably working on lack of time and budget had to make a middle of the road decision.

Film doesn't do this as much (it can), but digital, is too sensitive.   

In fact the prettiest way to work digital is to desat the whole image and paint back the saturation where you want it.  Slightly, but it does give more of a film impression.'

TMARK:
'When I first started seriously working with digital (1ds) I would look at the files and wonder where the light pollution was coming from.  I was looking for slow shutter and wide apertures and wondering if my modeling lights were polluting teh shadows, making them red(ish), yellow transitions in different frames, wondering if my packs were bad or flash tubes were bad.  I spent money at Flash Clinic and shot in blacked out rooms and realized its just different than film.  An easy fix in any case, but man I enjoy just picking up a yellow box from Duggal with my perfect yellow prints.'

James Russell:
'It's all a semi easy fix.  What changed with digital to film was with film, when we sent contact sheets they had to be semi close, but not perfect, with transparencies they had to be spot on (remember buying cases of the same emulsion?), but with digital with the 1ds, I'd just use the jpegs out of camera for the galleries and nobody seemed to mind, though I still think the 1ds transitioned better from film to digital than most of the newer digital cameras.

I think digital like the 5d3 is just too everything.   Too smooth, too color receptive, too . . . I dunno  . . . digital.

Then again I like the mft cameras because they don't have a huge ambient color range, they do noise up after 800 iso and they look t me like film . . . but I shot epr transparency film that even at 64 asa was still grainy, so what d I know?'

http://www.luminous-landscape.com/forum/index.php?topic=79179.80
   

Title: Re: The Optimum Digital Exposure
Post by: Jim Kasson on November 06, 2014, 10:08:21 am
Jim,

Your explanation does make sense. White balance does complicate things. I remember a recent thread when it was demonstrated that AdobeRGB contains colors that are not included in ProPhotoRGB.

However, wouldn't the same problem occur with the D65 white point of AdobeRGB?

Absolutely. UniWB is all about getting around that, and I think that WB is probably the long pole in the make-my-camera-histogram-look-like the-real-raw-histogram tent, since all RGB color spaces have the same gamut shape, measured in their own color space: it's a cube. Can I make a statement that the particular color space chosen doesn't matter? Not sure about that. I'll think on it.

Jim
Title: Re: Choice of raw converter
Post by: deejjjaaaa on November 06, 2014, 10:09:06 am
Maybe it's not surprising that Adobe software handles it well.

w/ all due respect it was not till "process 2010" that Adobe products could be called handling anything well...  
Title: Re: The Optimum Digital Exposure
Post by: deejjjaaaa on November 06, 2014, 10:13:47 am
Can I make a statement that the particular color space chosen doesn't matter?
for as long as you use UniWB you can - UniWB allows (along with proper contrast settings) to get existing OOC JPG even in sRGB (not even AdobeRGB) to get within <= 1/6 EV precision displaying clipping in raw channels... that was on each camera that I owned (Pentax, Panasonic, Olympus, Sony)... so if you are willing to live with a green tint the only hassle is to setup UniWB once (takes anywhere from 1 min to 30 min based on particular brand).
Title: Re: Choice of raw converter
Post by: Hans Kruse on November 06, 2014, 10:22:26 am
w/ all due respect it was not till "process 2010" that Adobe products could be called handling anything well...  

I would say PV2012!
Title: Re: The Optimum Digital Exposure
Post by: dwswager on November 06, 2014, 10:42:01 am

Two factors are involved in the misleading camera histogram. The first is white balance. For daylight, the red multiplier is close to 2x. The raw channel can be intact before white balance but blown after WB. The other factor is that the color space of AdobeRGB can not contain the gamut of the flower. If the camera makers don't want to change their approach to give a raw histogram, they could at least offer ProPhotoRGB as a color space for the JPEG preview. To eliminate red channel clipping in the camera histogram required an exposure compensation of -1 2/3 EV.

Bill

At least for Nikon add the selected Picture Control to the list of WB and ColorSpace.  Nikon introduced that Flat picture control for those post processing.  Of course, it is only applied to JPGs AND the historgram, not the RAW file.

Here is the big point though...there is no such thing as a Raw Histogram!  You could make a chart of Raw data values, but they have no real meaning until converted.  A histogram only makes sense once a basic set of assumptions about how to manipulate and distribute the RAW data into an image file are selected.  A camera could allow us to change those assumptions to suit our needs or if the camera knows the systems lower threshold and upper saturation point, and could calculate from the exposure metering where the scene falls, it could tell us how much headroom is still left before saturation.  Of course, metering would have to be done off the sensor plane in Live View.

The fundamental fact that Micheal brought up way back in the beginning of this thread is that camera manufacturers shun anything "not developed here".  I will take it one step further and say they are locked into historic thinking of how cameras and hence photographers worked.  While we certainly don't want to lose that functional paradigm, Digital offers some benefits outside that process paradigm that can be exploited to great advantage in-camera and in post processing, but cameras aren't designed to work that way.  Hence, they are not flexible enough in how they operate.  The DSLR manufacturers need a little consumer electronics perspective.

I don't want to start a flame war, but this is exactly the iPhone/Android dichotomy.  As long as you were willing to stay withing the operating paradigm of the iPhone it was great hardware that executes very well.  But as soon as you try to do something outside what Steve Jobs had already thought of or how he thought it should be done, the iPhone starts defeating you at every turn.  [Full Disclosure:  I own significant AAPL stock and have for many years.  My daughters both have iPhones and my wife an iPad 2.  I use a Samsung Note II.]

Title: Re: Choice of raw converter
Post by: deejjjaaaa on November 06, 2014, 10:48:29 am
I would say PV2012!

I am not going to argue w/ that... but as a whole the difference "2003" vs "2010" is greater than "2010" vs "2013" ... so I wonder - Knoll was the same, what changed at that moment in the development team.
Title: Re: The Optimum Digital Exposure
Post by: Eyeball on November 06, 2014, 10:52:49 am
Here is the big point though...there is no such thing as a Raw Histogram!  You could make a chart of Raw data values, but they have no real meaning until converted.

It seems to me that RawDigger would be evidence refuting both of your points above.

One of the reasons we want to see a raw histogram is to verify to what extent the tonal values were truly clipped or not.
Seems to me that camera manufacturers could do just a good a job as the RawDigger folks, if not better, since they know all the characteristics of their sensors and the processes being used for capture.
Title: Re: The Optimum Digital Exposure
Post by: deejjjaaaa on November 06, 2014, 10:52:57 am
Here is the big point though...there is no such thing as a Raw Histogram!  You could make a chart of Raw data values, but they have no real meaning until converted.  

they actually do... for as long as you understand that it is pre raw conversion/color transform/WB data
Title: Re: Choice of raw converter
Post by: Hans Kruse on November 06, 2014, 10:54:07 am
I am not going to argue w/ that... but as a whole the difference "2003" vs "2010" is greater than "2010" vs "2013" ... so I wonder - Knoll was the same, what changed at that moment in the development team.

I agree that PV2010 was very significant in demosaicing algorithms including noise and sharpening algorithms that improved the basic RAW conversion a lot. PV2012 (not 2013) added tone mapping algorithms that in my view totally changed what was possible. What is the bigger step I can't say, really, both were very significant and catapulted LR/ACR to the top. I almost never used Photoshop any more.
Title: Re: The Optimum Digital Exposure
Post by: Hans Kruse on November 06, 2014, 11:04:03 am
Here is the big point though...there is no such thing as a Raw Histogram!  You could make a chart of Raw data values, but they have no real meaning until converted.  A histogram only makes sense once a basic set of assumptions about how to manipulate and distribute the RAW data into an image file are selected.  

It seems to me that it would be easy to make a histogram of the values that exist in the RAW file. That should be meaningful enough as long as the clipping point is known for the sensor. E.g. that values larger than 15600 for a Canon 5D mkII from what I can read in the CR2 spec. Clearly enough you cannot get an RGB histogram until the decoding via the color matrix has been done.
Title: Re: The Optimum Digital Exposure
Post by: Eyeball on November 06, 2014, 11:13:19 am
Clearly enough you cannot get an RGB histogram until the decoding via the color matrix has been done.

Are you referring to demosaicing?  An RGB histogram can be produced without that.  In fact, software like RawDigger gives you RGGB histograms, which include both sets of green sensel readings.
Title: Re: The Optimum Digital Exposure
Post by: deejjjaaaa on November 06, 2014, 11:13:38 am
Clearly enough you cannot get an RGB histogram until the decoding via the color matrix has been done.
why do you need a decoding though ? you do not like to call it "RGB" histogram because the data is not yet from a proper colorimetric RGB color space - call it per-channel histogram.
Title: Re: The Optimum Digital Exposure
Post by: Hans Kruse on November 06, 2014, 11:15:35 am
Are you referring to demosaicing?  An RGB histogram can be produced without that.  In fact, software like RawDigger gives you RGGB histograms, which include both sets of green sensel readings.

Yes, of course, I stand corrected :)
Title: Re: The Optimum Digital Exposure
Post by: bjanes on November 06, 2014, 11:17:15 am
At least for Nikon add the selected Picture Control to the list of WB and ColorSpace.  Nikon introduced that Flat picture control for those post processing.  Of course, it is only applied to JPGs AND the historgram, not the RAW file.

Here is the big point though...there is no such thing as a Raw Histogram!  You could make a chart of Raw data values, but they have no real meaning until converted.  A histogram only makes sense once a basic set of assumptions about how to manipulate and distribute the RAW data into an image file are selected.  A camera could allow us to change those assumptions to suit our needs or if the camera knows the systems lower threshold and upper saturation point, and could calculate from the exposure metering where the scene falls, it could tell us how much headroom is still left before saturation.  Of course, metering would have to be done off the sensor plane in Live View.

Nikon does apply some adjustments such as linerarization to the raw file, but otherwise the raw file does reflect the voltages presented to the ADC and recorded in the raw file. The raw histogram that I envision is similar to what one gets with RawDigger. The white balance, contrast, saturation, etc are only encoded as metadata tags and do not affect the values in the raw file. With the D800 one can view the raw file and perform metering in live view, but I think that this is still done via the JPEG preview data.

Bill
Title: Re: The Optimum Digital Exposure
Post by: Hans Kruse on November 06, 2014, 11:17:19 am
why do you need a decoding though ? you do not like to call it "RGB" histogram because the data is not yet from a proper colorimetric RGB color space - call it per-channel histogram.

No, my mistake. A histogram can be made on the RGB pixels.
Title: Re: The Optimum Digital Exposure
Post by: bjanes on November 06, 2014, 11:23:27 am
Absolutely. UniWB is all about getting around that, and I think that WB is probably the long pole in the make-my-camera-histogram-look-like the-real-raw-histogram tent, since all RGB color spaces have the same gamut shape, measured in their own color space: it's a cube. Can I make a statement that the particular color space chosen doesn't matter? Not sure about that. I'll think on it.

Jim

Jim,
 If the histogram comes directly from the raw data and not from the JPEG preview data, the color space set on the camera for rendering of the JPG does not matter. Let us know what you decide on further reflection. Your  opinions are always valued.

Bill
Title: Re: The Optimum Digital Exposure
Post by: dwswager on November 06, 2014, 11:26:13 am
It seems to me that RawDigger would be evidence refuting both of your points above.

One of the reasons we want to see a raw histogram is to verify to what extent the tonal values were truly clipped or not.
Seems to me that camera manufacturers could do just a good a job as the RawDigger folks, if not better, since they know all the characteristics of their sensors and the processes being used for capture.

I think we are talking past each other.  Yes, one can look at the DAC output of the senor to see what values were represented.  However, the only useful things it tells us with respect to image making is (and we all agree, it's pretty damn useful) is if the sensor hit saturation on one end and/or it there is clipped data on the other end that didn't overcome the sensor response/noise threshold.  Considering we are only concerned with capturing the best data with which we will post process, that is enough.  

But it is not a histogram of the image until assumptions about the camera characteristics and scene are factored in.  Cameras are designed to get the best approximation of what the scene looked like.  I wonder how much thought camera makers actually give to post processing.  What we want is to make the best image from the data, regardless of what the scene actually looked like.  We want to duplicate the emotional impact of the scene as well as the visual impact.  Camera makers are still in the 'film' mode of giving the best final image possible.  Back then post processing was damn time, labor and experience intensive.  Today, we want the best data with which to post process.  Two different things!  
Title: Re: The Optimum Digital Exposure
Post by: deejjjaaaa on November 06, 2014, 11:45:28 am
But it is not a histogram of the image until assumptions about the camera characteristics and scene are factored in.

I see the raw histogram, I see the scene and I know my camera characteristics - so I have all 3 at the moment when shot was done... also real-time-in-EVF/LCD (or even available in post shot review) blinkies/zebra can be used together with raw histogram to enhance the understanding of the raw capture even further
Title: Re: The Optimum Digital Exposure
Post by: deejjjaaaa on November 06, 2014, 11:49:00 am
If the histogram comes directly from the raw data and not from the JPEG preview data, the color space set on the camera for rendering of the JPG does not matter.

and firmware can show both or one of these as you decide... and also technically firmware can imitate FRV's exposure correction emulation along with OE indication during post shot review, so that you can see where clipping is going to happen if you adjust your exposure to get more light.
Title: Re: The Optimum Digital Exposure
Post by: Jim Kasson on November 06, 2014, 12:00:03 pm
If the histogram comes directly from the raw data and not from the JPEG preview data, the color space set on the camera for rendering of the JPG does not matter. Let us know what you decide on further reflection.

Bill, I'm talking about the making the in-camera histogram that's derived from a JPEG preview serve as a raw clipping indicator, so in that case the in-camera histogram does not come directly from the raw data, although the world would be a better place if that were an option.

I've about come down on the side of saying that in theory (as opposed to close enough in practice, which we know is true already) you can make the in-camera histogram that's derived from a JPEG preview serve as a raw clipping indicator with the proper UniWB settings. However, I'm having to make some assumptions about the raw-to-colorimetric conversion process that aren't always met in practice.

Jim
Title: Re: The Optimum Digital Exposure
Post by: barryfitzgerald on November 06, 2014, 12:39:49 pm
My take fairly simple

ETTR useful for low light/high ISO (to keep noise down) not overly relevant to low ISO shooting though modern sensors are good and can pull quite heavily into the shadows at base ISO levels
Title: Re: The Optimum Digital Exposure
Post by: Eyeball on November 06, 2014, 12:51:32 pm
Bill, I'm talking about the making the in-camera histogram that's derived from a JPEG preview serve as a raw clipping indicator, so in that case the in-camera histogram does not come directly from the raw data, although the world would be a better place if that were an option.

I've about come down on the side of saying that in theory (as opposed to close enough in practice, which we know is true already) you can make the in-camera histogram that's derived from a JPEG preview serve as a raw clipping indicator with the proper UniWB settings. However, I'm having to make some assumptions about the raw-to-colorimetric conversion process that aren't always met in practice.

Jim

Jim, I have used UniWB on a few occasions and while I don't care much for the green previews, I found it insightful.  The doubt that I have had though is this:

Can the UniWB histogram be considered at all accurate beyond what it shows in terms of clipping on the right side of the histogram?

I can't express it very well but it always gives me the feeling that it is using a "linear" calculation to simulate what may not be linear (the white balance process) - kind of a "brute force" method, if you will.

I guess I could do some tests and compare the LCD histogram with what I see in RawDigger.
Title: Re: The Optimum Digital Exposure
Post by: bjanes on November 06, 2014, 01:09:08 pm
At least for Nikon add the selected Picture Control to the list of WB and ColorSpace.  Nikon introduced that Flat picture control for those post processing.  Of course, it is only applied to JPGs AND the historgram, not the RAW file.

I'll check into that picture control, but could you let us know how one accesses that flat picture control?

Thanks,

Bill

PS

On checking the Nikon web site online, I see that the linear option is available only on the D810 and cameras released subsequent to the release date of the D810. How could one achieve the same effect for the D800?
Title: Re: The Optimum Digital Exposure
Post by: Jim Kasson on November 06, 2014, 01:10:54 pm
My take fairly simple

ETTR useful for low light/high ISO (to keep noise down) not overly relevant to low ISO shooting though modern sensors are good and can pull quite heavily into the shadows at base ISO levels

That's interesting, exactly opposite to my perspective. You want to minimize read noise. I want to minimize photon/shot noise, and I'm not interested in SNRs below about 10.

Jim
Title: Re: The Optimum Digital Exposure
Post by: deejjjaaaa on November 06, 2014, 01:13:22 pm
That's interesting, exactly opposite to my perspective. You want to minimize read noise. I want to minimize photon/shot noise, and I'm not interested in SNRs below about 10.

sure, but some times you have to - when your exposure is limited by the situation for a given camera/lens that you have w/ you... and then by optimizing read noise you can still move something above your S/N = 10.
Title: Re: The Optimum Digital Exposure
Post by: Guillermo Luijk on November 06, 2014, 01:47:30 pm
It seems to me that RawDigger would be evidence refuting both of your points above.

One of the reasons we want to see a raw histogram is to verify to what extent the tonal values were truly clipped or not.
Seems to me that camera manufacturers could do just a good a job as the RawDigger folks, if not better, since they know all the characteristics of their sensors and the processes being used for capture.

I'm plotting my own RAW histograms since 2008 from DCRAW's RAW data extractions:

Canon 40D genuine RAW histogram where black (1023) and sat (13824) points are clearly seen:

(http://www.guillermoluijk.com/tutorial/dcraw/histsat.gif)


Once the linear data is corrected by those black and sat points:

(http://www.guillermoluijk.com/misc/retrato_HIS_corrected.gif)

It is easy to plot log histograms as RAWdigger does, very useful to photographers:

(http://www.guillermoluijk.com/misc/retrato_HIS_log.gif)

(http://www.guillermoluijk.com/misc/retrato.jpg)


Gabor Schorr's RAWnalyze also plotted RAW histograms time ago (although they were not as nice as mines ;D):

(http://www.libraw.org/sites/libraw.org/files/images/red-raw.png)


An interesting exercise I did with two Canon 350D RAW files 4 stops apart (one was the the ETTR; to be fair I clipped some few G pixels), was to merge them into an output 16-bit DNG RAW file with more DR than today's best Sony sensor's single shots. Looking at the RAW histograms is claryfying of the merging process (the merge took the best parts of each histogram, basically the most exposed non-clipped pixels):

(http://www.guillermoluijk.com/article/virtualraw/histos.gif)

(http://www.guillermoluijk.com/article/virtualraw/resultado_lite.jpg)


An histogram is an statistical tool, it doesn't need an image to exist. RAW histograms are very useful, and introducing them in any camera would be very simple. Camera makers are not interested.


Regards

Title: Re: The Optimum Digital Exposure
Post by: Eyeball on November 06, 2014, 01:52:22 pm
I'm plotting my own RAW histograms since 2008 from DCRAW's RAW data extractions:

Gabor Schor's RAWnalyze also plotted RAW histograms time ago (although they were not as nice as mines ;D):


An histogram is an statistical tool, it doesn't need an image to exist. RAW histograms are very useful, and introducing them in any camera would be very simple. Camera makers are not interested.


Yep, I've been following your work for a long time, Guillermo, and I was a big fan of RAWnalyze, too.  I still have a copy that works with my Canon cameras.  Gabor did some fine work before he passed away.
Title: Re: The Optimum Digital Exposure
Post by: deejjjaaaa on November 06, 2014, 01:53:52 pm
Gabor Schorr

may he rest in peace, he departed untimely.
Title: Re: The Optimum Digital Exposure
Post by: Jim Kasson on November 06, 2014, 04:04:49 pm
Can the UniWB histogram be considered at all accurate beyond what it shows in terms of clipping on the right side of the histogram?

I can't express it very well but it always gives me the feeling that it is using a "linear" calculation to simulate what may not be linear (the white balance process) - kind of a "brute force" method, if you will.

I thought through the preview color space issue, and my gut feeling appears to have been wrong. I'll post something about that soon, but first let me deal with your issue. I will try to restate it: are the intermediate points on a real raw histogram accurate? You'll notice that I changed your UniWB histogram to the real raw histogram. I did that because I didn't want to get into trying to figure out exactly how a proper UniWB histogram differs from a real raw histogram; answering that question for an arbitrary camera is beyond my pay grade.

Before I do that, let's talk about the x-axis on a raw histogram, and the size of the histogram buckets. One way to present a raw histogram is for the x axis to be linear, and all the buckets the same size. This has a certain mathematical purity, since the raw data is linear, but is not very useful to photographers. For one thing, it means that the entire right half of the histogram is the top stop of the dynamic range of the image, and the other 13 or so stops are all crammed into the left half. It also means that half the buckets are devoted to that top f-stop.

One way to deal with this situation to provide something more photographically useful has been to present the logarithm (base 2) of the input values as the x-axis. That's appealing, because one-stop intervals are all the some distance on the plot. In one of those happy accidents associated with the way beam current translated to brightness on a now-ancient cathode ray tube, you can get a rough cut at the logarithmic axis by scaling the raw values into the range [0,1] and raising each one to the power 0.45. In the arcane terminology of digital imaging, that's referred to as "gamma 2.2".

This may seem strange, but it's the same way that Lightroom, which uses a linear color space as its working space, presents its histograms. sRGB JPEG uses a gamma of 2.2 as well. So does Adobe 1998 RGB.

And now, we get to a key word.

"Mr. Clinton, are the histogram values thus presented accurate?"
"It depends on what the meaning of the word 'accurate' is."

Yes, those intermediate values are accurate, in that they're derived from the raw image in a repeatable, mathematically-well-specified manner. But are they useful? IMHO, not very. But then again, again, IMO, the intermediate values in and sRGB histogram don't tell me much. Sure, I can see that there are or aren't a lot of midtone values, but does that help me select the exposure? If we were working with 8-bit images, I could see histogram depopulation from too-aggressive tone curve moves, but those days are well behind us.

One thing you can do with the intermediate values is, once you decide to let something clip (say there's a sunlit window behind your subject), you can look for a little bump that indicates the brightest part of the scene that's not the window.

Does any of that help?

Jim





Title: UniWB histograms, camera color spaces, and clipping estimates
Post by: Jim Kasson on November 06, 2014, 04:37:16 pm
The question that I've been wrestling with today:

Given a UniWB in-camera histogram in an arbitrary colorimetric color space, does clipping on any channel indicate clipping on the raw image?

My initial guess was "Yes", but I don't believe that any more. Let me walk you through my reasoning, and please tell me if I've made an error.

Warning: mathematics ahead
.

I will use this notation: matrices, M, in bold uppercase, column vectors, v,  in bold lowercase.

OK, let's get started.

Let's say that a pixel in a raw file can be represented as the column vector r. If r, g, and b are raw values in each plane (yeah, yeah, I know there are two g values; cut me a little slack here), r = [r g b]' where ' is the symbol for transposition.

Now let's say that the camera manufacturer has provided me with a compromise matrix, C, for converting raw values to some colorimetric color space. If c is a column vector representing a color in that colorimetric space,

c = C * r

Now let's say that we've come up with a diagonal matrix, WB, that when combined with the other variables in the following way:

c = C * WB * r

makes the white point of the raw space the same as the white point of the colorimetric one.

We can create a new matrix, M, that does both the color space conversion and the white point mapping:

M = C * WB

Lets say that M = [a11 a12 a13; a21 a22 a23; a31 a32 a33]

Having the white point of the raw color space and the colorimetric one be the same means that:

[1 1 1]' = M * [1 1 1]'

And that means

a11 + a12 + a13 = 1
a21 + a22 + a23 = 1
a31 + a32 + a33 = 1

Having each raw space clipping point [1 x x]' [x 1 x]' [x x 1]' where x could be anything, map to the clipping points of the colorimetric color space means that


[1 x x]' = M * [1 x x]'
[x 1 x]' = M * [x 1 x]'
[x x 1]' = M * [x x 1]'

and that means

a11 = 1 and a12 = a13 = 0
a22 = 1 and a21 = a23 = 0
a33 = 1 and a31 = a32 = 0

which means that M is the identity matrix, which means that the colorimetric color space has to be the same as the raw space.

So the UniWB trick will work best where all the raw channels approach clipping at the same time; ie, for magenta-ish highlights (since raw images are usually overly green-sensitive).

This is not a wonderful thought. Please, somebody show me where I erred!

Jim

Title: Re: The Optimum Digital Exposure
Post by: Jim Kasson on November 06, 2014, 06:46:16 pm
sure, but some times you have to - when your exposure is limited by the situation for a given camera/lens that you have w/ you... and then by optimizing read noise you can still move something above your S/N = 10.

I hear you, but it's just not a problem in almost all my photographic situations. Let's work through an example with the D810. FWC is about 75,000 electrons. For a shot noise sigma of 10 electrons, I need a mu of 100 electrons.

75,000/100 = 750.

log2(750) = 9.5 stops, at base ISO.

At base ISO, the D810 read noise is about 5 electrons.

http://blog.kasson.com/?p=7656

With my 100 electron signal, the noise is

sqrt(shotNoise ^2 + readNoise^2) = sqrt(5^2 + 10^2) = sqrt(125) = 11 electrons.

So the RN has increased the total noise by one measly electron.

Well, I hear you thinking, read noise isn't white. There are low frequency components that make it uglier than photon noise, so it should get more weight.

There is merit to that way of thinking. I've done an analysis of the D810 read noise vs low-pass filter cutoff frequency, and, with the exception of the really low ISOs (100; I didn't test 64, but it's probably worse), you could argue that you ought to weight the read noise by at most a factor of two in one direction and a factor of four in the other. But I started with base ISO, so let's weight it by 8. so, at Low frewuency:

sqrt(shotNoise ^2 + readNoise^2) = sqrt(40^2 + 10^2) = sqrt(1700) = 41 electrons.

This sound awful, but because it occurs at such low frequency, and the photon noise SNR is 10 or better, you just don't see it.

Now, let's consider a higher ISO. 3200 is a nice number, since the number of electrons per count on the D810 is 1/10 at that point. Now, full scale is 1638 electrons. Our 100 electron signal is log2(1638/100) = 4 stops down from full scale. The read noise of the D810 at ISO 3200 is about 2 electrons. Multiplying it by 4 to get a worst-case number gives 8.

sqrt(shotNoise ^2 + readNoise^2) = sqrt(8^2 + 10^2) = sqrt(164) = 13 electrons.

It's all very well to talk about numbers, but what do the images look like? I went looking for the D810's read noise in this series of tests:

http://blog.kasson.com/?p=7886

I found that I had to get the average exposure of the bright parts of the test image down to low single-digit electron counts before the low frequency component of the read noise was readily apparent.

YMMV, of course.

Jim

Title: Re:
Post by: Jim Kasson on November 07, 2014, 09:20:13 am
Unless the article in the link I posted flawed, the ratio of bits representing photon fluctuations to the bits representing the total signal amplitude is constant for all exposures, i.e. for all signal-to-noise ratios. Reducing the S/N does not change the bit ratio.

Is the article flawed?

Is it true the ratio of bits representing photon fluctuations to the bits used for the total signal remain constant?

Sorry it's taken me so long to respond. I find no error in the article once you accept the assumptions, some of which aren't stated. I'm not sure what the utility of the observation is in photography.

Jim
Title: Re: The Optimum Digital Exposure
Post by: deejjjaaaa on November 07, 2014, 10:20:55 am
Let's work through an example with the D810.
no, no - I 'd take a Canon camera or Sony A7s - but not like something like D810e even it is not the purest example if ISO-less sensor...
Title: Re: UniWB histograms, camera color spaces, and clipping estimates
Post by: deejjjaaaa on November 07, 2014, 10:30:48 am
Please, somebody show me where I erred!

nobody was talking about mathematically exact precision - getting clipping indication with your subjects under daylight to tungsten light spectrum within 1/6 EV is good enough and it does (as you test with rawdigger or FRV when you tune your OOC JPG settings)... do you need to be precise to 0.00000000000001 EV ? nope...

Title: Re: The Optimum Digital Exposure
Post by: Jim Kasson on November 07, 2014, 11:13:12 am
no, no - I 'd take a Canon camera or Sony A7s - but not like something like D810e even it is not the purest example if ISO-less sensor...

I don't have any Canons around. I could work through it all with the a7S, if that would be useful. Increasing the ISO on the a7S from 1600 to 2000 triggers the change in conversion gain, which lowers read noise dramatically. Is that what you're looking for?

BTW, I'm still struggling with getting a good low-frequency read noise metric from my data.

Jim
Title: Re: UniWB histograms, camera color spaces, and clipping estimates
Post by: Jim Kasson on November 07, 2014, 11:17:57 am
nobody was talking about mathematically exact precision - getting clipping indication with your subjects under daylight to tungsten light spectrum within 1/6 EV is good enough and it does (as you test with rawdigger or FRV when you tune your OOC JPG settings)... do you need to be precise to 0.00000000000001 EV ? nope...

That's good, because it's going to depend on the compromise matrix from raw to JPEG color space, and all bets are off if LUT-based color space conversion is used in your raw processor. That said, I've found UniWB to work pretty well in practice with a JPEG space of Adobe 1998 RGB. It's just going to be hard to figure out in general if Bill's PPRGB JPEG color space is better or worse. I might be able to do it for a particular camera.

Jim
Title: Re: The Optimum Digital Exposure
Post by: deejjjaaaa on November 07, 2014, 11:26:33 am
I don't have any Canons around. I could work through it all with the a7S, if that would be useful. Increasing the ISO on the a7S from 1600 to 2000 triggers the change in conversion gain, which lowers read noise dramatically. Is that what you're looking for?
my logic, naturally, is that the further away you are from ISO-less camera the more you are going to use that play with gain (ISO), right ? and the closer you are to the ISO-less camera (may be Nikon D7000 is the ideal example of that in the nature - except cameras where ISO is only by tag in raw file) the less you might be inclined to play with gain (ISO) - unless you want for OOC JPG or postshot-review-in-camera purposes... and naturally if you are shooting some action with D4/D4s (or even A7s) - those are further away from ISO less than D810 - the more chances it seems you have that some important part of the image will be the area below your S/N = 10 and you might get that above 10 by pushing gain/ISO... no ? you have D4 as I recall too.
Title: Re: UniWB histograms, camera color spaces, and clipping estimates
Post by: deejjjaaaa on November 07, 2014, 11:32:36 am
It's just going to be hard to figure out in general if Bill's PPRGB JPEG color space is better or worse. I might be able to do it for a particular camera.
I think if you are creating a model you need to introduce some tolerances there too... I don't think it makes sense to talk about anything related to camera hoping for any kind of precision precision to be better than 1/6 EV, no ? is there a real need to be more precise than 1/6 EV with clipping ? are you really going to tune your exposure that close ? if I could I 'd then rather dial 1/3-1/2 ev back right away (one Iliah Borg always mention that some cameras at least are non linear near clipping)... I talking not about any clipping of course, but clipping in valuable part of the image... as sometimes you want to clip (sacrifice) some specular reflections to achieve the proper exposure for the important parts of the image.
Title: Re: The Optimum Digital Exposure
Post by: Jim Kasson on November 07, 2014, 11:36:16 am
my logic, naturally, is that the further away you are from ISO-less camera the more you are going to use that play with gain (ISO), right ? and the closer you are to the ISO-less camera (may be Nikon D7000 is the ideal example of that in the nature - except cameras where ISO is only by tag in raw file) the less you might be inclined to play with gain (ISO) - unless you want for OOC JPG or postshot-review-in-camera purposes... and naturally if you are shooting some action with D4/D4s (or even A7s) - those are further away from ISO less than D810 - the more chances it seems you have that some important part of the image will be the area below your S/N = 10 and you might get that above 10 by pushing gain/ISO... no ? you have D4 as I recall too.

I can't argue with any of that. I appear to be biased towards getting more light on the sensor than most. With the camera on a tripod, I make most of my exposures at base ISO. When I can't use base ISO, my first thought is to let the histogram slide to the left, and push in post. I only do that for two or three stops usually, because of possible twists, but I've done five in tests successfully with the D4, D800, D810, a7R. When you do that with the M240 you get the dreaded green shadows.  If I get to ISO 800 with the a7S, I jump to 2000 if I can fit the scene in to take advantage of the increased conversion gain.

So we're not that far apart, really. I suspect I put away my camera when the light gets dim sooner than you do. I hardly ever go above ISO 3200 on any camera.

Jim
Title: Re: UniWB histograms, camera color spaces, and clipping estimates
Post by: Jim Kasson on November 07, 2014, 11:38:36 am
I think if you are creating a model you need to introduce some tolerances there too... I don't think it makes sense to talk about anything related to camera hoping for any kind of precision precision to be better than 1/6 EV, no ? is there a real need to be more precise than 1/6 EV with clipping ? are you really going to tune your exposure that close ? if I could I 'd then rather dial 1/3-1/2 ev back right away (one Iliah Borg always mention that some cameras at least are non linear near clipping)... I talking not about any clipping of course, but clipping in valuable part of the image... as sometimes you want to clip (sacrifice) some specular reflections to achieve the proper exposure for the important parts of the image.

Yes, but it's a lot tougher to do a model like that and the parameters are more open to debate and error. I was trying to work with a pencil and paper model. I don't think it's worth the effort for me to develop a computer one.

Jim
Title: Re: UNIWB ETTR
Post by: bjanes on November 07, 2014, 12:17:30 pm
Bill, I'm talking about the making the in-camera histogram that's derived from a JPEG preview serve as a raw clipping indicator, so in that case the in-camera histogram does not come directly from the raw data, although the world would be a better place if that were an option.

I've about come down on the side of saying that in theory (as opposed to close enough in practice, which we know is true already) you can make the in-camera histogram that's derived from a JPEG preview serve as a raw clipping indicator with the proper UniWB settings. However, I'm having to make some assumptions about the raw-to-colorimetric conversion process that aren't always met in practice.

Jim

I've conducted some experiments with daylight illumination (actually 3200K + 80a filter) with the D800e using normal white balance and UNIWB and comparing the camera histogram to the raw histogram as shown by RawDigger. The camera was set to AbobeRGB and the standard picture control was chose.

This image shows nominal exposure histograms by the camera and RawDigger (with the green channels averaged) at daylight WB and at UNIWB. The red channel is blown in the camera histogram, but is 2 EV below clipping in RawDigger. The UNIWB preview gives a better preview and shows no clipping, contrary to the Sunlight WB

(http://bjanes.smugmug.com/Photography/Red-Flower-WB/i-QGP5TZQ/1/XL/101-01-10-Histos-XL.png)

Here are the ACR histograms for AdobeRGB and ProPhotoRGB for the sunlight WB. Note that the AdobeRGB rendering shows red channel clipping similar to that shown in the camera histogram.

(http://bjanes.smugmug.com/Photography/Red-Flower-WB/i-L8XCJRm/0/O/101-01-ACR_AdobeRGB.png)
(http://bjanes.smugmug.com/Photography/Red-Flower-WB/i-Bg9pPJw/0/O/101-01-ACR_ProPhoto.png)

And the ACR Adobe RGB histogram for the UNIWB shot at the metered exposure. Note that the red is no longer blown, similar to what was seen in the camera histogram.
(http://bjanes.smugmug.com/Photography/Red-Flower-WB/i-CztTDgj/0/O/101-10_ACR_AdobeRGB.png)

Giving 2EV more exposure moves the RawDigger two stops to the right and near saturation (a good ETTR exposure). The UNWB histogram shows clipping on the camera histogram, since AdobeRGB can not accommodate the camera gamut at this exposure level. However, the UNIWB image can be rendered into ProPhotoRGB by ACR with no clipping. One would likely use negative exposure compensation in ACR for the ETTR image. This could cause hue shifts and it is not clear if the increased SNR would be worth it in terms of SNR and color accuracy.

(http://bjanes.smugmug.com/Photography/RedFlower2/i-rZk5sz5/0/XL/ExpPlus2-XL.png)

And the ACR histograms without exposure adjustment. Screen captures from a WideGamut monitor, with image assigned Monitor profile and then converted to sRGB for web viewing. Some clipping is unavoidable with these transformations.
(http://bjanes.smugmug.com/Photography/Red-Flower-WB/i-PSHz5G3/0/O/101-15_ACR_ProPhot.png)
(http://bjanes.smugmug.com/Photography/Red-Flower-WB/i-DDXkMj2/0/O/101-15_ACR_ProPhotoExpMinus2EV.png)
Title: Re: The Optimum Digital Exposure
Post by: Ray on November 08, 2014, 10:34:51 pm
Perhaps what is lacking in the current trend of this thread towards the minutiae of ETTR precision, is advice on the practicalities of achieving the desired ETTR exposure in the field, in a manner which allows one to get the shot, or capture the moment.

If one is shooting a static scene with camera on tripod, there should be no problem at all. One simply brackets exposure with mirror up.

The difficulties occur when the camera is hand-held. Choosing the appropriate aperture first to achieve the desired DoF usually applies, although it doesn't have to apply. One can keep the shutter speed constant and bracket aperture if DoF is not an issue. This is not something I've experimented with, so far. Maybe I should.

What I used to do, when shooting with Canon DSLRs, was frequently bracket exposure at a fixed aperture. This was not entirely satisfactory because sometimes the greatest exposure (the slowest shutter speed) which produced the most accurate ETTR shot, was too slow for a sharp image when the camera was hand-held, and/or when the subject was not perfectly stationary.
I would have preferred to have been able to bracket ISO in such circumstances, but my Canon cameras did not have this facility. Maybe later models now have this feature.

What pleases me with my current Nikon cameras, both D800E and D7100, is that one can easily separate exposure and focusing by pressing the appropriate buttons whilst the camera viewfinder is still held to eye. Thus there is no need for any bracketing, unless the scene is so contrasty that one chooses to take different exposures in order to merge to HDR.

Again, this procedure was not possible with my Canon cameras (the latest was the 50D). Or perhaps it was possible but I never discovered or realised that it was.

Essentially, with my Nikon cameras, using a single focusing square with camera in manual mode,  I can assign focusing to the AF-ON button, and exposure to the half-pressed shutter button. If I want to expose for the sky, I can move the camera, and/or focusing square in the viewfinder, till the focusing square covers the brightest part of the scene that I consider merits an ETTR.

With forefinger half depressing the shutter button, I can easily change the shutter speed by turning the wheel with my thumb, until the exposure gauge at the foot of the viewfinder looks right for an ETTR. After achieving the optimal exposure setting, I can then move the focusing square to the part of the scene that I want to be in precise focus, and whilst still keeping the shutter button half depressed, press the AF-ON button, recompose the scene and take the shot.

I can also change the order of these two processes. I can focus first. Having pressed the AF-ON button, there is no need to keep my thumb on the button. Focus is locked. I can then swing the focusing square to the brightest part of the scene that I want an optimal exposure for, make the appropriate shutter speed adjustments whilst still looking through the viewfinder, and with forefinger half-depressing the shutter button recompose the scene and take the shot.

If I find that the adjusted shutter speed required for an ETTR is too slow, I have two options. I can simply use what I think is an appropriate shutter speed and underexpose. Alternatively, since Nikon cameras are not truly ISO-less, there may be a noticeable advantage in raising ISO if a significant raise in ISO is required. This I can do by pressing the ISO button with the left forefinger, then turning the wheel behind the shutter button, using the right thumb. This procedure can also be done with eye still glued to the viewfinder. Voila! What could be simpler!  ;D
Title: Re:
Post by: Torbjörn Tapani on November 08, 2014, 11:23:29 pm
EVF with histogram, blinkies, zebras, peaking. That would be simpler. And IBIS.
Title: Re: The Optimum Digital Exposure
Post by: Ray on November 09, 2014, 12:45:39 am
Not for me. It would be impossible because I don't have a camera with an EVF. I tried a Panasonic FZ200 a while back but didn't like the EVF and sold the camera. 
Title: Re: The Optimum Digital Exposure
Post by: jjj on November 09, 2014, 05:00:57 am
I'll check into that picture control, but could you let us know how one accesses that flat picture control?
PS

On checking the Nikon web site online, I see that the linear option is available only on the D810 and cameras released subsequent to the release date of the D810. How could one achieve the same effect for the D800?
I know you use Nikon, but you can tweak in camera picture styles with Canon (http://www.canon.co.jp/imaging/picturestyle/file/eos-utility.html) and also install 3rd party variations like Marvels LowC  (http://marvelsfilm.wordpress.com/marvels-cine-canon/)which were developed for video shooting where there is no raw option and these flat profiles give you more wiggle room in the grade than the processed jpeg. So it may be worth investigating in Nikon also allow that customisation of the in camera looks.
I use the Marvels LowC as my default for stills too as it give me a preview image that's more useful for exposure than a more cooked standard jpeg file.
Title: Re: The Optimum Digital Exposure
Post by: dwswager on November 09, 2014, 10:57:02 am
If we are going all 'Holy Grail', then what I want as a photographer is simply a metering mode that would estimate the saturation point of the sensor and provide an exposure just below that point.  Without metering at the sensor plane and over all pixels, that is a problem, especially in real time.  I then want 2 numbers from the analyzed shot +/-X and +/-Y where X is the stops the shadows are inside or outside the noise threshold of the sensor and Y are the stops the highlights are inside or outside the saturation point. (Note the shadows part is impossible from one shot if the shadows are clipped because there is no way to tell when they would come above the sensor threshold.  Energy deposition can be integrated over time and saturation value estimated.)

However, what is possible right now is a DR-AEB mode.  That is Dynamic Range Auto Exposure Bracketing mode. Lets assume it requires Matrix Metering, Raw quality and A Priority and continuous shooting mode.  The Settings would be

Bracket Steps in Stops
Shadow Shift in Stops

We don't need to specify the number of shots because the camera is going to do it for us (That's what computers are for!).

Lets assume we set the Bracket Steps to 1.5 and Shadow Shift to 4.  When I release the shutter it takes the 1st shot at the metered exposure and analyses the data.  If the highlights are clipped, it continues taking shots 1.5 stops less exposure until it gets one where the highlights have not saturated the sensor.  Then it moves to the shadows and continues taking shots with additional exposure 1.5 stops above the metered value until the lowest pixel value is 4 stops above the threshold limit of the sensor.  Additional constraints on this are frame rate, buffer capacity and processor speed.  The camera would report back if successful or unsuccessful based on the available shutter speeds to comply with request at the selected aperture and ISO.

In another topic I agreed with a poster that predicted that processing power was the likely ingredient for key technical advancements.  That and the ability to run 3rd party apps on camera like Helicon Remote or specialty bracketing and other stuff I haven't even thought of!
Title: Re: The Optimum Digital Exposure
Post by: Fine_Art on November 09, 2014, 12:21:47 pm
The entire conversation around exposure revolves around the idea you have to get it all in one shot. This is, for many types of photography, a carry over from film that is no longer real. For pro sports or active wildlife it is a real constraint. For a lot of other styles we can embrace the freedom of a stream of data capture over time, using software to put it all back together.

If you missed ETTR but got a 3 shot bracket, you can do something like a median add that weights all frames to an average then adds them, probably creating high enough levels that you are filling a 16 or 32 bit space. You wipe out hot pixels, any extreme level error in a frame (like noise), while getting lots of data.

The light in the scene gives you a stream of data. Stopping motion limits our individual frame to a set amount of data. We can still add more frames.
Title: Re: The Optimum Digital Exposure
Post by: dwswager on November 09, 2014, 01:49:13 pm
The entire conversation around exposure revolves around the idea you have to get it all in one shot. This is, for many types of photography, a carry over from film that is no longer real. For pro sports or active wildlife it is a real constraint. For a lot of other styles we can embrace the freedom of a stream of data capture over time, using software to put it all back together.

If you missed ETTR but got a 3 shot bracket, you can do something like a median add that weights all frames to an average then adds them, probably creating high enough levels that you are filling a 16 or 32 bit space. You wipe out hot pixels, any extreme level error in a frame (like noise), while getting lots of data.

The light in the scene gives you a stream of data. Stopping motion limits our individual frame to a set amount of data. We can still add more frames.

First, I think think getting it in 1 shot, out of the camera is the camera maker's paradigm.  I think that is why they are locked into giving only tools that follow that model.  Even AEB is designed to give the 1 good shot out of 3 - 9 alternatives.  It also isn't something we should lose as a lot of situations call for just that. 

Also, when the DR of the scene is within the capture ability of the sensor in one shot, there are lots of times when one shot is preferable, even when we want to push it right to help the shadows and intend to post process later.  There are inherent penalties and potential problems involved in composting multiple shots.  I use Focus Stacking, but in no way think that is a total replacement for Tilts as a method to increase DOF.  Both are tools that have their strengths and limitations and should be used appropriately.

This brings me right back to processing power and ability to run 3rd party applications in camera.  This does not have to be the 'locked down' versus 'free for all' battle like iPhone/Android.  I think a camera manufacturer that opened up their system to 3rd party apps would benefit and could even absorb functionality into the base camera like Microsoft did with Windows.  I'm not asking the manufacturers to allow 3rd parties to alter how the 'base' camera worked per se, just allow them to utilize that functionality in new and useful ways.
Title: Re: The Optimum Digital Exposure
Post by: bjanes on November 09, 2014, 06:11:37 pm
I know you use Nikon, but you can tweak in camera picture styles with Canon (http://www.canon.co.jp/imaging/picturestyle/file/eos-utility.html) and also install 3rd party variations like Marvels LowC  (http://marvelsfilm.wordpress.com/marvels-cine-canon/)which were developed for video shooting where there is no raw option and these flat profiles give you more wiggle room in the grade than the processed jpeg. So it may be worth investigating in Nikon also allow that customisation of the in camera looks.
I use the Marvels LowC as my default for stills too as it give me a preview image that's more useful for exposure than a more cooked standard jpeg file.

Nikon does allow the PictureStyles to be adjusted. Brightness, contrast, and saturation can all be adjusted. It is a lot of work, but one can check the contrast curve by shooting a Stouffer wedge and rendering the image in camera or with NikonCaptureNX2 or its successor which duplicate in camera processing and with CaptureNX2 one can perform multiple renderings using just one shot. Imatest makes plotting the contrast curve easy.

For those rendering into ProPhotoRGB with ACR or other software, saturation clipping must be considered as well. In the example with the red flower I posted, one must underexpose the camera light meter value by one stop to avoid clipping the red channel with daylight white balance. An ETTR exposure of the red flower requires 2 stops over the metered value. UNIWB can help or one could decrease the saturation in picture control, but the preview image would be washed out. Unfortunately, there does not seem to be much interest in these considerations.

Bill
Title: Re: The Optimum Digital Exposure
Post by: thierrylegros396 on November 10, 2014, 04:38:12 am
Another possibility is to know really well your gear.

After some trials, you'll know how your gear reacts to such situations (red or yellow flowers, snow, dark background,...).

That's why I prefer to keep my gear as long as possible.
Title: Re: The Optimum Digital Exposure
Post by: jjj on November 18, 2014, 09:09:28 am
First, I think think getting it in 1 shot, out of the camera is the camera maker's paradigm.  I think that is why they are locked into giving only tools that follow that model. 
Not exactly a camera maker's paradigm, but a photographer's one. The reality is that for most people's photos, one shot is the only option and even if it isn't, most people do not want multiple slight variations to then faff around with either.
Title: Re: The Optimum Digital Exposure
Post by: Ray on November 18, 2014, 08:59:25 pm
Obviously there is a range of situations in photography and no single technique is suitable for all of those situations.

I'd say the least problematic of those situations, with regard to achieving a good ETTR shot, is when the subject is static and one has a tripod. Shutter speed should not then be a problem. One can bracket to one's heart's content. One doesn't even have to rely upon one's camera's limitation of +/- 2 EV, or +/- 3 EV, if that's the case. Ansel Adams' zone system doesn't apply.

At the other end of the spectrum, when trying to capture a precise moment, whether in sports or for a Henri Cartier-Bresson type of shot, bracketing exposure would not be the best approach. The best ETTR exposure from all the bracketed shots might have a shutter speed which was too slow, and/or might be a less-than-ideal moment of capture. Even a difference of a fraction of a second can be critical when capturing the moment.

In such circumstances, bracketing ISO would be preferable. The best exposure might still not be the best moment captured, but at least the shutter speed should be sufficient.

Choosing a shutter speed sufficient to freeze the action and/or camera shake, and choosing the aperture required for the desired DoF, or even an aperture because one knows it is the sharpest, should be under the control of the photographer, as far as is practical. Allowing the camera to make that choice for one, is not ideal for the creative photographer.

One of the great attractions of the current choice of Nikon cameras, for me, is their essentially ISO-less nature. They are not completely ISO-less, but most of the time are close enough to being  ISO-less, for me, because I'm not completely obsessive about noise.  ;)

For example, as I mentioned before in this thread, in relation to a comment from Bill Janes, if one underexposes 6 stops with the Nikon D800E at base ISO, instead of using ISO 6400, one loses about 0.9 EV of DR, which is noticeable. However, at least half of this loss occurs between the base ISO of 100 and ISO 200.

In other words, if one underexposes by one full stop at ISO 100, on the D800E, instead of using the same exposure at ISO 200, which would produce an ETTR shot at ISO 200, one loses one full stop (or EV) of DR. However, if one increases the ISO setting to 200, using the same exposure, one loses only 0.47 EV. One gains an improvement of at least 1/2 a stop of DR. That's noticeable.

For this reason, if one wishes to use the D800E as though it's ISO-less, one should use ISO 200 as a base ISO. For example, let's consider what happens if one underexposes one full stop at ISO 200 instead of increasing ISO to 400. According to DXOMark (ain't DXO wonderful, Jeremy  ;D ), one loses only 0.08 EV in DR. That's totally irrelevant.

What happens if one underexposes 4 stops at ISO 200, instead of using ISO 3200? One loses 1/10th of a stop of DR. Still irrelevant.

How about a 5-stop underexposre at ISO 200, instead of using ISO 6400? We now lose 0.37 EV of DR, about 1/3rd of a stop. Hmmm! Not particularly relevant for me, although pixel-peepers might consider it so.  ;)

Conclusion? If you want to take advantage of the ISO-less nature of the D800E, and always choose your own aperture and shutter speed, then use ISO 200 as base whenever you are confident that it will not result in blown highlights as a result of your chosen aperture and shutter speed. The exposure indicator in the viewfinder can be a useful guide, allowing for the fact that a certain degree of overesposure, as shown in that indicator, might be a 'correct', or ETTR exposure for RAW shooters.

Of course, DR is not everything but it seems to be the one parameter that can be most influenced by changes in ISO settings. SNR at 18% grey is largely unaffected on the D800E by choosing a higher ISO instead of underexposing. For example, a change of 3dB in SNR is equal to a change of one stop of exposure. A 6-stop underexposure at ISO 100, on the D800E, should result in an 18dB reduction in SNR at 18% (skin tones). Using ISO 6400 instead of underexposing 6 stops, results in a lowering of SNR by 17.9 stops. One gains 0.1dB. Irrelevant!