The Optimum Digital Exposure

[Eyeball]

If the camera makers don't want to change their approach to give a raw histogram, they could at least offer ProPhotoRGB as a color space for the JPEG preview. To eliminate red channel clipping in the camera histogram required an exposure compensation of -1 2/3 EV.

I agree with everything you said, Bill, and the ProPhoto alternatives does sound like a decent compromise.
I suspect though that the camera manufacturers don't provide raw or ProPhoto alternatives for usability concerns for the more general customer base.  They have to weigh the benefits to a few vs. the possible confusion/problems for the many.
ProPhoto is going to look even worse in a Jpeg that somebody assumes is in sRGB (consciously or unconsciously) and it may even begin to stress those measly 8-bits in a Jpeg, even for someone who understands color management.
Maybe they could bury the option about 3 levels down in the menu with a warning.  

[bjanes]

Nice example, Bill. A custom WB can bring the in-camera histogram more into line (but not perfectly so) with the actual raw histogram, but then it becomes useless for judging color. You probably know that well, but it should be mentioned.

That's an interesting idea, but then we could have the opposite problem: the real raw histogram clipping while the PPRGB-derived histogram looks fine.

Jim

Jim,

I do have UNIWB loaded into a custom bank on my camera, but I rarely use it. Too much trouble. In my example, the metered exposure is not optimum, but it does give good results at base ISO.

I don't quite understand your second statement. If the raw histogram is clipped, channels are blown and wouldn't the PPRGB histogram also be clipped? I don't doubt that you are correct, but my understanding is lacking.

Bill

[Eyeball]

That's an interesting idea, but then we could have the opposite problem: the real raw histogram clipping while the PPRGB-derived histogram looks fine.

But does that ever happen?  I can't think of any time I have seen an image unclipped in Lightroom, for example, but that showed up clipped in RawDigger (at least before moving the LR sliders around forcing LR to do more highlight recovery).

[bjanes]

But does that ever happen?  I can't think of any time I have seen an image unclipped in Lightroom, for example, but that showed up clipped in RawDigger (at least before moving the LR sliders around forcing LR to do more highlight recovery).

LR with PV 2012 does automatic highlight recovery and this may hide channel clipping. If you use PV2010 with a linear tone curve (point curve = linear, and sliders zeroed in the main panel), you get a better approximation of what RawDigger shows.

Bill

[Jim Kasson]

I don't quite understand your second statement. If the raw histogram is clipped, channels are blown and wouldn't the PPRGB histogram also be clipped? I don't doubt that you are correct, but my understanding is lacking.

Bill, I said that without a lot of thought. The basis for my thinking was that there are non-visible colors in the PPRGB gamut, including the blue primary (and, kinda, the green one, although that's a niggle). But camera native capture color spaces aren't linear transforms of XYZ, and what passes for a gamut in these spaces isn't what we think of as a gamut in a tristimulus color space traceable to 1931 XYZ.

So, you may be right. I'll think on it.

Jim

[Eyeball]

LR with PV 2012 does automatic highlight recovery and this may hide channel clipping. If you use PV2010 with a linear tone curve (point curve = linear, and sliders zeroed in the main panel), you get a better approximation of what RawDigger shows.

Yep, I realize that and that's why I mentioned it as a possible caveat.  I normally use a linear tone curve but always use PC2012 now.

I guess it's theoretically possible for a camera to produce colors that are out of ProPhoto gamut but it seems like with today's cameras we would be talking very small fractions of a stop if it would happen at all, not the 1 or more stops of deviation that we can sometimes see with the current histograms.

[bjanes]

How is it possible to alter photon noise?
How do you make improvements in photon noise?
Isn't the photon noise level a constant ratio in terms of the integer raw data?

Perhaps this Stanford Computer Optics article is incorrect or misleading?

http://www.stanfordcomputeroptics.com/technology/dynamic-range/photon-noise.html

The article could be misleading if not properly interpreted. It confirms a basic tenet of ETTR in that one should maximize the exposure. However, is discredits the idea that ETTR has to do with the number of levels in brighter f/stops. Those extra levels do a good job of quantizing noise but are not needed to record the information in the image. The presumed MFDB advantage in having a 16 bit file is a myth.  However, the article does not address read noise. With many cameras, increasing the ISO will reduce read noise and incidentally move the histogram to the right.

Bill

[Jim Kasson]

I don't quite understand your second statement. If the raw histogram is clipped, channels are blown and wouldn't the PPRGB histogram also be clipped? I don't doubt that you are correct, but my understanding is lacking.

Bill, I've thought about it a little more, and I've come up with one significant problem. The white point of PPRGB is a linear transform of the CIE 1931 Standard Observer's response to the D50 illuminant. What if the white point of the native sensor is different from that? Specifically, most sensor's white points appear to be magenta-ish (the corrected JPEG looks green). If the sensor is exposed to a color with the chromaticity of its native white point (eliding capture metameric errors), as that color gets more illuminated, the PPRGB histogram will indicate clipping in its red and blue channels before the those channels of the sensor actually clip. Conversely, If the sensor is exposed to a color with the chromaticity of D50 (eliding capture metameric errors), as that color gets more illuminated, the PPRGB histogram will not indicate clipping when the green channel(s) of the sensor start to clip.

I've used a little shorthand in the above explanation, and will expand it to the point of tediousness if it's not clear.

I think that's right, anyway.  Does that make sense to you?

Jim

[deejjjaaaa]

Correct in every respect, except it's an increase in the SNR, which is I imagine what you meant to say..

yes, typing too fast.

[deejjjaaaa]

If the camera makers don't want to change their approach to give a raw histogram, they could at least offer ProPhotoRGB as a color space for the JPEG preview.

then the issue is not with a colorspace for OOC JPG, the issue is with the color transform... you can very well do a proper color transform into sRGB to reflect raw clipping properly using camera's blinkies/zebra and then you do an improper transform into ppRGB to show clipping when all raw channels are safe and sound

PS: you are not getting proper display on your LCD/EVF anyways with ppRGB or similar color spaces spaces for OOC JPG and that is naturally a concern for a camera manufacturers... and if you do not care about proper display then you might as well just tune your OOC JPG to use UniWB, proper contrast, etc... a lot of cameras are quite tunable to show real clipping in raw data... framing does not suffer and proper raw exposure still damage your OOC JPG in any color space, so having a green tint does not change a lot...  so for as long as you not getting true raw histogram (or blinkies/zebra based on raw data) it really does not matter if you have ppRGB... sRGB or AdobeRGB is enough

[Rand47]

Ray, FineArt...

Thanks for your feedback.  I find your perspectives on this fascinating.

Rand

[williamchutton]

mr Kasson naturally means not absolute reduction in photon noise, but reduction in S/N ratio, which naturally involves increase in absolute numbers, but reduction in ratio.

Unless the article in the link I posted flawed, the ratio of bits representing photon fluctuations to the bits representing the total signal amplitude is constant for all exposures, i.e. for all signal-to-noise ratios. Reducing the S/N does not change the bit ratio.

Is the article flawed?

Is it true the ratio of bits representing photon fluctuations to the bits used for the total signal remain constant?

[Hans Kruse]

Maybe digital noise 'was' always ugly, but that could soon be history. Have you see the noise from the Olympus OMD E-M5?

http://robinwong.blogspot.co.uk/2014/10/shoot-model-portrait.html

Most times I'd pass on ETTR to get that noise. Bad taste? Everyone can decide for themselves. But many users are happy about the grain-like effect and I expect all manufacturers are working to copy it.

A lot of things are being tweaked in software that didn't happen when Michael first wrote the article. Some smart guys think ETTR conflicts with such software tweaks:

http://chromasoft.blogspot.co.uk/2009/09/why-expose-to-right-is-just-plain-wrong.html

http://theonlinephotographer.typepad.com/the_online_photographer/2011/10/expose-to-the-right-is-a-bunch-of-bull.html

I bracket my shots always to avoid clipping essential highlights and in severe cases also to be able to to an HDR merge of several exposures. I have not seen color shifts that I could see with my eyes. I have often edited on shot in the bracket sequence and copied the edit to all others in Lightroom and adjusted the exposure for the difference in camera exposure and there are very very close in look except for possible blown highlights and noise in shadows. I have nor done any "scientific" tests but for landscape photography I have not seen color shifts using this method that I could see on my wide gamut display.

The other reason that I bracket for landscape photography is to not chimp all the time which takes the attention away from what I really want do: Shoot different compositions and sometimes very fast when the light changes very quickly. In such cases there is no time to chimp, check histograms and do exposure compensations. I could alternatively always underexposure enough that I would not get blown highlights but that would compromise IQ. Using this method I always get the shots where I see others don't get them since they spend too much time fiddling with the camera

In some years I'm pretty sure these problems will have disappeared by sensors that automatically will avoid blown highlights and can be exposed for shadows as much as desired by using a non-global shutter. Basically a shutter per pixel.

[Herb]

Unless the article in the link I posted flawed, the ratio of bits representing photon fluctuations to the bits representing the total signal amplitude is constant for all exposures, i.e. for all signal-to-noise ratios. Reducing the S/N does not change the bit ratio.

Is the article flawed?

Is it true the ratio of bits representing photon fluctuations to the bits used for the total signal remain constant?

These articles are too technical for me. My best understanding is the diagram posted by Guillermo Luijk here:

http://www.luminous-landscape.com/forum/index.php?topic=56906.40

The current thinking seems to be that Canons needlessly add noise after ISO amplification whereas Sony and others do not.  Consequently, ETTR can improve the S/N for Canon; much less so for Sony. (John Sheehy was preaching the problem with Canon a decade ago on dpreview.)

My impression is that, as time goes on, manufacturers will do more re-work with software before even the raw is delivered. Those purists who hope to understand the workings of hardware so that they can optimize their quality may be increasingly frustrated. Engineers are constrained by the laws of nature; software writers are not. In their world, anything goes. Whether the results are good or bad for you may depend on your outlook.

James Russell, posting as bcooter:

'A lot of photographers raised in the digital age don't know how film looked.   To them the standard look is Canon on nikon dslrs which are somewhat overly smooth and somewhat global in color.  Also digital tends to pick up a great deal of ambient color.  In other words a brown room makes for a brown photo, even with specific lighting.   The olympus look more like film, in the fact film was kind of dumb.  It saw what it saw and didn't usually pick up ambient color.  Also with film, once you learned a specific film you knew how it would react regardless of setting.  (That one is hard to explain but you know it when you see it).....

Now this one blows me away.  i'm usually not one to say this camera costs less than that camera, but for a $1,200 camera (the em-5 I bought new) vs. a $6,000 Canon there should be a difference titled towards the higher price and in image quality there isn't, it's the other way.   maybe the Canon has a fraction more noise reduction and a fraction more detail (on this I'm really not sure), but the look is not near as nice or specific . . . or film like.

Obviously olympus is on to something good, because Sony and Fuji have tried to emulate the look.   I don't think they'll hit the built quality, but the looks like a camera style seems to have caught on.'

http://www.luminous-landscape.com/forum/index.php?topic=86337.msg701314#msg701314

I'm guessing the film-like look and grain-like noise that James Russell and others admire is somewhat dependent on Olympus working software magic on a file that has NOT been ETTR'd. Am I mistaken? If the only downside of not using ETTR is to live with the grain-like noise of the E-M5, I'm OK. I'd even choose ISO 800 to GET that look most of the time. Just my outlook; YMMV. Everyone comes at it from a different angle. Living in the UK, we look with envy on the beautiful reflected sunlight that those in the US seem to have on tap. Those folk need ND filters; here we're struggling to achieve usable speeds. For anyone shooting a wedding here today - wet and windy - ETTR will be the least of their worries.

[Herb]

... I have nor done any "scientific" tests but for landscape photography I have not seen color shifts using this method that I could see on my wide gamut display.

That's interesting. You use Canon which is a conservative manufacturer; I wonder would it be any different for Sony, Olympus or Fuji?

Anyway; I had a look at your site. I think you should keep doing whatever it is you're doing! Beautiful work.

[Hans Kruse]

That's interesting. You use Canon which is a conservative manufacturer; I wonder would it be any different for Sony, Olympus or Fuji?

Anyway; I had a look at your site. I think you should keep doing whatever it is you're doing! Beautiful work.

Thanks I use both Canon 5D mkIII and Nikon D810 (D800E before). Especially with the D810 I can now make pictures from a single RAW file that I had to HDR merge with the Canon. By HDR merge I mean merging a bracket sequence using the Photomatix 32 bit plugin and edit the resulting 32 bit TIFF file in Lightroom.

The articles linked were written before Lightroom 4 which changed everything. Before that with LR3 I often struggled getting the results I liked in Lightroom. Since LR4 this has totally changed. Funny to think back on those threads here in the LuLa forum where people speculated if it was even worth while upgrading to LR4  With LR4.1 we also got support for 32 bit TIFF files which saved the day for my Canon.

In more challenging lighting situations I reach for the Nikon more and more. I shoot both systems since I have mostly Canon and Nikon shooters on my workshops.

I have a Sony RX100 III but have not checked for color shifts. As from what I see LR4 and LR5 does an amazing job of moving exposure and keep the look with only very minor differences (except for blown highlights and noise). If I take a bracket sequence -1EV, 0EV and +1 EV in the camera and in Lightroom edit the 0EV and copy the edits to the -1EV and +1EV and adjust the -1EV with +1EV  and the +1EV with -1EV in LR and compare they look really very very similar (except for blown highlights and noise). I use the highlights and shadows sliders a lot in LR and noise does come out if the exposure is not optimal: The most exposed from the bracket sequence that does not have essential highlights blown and judged in Lightroom. The automatic highlight recovery does on a few occasions cause artifacts in small areas where one or more channels are blown. I such cases I check with Rawdigger to see where the blown out areas are and then check to exposures against each other. Often I realize that is not a problem where I thought there was. Having the optimal exposure and especially for landscape photography I can edit a photo to where I want without any problems with noise and in probably 95%+ of the cases also with the Canon!

A little side note: I wish that Lightroom would implement a RAW histogram and clipping indicators from the RAW file. It would save the round trip to Rawdigger in cases where I suspect there could be as issue with clipping.

Basically my shooting philosophy has been for years now for landscapes to postpone the choice of the optimal exposure to review Lightroom. Memory cards are large and computers are fast so it is a quick process in Lightroom.

[Eyeball]

The olympus look more like film, in the fact film was kind of dumb.  It saw what it saw and didn't usually pick up ambient color.

Can someone explain to me how this is/was possible?

I understand how different films had different color responses but how does film NOT pick up ambient color in subjects (the brown room mentioned or I suppose green reflected off of human subjects in an outdoor shot).

[Ray]

Ray, FineArt...

Thanks for your feedback.  I find your perspectives on this fascinating.

Rand

Your welcome! Thanks for the courtesy.

[Bart_van_der_Wolf]

Can someone explain to me how this is/was possible?

I understand how different films had different color responses but how does film NOT pick up ambient color in subjects (the brown room mentioned or I suppose green reflected off of human subjects in an outdoor shot).

Hi,

It's an interesting observation, but there is no physical explanation possible (based on the information that was provided). Most of such observations are based on totally different color rendering workflows, which could introduce all sorts of color differences.

Cheers,
Bart

[bjanes]

Bill, I've thought about it a little more, and I've come up with one significant problem. The white point of PPRGB is a linear transform of the CIE 1931 Standard Observer's response to the D50 illuminant. What if the white point of the native sensor is different from that? Specifically, most sensor's white points appear to be magenta-ish (the corrected JPEG looks green). If the sensor is exposed to a color with the chromaticity of its native white point (eliding capture metameric errors), as that color gets more illuminated, the PPRGB histogram will indicate clipping in its red and blue channels before the those channels of the sensor actually clip. Conversely, If the sensor is exposed to a color with the chromaticity of D50 (eliding capture metameric errors), as that color gets more illuminated, the PPRGB histogram will not indicate clipping when the green channel(s) of the sensor start to clip.

I've used a little shorthand in the above explanation, and will expand it to the point of tediousness if it's not clear.

I think that's right, anyway.  Does that make sense to you?

Jim,

Your explanation does make sense. White balance does complicate things. I remember a recent thread when it was demonstrated that AdobeRGB contains colors that are not included in ProPhotoRGB.

However, wouldn't the same problem occur with the D65 white point of AdobeRGB? Another wider colorspace for the camera rendering of the preview JPEG would be Wide-GamutRGB, which does not include any imaginary colors. I understand that this space is available in some Canon cameras.

Bill