The Optimum Digital Exposure

[Herb]

... with LR3 I often struggled getting the results I liked in Lightroom. Since LR4 this has totally changed.

IIRC, Michael's original article acknowledged Thomas Knoll as the originator of the ETTR approach. Maybe it's not surprising that Adobe software handles it well. Perhaps the developers are still aware that people will be exposing this way.

OTOH, Japanese developers may be less accommodating.

The Olympus cameras are widely praised for good color. 'Looks like film'. Portra is often cited. 'Long shoulder'. I wonder, though, if getting this good color requires leaving the camera to do its own thing with exposure. If thousands of hours of development work has gone into building a low exposure system that achieves a long shoulder, but the users consistently ETTR..... won't that dilute the benefits?

Ctein has a couple of good articles on The Online Photographer basically implying that Olympus does whatever gives good results rather than ensuring blind compliance with ISO or other standards:

http://theonlinephotographer.typepad.com/the_online_photographer/2012/09/why-iso-isnt-iso.html

http://theonlinephotographer.typepad.com/the_online_photographer/2012/10/raw-is-not-raw.html

Just about everyone says that, for all its shortcomings, Olympus software gives the best color. Maybe that's not surprising... and maybe getting that color necessitates passing on ETTR?

[Herb]

Can someone explain to me how this is/was possible?

I understand how different films had different color responses but how does film NOT pick up ambient color in subjects (the brown room mentioned or I suppose green reflected off of human subjects in an outdoor shot).

In the same vein:

James Russell:
'I love the fact that with film, that film was kind of stupid.  Once you zoned in on a film stock, you knew how it would react regardless of ambient colour bounce, but with digital, it just seems like it's always hit and miss and like T says, the back end on digital is a monster.

I'm amazed I can shoot 10 subjects on white and have to adjust skin tones (usually a lot)  to stop casting, red in shadows, yellow in transitional areas, etc. etc.

You see it in cinema and television also.  Watch someone set at a desk and drop their head.  The brown (red) of the desk just throws their faces red in the medium shadows) and then they go back up to the key and they go yellow, because the colorist probably working on lack of time and budget had to make a middle of the road decision.

Film doesn't do this as much (it can), but digital, is too sensitive.   

In fact the prettiest way to work digital is to desat the whole image and paint back the saturation where you want it.  Slightly, but it does give more of a film impression.'

TMARK:
'When I first started seriously working with digital (1ds) I would look at the files and wonder where the light pollution was coming from.  I was looking for slow shutter and wide apertures and wondering if my modeling lights were polluting teh shadows, making them red(ish), yellow transitions in different frames, wondering if my packs were bad or flash tubes were bad.  I spent money at Flash Clinic and shot in blacked out rooms and realized its just different than film.  An easy fix in any case, but man I enjoy just picking up a yellow box from Duggal with my perfect yellow prints.'

James Russell:
'It's all a semi easy fix.  What changed with digital to film was with film, when we sent contact sheets they had to be semi close, but not perfect, with transparencies they had to be spot on (remember buying cases of the same emulsion?), but with digital with the 1ds, I'd just use the jpegs out of camera for the galleries and nobody seemed to mind, though I still think the 1ds transitioned better from film to digital than most of the newer digital cameras.

I think digital like the 5d3 is just too everything.   Too smooth, too color receptive, too . . . I dunno  . . . digital.

Then again I like the mft cameras because they don't have a huge ambient color range, they do noise up after 800 iso and they look t me like film . . . but I shot epr transparency film that even at 64 asa was still grainy, so what d I know?'

http://www.luminous-landscape.com/forum/index.php?topic=79179.80
   

[Jim Kasson]

Jim,

Your explanation does make sense. White balance does complicate things. I remember a recent thread when it was demonstrated that AdobeRGB contains colors that are not included in ProPhotoRGB.

However, wouldn't the same problem occur with the D65 white point of AdobeRGB?

Absolutely. UniWB is all about getting around that, and I think that WB is probably the long pole in the make-my-camera-histogram-look-like the-real-raw-histogram tent, since all RGB color spaces have the same gamut shape, measured in their own color space: it's a cube. Can I make a statement that the particular color space chosen doesn't matter? Not sure about that. I'll think on it.

Jim

[deejjjaaaa]

Maybe it's not surprising that Adobe software handles it well.

w/ all due respect it was not till "process 2010" that Adobe products could be called handling anything well...  

[deejjjaaaa]

Can I make a statement that the particular color space chosen doesn't matter?

for as long as you use UniWB you can - UniWB allows (along with proper contrast settings) to get existing OOC JPG even in sRGB (not even AdobeRGB) to get within <= 1/6 EV precision displaying clipping in raw channels... that was on each camera that I owned (Pentax, Panasonic, Olympus, Sony)... so if you are willing to live with a green tint the only hassle is to setup UniWB once (takes anywhere from 1 min to 30 min based on particular brand).

[Hans Kruse]

w/ all due respect it was not till "process 2010" that Adobe products could be called handling anything well...  

I would say PV2012!

[dwswager]

Two factors are involved in the misleading camera histogram. The first is white balance. For daylight, the red multiplier is close to 2x. The raw channel can be intact before white balance but blown after WB. The other factor is that the color space of AdobeRGB can not contain the gamut of the flower. If the camera makers don't want to change their approach to give a raw histogram, they could at least offer ProPhotoRGB as a color space for the JPEG preview. To eliminate red channel clipping in the camera histogram required an exposure compensation of -1 2/3 EV.

Bill

At least for Nikon add the selected Picture Control to the list of WB and ColorSpace.  Nikon introduced that Flat picture control for those post processing.  Of course, it is only applied to JPGs AND the historgram, not the RAW file.

Here is the big point though...there is no such thing as a Raw Histogram!  You could make a chart of Raw data values, but they have no real meaning until converted.  A histogram only makes sense once a basic set of assumptions about how to manipulate and distribute the RAW data into an image file are selected.  A camera could allow us to change those assumptions to suit our needs or if the camera knows the systems lower threshold and upper saturation point, and could calculate from the exposure metering where the scene falls, it could tell us how much headroom is still left before saturation.  Of course, metering would have to be done off the sensor plane in Live View.

The fundamental fact that Micheal brought up way back in the beginning of this thread is that camera manufacturers shun anything "not developed here".  I will take it one step further and say they are locked into historic thinking of how cameras and hence photographers worked.  While we certainly don't want to lose that functional paradigm, Digital offers some benefits outside that process paradigm that can be exploited to great advantage in-camera and in post processing, but cameras aren't designed to work that way.  Hence, they are not flexible enough in how they operate.  The DSLR manufacturers need a little consumer electronics perspective.

I don't want to start a flame war, but this is exactly the iPhone/Android dichotomy.  As long as you were willing to stay withing the operating paradigm of the iPhone it was great hardware that executes very well.  But as soon as you try to do something outside what Steve Jobs had already thought of or how he thought it should be done, the iPhone starts defeating you at every turn.  [Full Disclosure:  I own significant AAPL stock and have for many years.  My daughters both have iPhones and my wife an iPad 2.  I use a Samsung Note II.]

[deejjjaaaa]

I would say PV2012!

I am not going to argue w/ that... but as a whole the difference "2003" vs "2010" is greater than "2010" vs "2013" ... so I wonder - Knoll was the same, what changed at that moment in the development team.

[Eyeball]

Here is the big point though...there is no such thing as a Raw Histogram!  You could make a chart of Raw data values, but they have no real meaning until converted.

It seems to me that RawDigger would be evidence refuting both of your points above.

One of the reasons we want to see a raw histogram is to verify to what extent the tonal values were truly clipped or not.
Seems to me that camera manufacturers could do just a good a job as the RawDigger folks, if not better, since they know all the characteristics of their sensors and the processes being used for capture.

[deejjjaaaa]

Here is the big point though...there is no such thing as a Raw Histogram!  You could make a chart of Raw data values, but they have no real meaning until converted.  

they actually do... for as long as you understand that it is pre raw conversion/color transform/WB data

[Hans Kruse]

I am not going to argue w/ that... but as a whole the difference "2003" vs "2010" is greater than "2010" vs "2013" ... so I wonder - Knoll was the same, what changed at that moment in the development team.

I agree that PV2010 was very significant in demosaicing algorithms including noise and sharpening algorithms that improved the basic RAW conversion a lot. PV2012 (not 2013) added tone mapping algorithms that in my view totally changed what was possible. What is the bigger step I can't say, really, both were very significant and catapulted LR/ACR to the top. I almost never used Photoshop any more.

[Hans Kruse]

Here is the big point though...there is no such thing as a Raw Histogram!  You could make a chart of Raw data values, but they have no real meaning until converted.  A histogram only makes sense once a basic set of assumptions about how to manipulate and distribute the RAW data into an image file are selected.  

It seems to me that it would be easy to make a histogram of the values that exist in the RAW file. That should be meaningful enough as long as the clipping point is known for the sensor. E.g. that values larger than 15600 for a Canon 5D mkII from what I can read in the CR2 spec. Clearly enough you cannot get an RGB histogram until the decoding via the color matrix has been done.

[Eyeball]

Clearly enough you cannot get an RGB histogram until the decoding via the color matrix has been done.

Are you referring to demosaicing?  An RGB histogram can be produced without that.  In fact, software like RawDigger gives you RGGB histograms, which include both sets of green sensel readings.

[deejjjaaaa]

Clearly enough you cannot get an RGB histogram until the decoding via the color matrix has been done.

why do you need a decoding though ? you do not like to call it "RGB" histogram because the data is not yet from a proper colorimetric RGB color space - call it per-channel histogram.

[Hans Kruse]

Are you referring to demosaicing?  An RGB histogram can be produced without that.  In fact, software like RawDigger gives you RGGB histograms, which include both sets of green sensel readings.

Yes, of course, I stand corrected

[bjanes]

At least for Nikon add the selected Picture Control to the list of WB and ColorSpace.  Nikon introduced that Flat picture control for those post processing.  Of course, it is only applied to JPGs AND the historgram, not the RAW file.

Here is the big point though...there is no such thing as a Raw Histogram!  You could make a chart of Raw data values, but they have no real meaning until converted.  A histogram only makes sense once a basic set of assumptions about how to manipulate and distribute the RAW data into an image file are selected.  A camera could allow us to change those assumptions to suit our needs or if the camera knows the systems lower threshold and upper saturation point, and could calculate from the exposure metering where the scene falls, it could tell us how much headroom is still left before saturation.  Of course, metering would have to be done off the sensor plane in Live View.

Nikon does apply some adjustments such as linerarization to the raw file, but otherwise the raw file does reflect the voltages presented to the ADC and recorded in the raw file. The raw histogram that I envision is similar to what one gets with RawDigger. The white balance, contrast, saturation, etc are only encoded as metadata tags and do not affect the values in the raw file. With the D800 one can view the raw file and perform metering in live view, but I think that this is still done via the JPEG preview data.

Bill

[Hans Kruse]

why do you need a decoding though ? you do not like to call it "RGB" histogram because the data is not yet from a proper colorimetric RGB color space - call it per-channel histogram.

No, my mistake. A histogram can be made on the RGB pixels.

[bjanes]

Absolutely. UniWB is all about getting around that, and I think that WB is probably the long pole in the make-my-camera-histogram-look-like the-real-raw-histogram tent, since all RGB color spaces have the same gamut shape, measured in their own color space: it's a cube. Can I make a statement that the particular color space chosen doesn't matter? Not sure about that. I'll think on it.

Jim

Jim,
 If the histogram comes directly from the raw data and not from the JPEG preview data, the color space set on the camera for rendering of the JPG does not matter. Let us know what you decide on further reflection. Your  opinions are always valued.

Bill

[dwswager]

It seems to me that RawDigger would be evidence refuting both of your points above.

One of the reasons we want to see a raw histogram is to verify to what extent the tonal values were truly clipped or not.
Seems to me that camera manufacturers could do just a good a job as the RawDigger folks, if not better, since they know all the characteristics of their sensors and the processes being used for capture.

I think we are talking past each other.  Yes, one can look at the DAC output of the senor to see what values were represented.  However, the only useful things it tells us with respect to image making is (and we all agree, it's pretty damn useful) is if the sensor hit saturation on one end and/or it there is clipped data on the other end that didn't overcome the sensor response/noise threshold.  Considering we are only concerned with capturing the best data with which we will post process, that is enough.  

But it is not a histogram of the image until assumptions about the camera characteristics and scene are factored in.  Cameras are designed to get the best approximation of what the scene looked like.  I wonder how much thought camera makers actually give to post processing.  What we want is to make the best image from the data, regardless of what the scene actually looked like.  We want to duplicate the emotional impact of the scene as well as the visual impact.  Camera makers are still in the 'film' mode of giving the best final image possible.  Back then post processing was damn time, labor and experience intensive.  Today, we want the best data with which to post process.  Two different things!  

[deejjjaaaa]

But it is not a histogram of the image until assumptions about the camera characteristics and scene are factored in.

I see the raw histogram, I see the scene and I know my camera characteristics - so I have all 3 at the moment when shot was done... also real-time-in-EVF/LCD (or even available in post shot review) blinkies/zebra can be used together with raw histogram to enhance the understanding of the raw capture even further