Precise Digital Exposure

[AlterEgo]

You originally said 'demosaic and colour transform'.  Twice.  Unless "colour transform" includes white balance.  I assumed colour transform means conversion to a colour space.  

But once again, this is all dodging around the point of the discussion.

sorry... I forgot to insert the WB between demosaick and color transform... WB operation is closely related to a color transform as ColorMatrix tags are part of dcp profile (in fact if your profile has only CM tags - those are guiding both WB and color transform operations) and potential interpolation of various matrices/luts from dcp profile also depends on your selection of white balance in ACR/LR UI (when you have a dual illuminant profile).

[bernie west]

So how is it possible for the exposure slider to 'normalise' blown highlights in the rendered data then?  And no, I'm not talking about 'highlight reconstruction' from 1 or 2 channels.  I'm talking about when all three channels are blown in the LR histrogram (and in the jpg preview).  If the slider worked on the post rendered data, then it shouldn't be able to pull blown highlights back into normal ranges.

[Guillermo Luijk]

For those interested I opened a specific thread not to deviate the Precise Digital Exposure discussion with the hue shift off topic, here:

http://forum.luminous-landscape.com/index.php?topic=99659.0

Regards

[AlterEgo]

So how is it possible for the exposure slider to 'normalise' blown highlights in the rendered data then?  And no, I'm not talking about 'highlight reconstruction' from 1 or 2 channels.  I'm talking about when all three channels are blown in the LR histrogram (and in the jpg preview). 

then Adobe invents the data for all 3 RGB channels to make it look nice visually - for example you can a try to produce nice gradual transition to "white"... I am not sure why to you think that it is necessary to have "raw" data to engage in inventing things that looks nice ? the mere fact that some software (PS ?) does not invent things does not mean anything except that it was not designed to do so.

If the slider worked on the post rendered data, then it shouldn't be able to pull blown highlights back into normal ranges.

why ? they are all numbers and you do whatever you want with them...

[bernie west]

You really don't make much sense.


Adobe doesn't invent data for all three channels, as you could confirm for yourself if you did a comparison between a dcraw conversion and a LR conversion.

Recovering detail from a nominally blown raw file doesn't appear to be a case of making a "nice gradual transition to "white".  It's about bringing actual detail back into the image.  

[bernie west]

Top one - jpg out of camera.  Bottom one - highlights recovered in Lightroom.

[AlterEgo]

You really don't make much sense.

 

Adobe doesn't invent data for all three channels, as you could confirm for yourself if you did a comparison between a dcraw conversion and a LR conversion.

if the data in all 3 channel is "clipped" what do you think is happening ?

You appear to be trolling.  Have you ever recovered detail from a nominally blown raw file?  I have, many times, and no, it's not a case of making a "nice gradual transition to "white".  It's about putting actual detail back into the image.  

Bernie, that's why I suggested to read the originals, not watch 30 minutes hearsay  

PS: if the details "appear" that simply means "nominally blown raw file" was not "nominally blown" in all channels

[digitaldog]

if the data in all 3 channel is "clipped" what do you think is happening ?

ASAIK, based on text from Thomas and Eric at Adobe, if one (perhaps two?) channels are clipped, they can reconstruct them from the remaining data. I don't believe they've ever indicated they can do this if all three are actually clipped although PV2012 *may* be different.

[AlterEgo]

Top one - jpg out of camera.  Bottom one - highlights recovered in Lightroom.

that does not illustrate anything, Bernie... what you see on screen (and what raw converter in camera saves as "OOC JPG") is not what is inside the raw converter after demosaick and all those operations before exposure adjustment for its code to work with, that's it... when you play with exposure adjustment in LR/ACR the code does not work with some fixed result of some previous exposure adjustment (like when you load that "OOC JPG")...

[bernie west]

if the data in all 3 channel is "clipped" what do you think is happening ?

Exactly (in a broad sense) what is happening in DCRAW when all three channels are clipped in the jpeg and LR histogram.  As I said, you can simply test this yourself by doing a side by side raw conversion with DCRAW vs LR.  Have you ever tested this yourself?  It doesn't sound like it.  I and a lot of others have, and the only differences are small differences.  BOTH raw converters recover more or less the same highlight detail.  LR, because it isn't truly linear as described by a number of posters in this thread, can get some funky stuff happening in the colours sometimes.

PS: if the details "appear" that simply means "nominally blown raw file" was not "nominally blown" in all channels

I don't think you've really followed what this debate  is actually about.  No one is suggesting that data blown in the actual raw data itself can be saved (other than through rebuilding via other channels where possible).  What we are talking about is where the LR histogram indicates that all three channels are blown.  According to your description, that data is lost.  According to reality, it's not necessarily blown.  THAT'S the whole point about ETTR.  

[bernie west]

that does not illustrate anything, Bernie... what you see on screen (and what raw converter in camera saves as "OOC JPG") is not what is inside the raw converter after demosaick and all those operations before exposure adjustment for its code to work with, that's it... when you play with exposure adjustment in LR/ACR the code does not work with some fixed result of some previous exposure adjustment (like when you load that "OOC JPG")...

Ok, well if that's the case, then the LR environment would have to be greater than 16bit to hold all this extra headroom that's outside the 16bit limit after white balance and gamma etc.

[AlterEgo]

ASAIK, based on text from Thomas and Eric at Adobe, if one (perhaps two?) channels are clipped, they can reconstruct them from the remaining data.

we were talking about the case when everything is clipped - then no details can be invented ("reconstructed"... one can understand how data is reconstructed in case of ECC codes - because it was not actually lost) but raw converter can keep the area nicely white (invent the color to fill - no details) for a pleasant visual look

for Bernie's benefit some quotes from EChan

"the DNG processing model performs a linearization of the original raw image values followed by demosaicing, then white balance. All of the other image stages follow. So to answer your question, all of the image ops except for linearization (which isn't under user control anyways) happens after demosaicing. "

"In the minus direction, the only difference is that ACR tries to keep clipped whites white (i.e., we did not feel it was photographically useful to let speculars turn into gray blobs, though we still let users accomplish that with the point curve).  Remember, when reducing (software) Exposure, there's nothing "above" the sensor saturation point (very unlike at capture time, where reducing the capture exposure can indeed record additional information), so there is a question of how to treat the whites."

[AlterEgo]

Exactly (in a broad sense) what is happening in DCRAW when all three channels are clipped in the jpeg and LR histogram.

we are not talking about dcraw or LR histogram - we are talking about when exposure correction is done inside LR/ACR... that is after demosaick, wb/color transform = not with raw data...

[AlterEgo]

According to your description, that data is lost.  

yes, data is lost (forever that is) - some details and color can be invented/guessed (plus Adobe tries to make it visually nice while inventing/guessing) if 1 or 2 channels still not clipped... if 3 are clipped - see the quote from EChan above... this is not a reconstruction like with ECC, this is purely guesswork - but it works quite close to the reality (or simply logical/pleasant from visual perspective) in a lot of cases

[bernie west]

we are not talking about dcraw or LR histogram - we are talking about when exposure correction is done inside LR/ACR... that is after demosaick, wb/color transform = not with raw data...

We actually were, which is why I made the point earlier that you don't seem to be following the debate very well.  Follow the quote trails back and you'll see.

[bernie west]

yes, data is lost (forever that is) - some details and color can be invented/guessed (plus Adobe tries to make it visually nice while inventing/guessing) if 1 or 2 channels still not clipped... if 3 are clipped - see the quote from EChan above... this is not a reconstruction like with ECC, this is purely guesswork - but it works quite close to the reality (or simply logical/pleasant from visual perspective) in a lot of cases

We're talking past each other.  Once again, no one is talking about the situation where channels are clipped in the linear off sensor raw data.  We are talking about where the data is clipped in the LR histogram and what would be a rendered jpg.  I.e. we are talking about ETTR.

And you haven't addressed a fundamental point from my previous post:  That is, for your description of how the exposure slider works to be accurate, it would have to be working in a data space greater than 16 bits.  So the question is:  Does it do this?  I can imagine that it could be doing this, but as some other poster said, it doesn't make a lot of sense (unless we are missing something).  

[AlterEgo]

We actually were, which is why I made the point earlier that you don't seem to be following the debate very well.  Follow the quote trails back and you'll see.

may be you did, myself just interested in a code flow /stages/ inside ACR/LR...

[AlterEgo]

We are talking about where the data is clipped in the LR histogram

then I am sorry - I have no interest in considering LR (or ACR) histogram at all, when someone needs to see the situation one needs to use tools like rawdigger or fastrawviewer or may be using UniWB raw files with special profile that compensates WB, hidden expocorrections, etc and process 2010 in LR/ACR - too much hassle then... as I am not using LR, but rather using ACR I have no issues to open raw in FRV from XnView/PhotoMechanic/whatever, understand what is (and where, and how) actually clipped and then call PS/ACR from FRV to work w/o paying attention to the histogram to guess what is really clipped...

again - if you want to discuss how other rawconverters do that or how/what histogram in ACR/LR works to indicate something please do not ask me (or argue with me or point me to) in this thread... my interest here is very narrow - that is data processing stages inside ACR/LR

That is, for your description of how the exposure slider works to be accurate

you fail to consider that it is not my description - it is how processing was explained (more then once) by developers, that is exposure correction stage (pull/push/leave as is) in ACR/LR is after demosaick, wb/color transform (dcp has several parts, matrices /CM, FM/ and HueSatMap LUTs are applied before exposure correction)

 

[Anders_HK]

Lets get back on subject…

It is very simple, optimal capture is where as many photons as possible are collected, without clipping any channel.
...
Expose ETTR, or for mid tones if concerned about hue twists. Once capture has been made, we can apply any kind of processing.

Where to locate the maximum of photons collected, and how ?? ?
Ensure shadows will be adequate collected ?


EXAMPLES:

How to expose the attached scenes precisely; Ziczac Bridge, View from Resturant, Bierstadt (painting by)?

Consider exposing for allowing in processing for an extended shoulder transition into highlights of around 1/3 stops more than available highlight recovery (Velvia slide film, page 19-20 in my paper), and for maintaining as much quality pixel info as possible in the "important" parts of image.

Having determined the following extreme points per 1/3 stops increment testing (textural maintained):
(A) Stops above 18% grey card just prior to clipping of first channel
(B) Stops above 18% grey card just prior to clipping of last channel
(C) Stops below 18% grey card just prior black point
(D) Same as C) with maximum negative exposure in post. My assumption is that deliberate negative exposure is only of interest to extend shadow end due benefits of ETTR to other data.

I can check for what falls on -2 ~ -3 stops and upwards which is what for my digital back per my trial and error appear to maintain quality data when some require to be brought upwards in post.

I can spot meter for (A) or (B) above, or if I want to expose per knowing more for any of remaining in above. After, all I need is one shot. To add, although my Leaf back has an excellent RAW histogram, due 80MP it is slower to use than a DSLR and more battery hungry.

Anders

[digitaldog]

Lets get back on subject…
Where to locate the maximum of photons collected, and how ?? ?

What I did was akin to a film test. Bracket a controlled studio setup with a very white tile (BableColor). View where in my raw converter of choice I really clipped all three channels compared to no clipping the exposure below that.

Ensure shadows will be adequate collected ?

That's rather the point of ETTR (optimal exposure).
Once I know the limits of what the sensor can take based on sound exposure of the scene, I know how much more to Expose To The Right of the recommendation of the meter (in my test, incident).