Color profiling while Exposing To The Right

[Ellis Vener]

deejjjaaaa, " only developers know... neither does Schewe... people like Eric Chan know." Jeff Schewe knows far more in that direction than you give him credit for.

If you want to involve Eric Chan in this discussion all you have to do is look up his profile here and ask him what his thoughts on the subject are.

[Schewe]

you are trying to avoid answering the question - what is the proper exposure of the target from the software (that makes a profile) standpoint... I understand that you do not know the answer, only developers know... neither does Schewe... people like Eric Chan know... hence this is an attempt to summon such people.

Actually, I do know...when making any sort of profile whether DNG or ICC, the photographed (or printed) target chromaticity is evaluated and compared to the expected known target chromaticity and the algorithms calculate what color transforms need to be applied to make the photographed target chromaticity match the expected actual target chromaticity. The further away from the known target chromaticity that a photographed target chromaticity is, the less accurate and less smooth the resulting profiles will be. Obviously, the photographed target chromaticity will be impacted by the photographed targets exposure.

The aim should be an "accurate" exposure that makes the luminance values of the photographed target chromaticity be as close to the expected actual target chromaticity for the best and most accurate (and smoothest) results. Over or under exposing is suboptimal. ETTR of the photographed target chromaticity would be a mistake.

[deejjjaaaa]

Actually, I do know...

then where are the numbers, Jeff ? you have a raw file with xrite passport shot, you open it in ACR - please explain how do you evaluate by numbers whether this shot is good or not, starting with the settings in ACR that shall be used for that... like you know - WB set off which patch ? exposure shall be EV0, contrast 0, the rest of sliders @ that tab 0, no sharpening, NR ? all zero ?, tone curve = linear, etc, etc... then what ? you use a color sampler tool and which patch you shall check (first) and what shall be RGB values there, what are the tolerances ?

imagine that you are doing a repro work...

that is what I am trying to understand first of all - how do I evaluate the target shot before actually making a profile... and the only things I hear back are adjectives, but no numbers.

PS: and the next question will be - how do you evaluate that profile you have - and I am not talking about some artistic intents, because you can't argue about tastes - but I 'd assume we shall have something more substantial that your eyesight to see if the profile built off one target can render another target (different type - not like 2 passports) close to the what the colors shall be... otherwise the whole profiling thing is a herbalife and unless you have a really really odd light then you are better off w/ a standard profile from Adobe + proper WB and profiles for artistic intents are better done by just editing a base profile w/o actually shooting a target, are they not ?

[deejjjaaaa]

deejjjaaaa, " only developers know... neither does Schewe... people like Eric Chan know." Jeff Schewe knows far more in that direction than you give him credit for.

If you want to involve Eric Chan in this discussion all you have to do is look up his profile here and ask him what his thoughts on the subject are.

I guess he is too busy w/ whatever releases and people in the know shall appear only if the discussion is interesting...

[deejjjaaaa]

You're playing a game of semantics.  

I am just asking for some numbers... I understand that a lot of art might be involved but absence of any numbers in the whole profiling thing sounds odd   ... |

Like the recent topic = http://www.luminous-landscape.com/forum/index.php?topic=70491.msg558028#msg558028

"Today I made the usual dual illuminant camera profiles I make for each new model I test.
I find them to be much more accurate than Adobe Standard."

And when I asked how did the author actually found that they are more accurate (how dare I ?!) - I heard the following

dejjjaaaa, your argumentative and insulting posts add absolutely nothing of substance to this forum, I wish you would just to over to DRP to do your trolling.  This is not that kind of forum.

Ignore the people with missing brain cells.

 

and topic starter did not post the original raw files (he was asked), I 'd assume he was afraid to show how the target was actually shot   /"You'd see a CC24 covering almost 95% of the frame." - that is the first sign, right... fill the frame with the target/

[Schewe]

then where are the numbers, Jeff ? you have a raw file with xrite passport shot, you open it in ACR - please explain how do you evaluate by numbers whether this shot is good or not, starting with the settings in ACR that shall be used for that... like you know - WB set off which patch ? exposure shall be EV0, contrast 0, the rest of sliders @ that tab 0, no sharpening, NR ? all zero ?, tone curve = linear, etc, etc... then what ? you use a color sampler tool and which patch you shall check (first) and what shall be RGB values there, what are the tolerances ?

I use Bruce Lindbloom's ColorChecker Calculator to determine what the target chromaticity should be in ProPhoto RGB and simply read the patches in ACR set to ProPhoto RGB...pretty simple really. Just set the calculator to ProPhoto RGB...

If you are lazy, you can download a synthetic ProPhoto RGB target file from Chris Murphy's web site. It gives you the RGB target readouts of ProPhoto RGB. Save it as a TIFF and you can load the photographed target brackets and the synthetic target in ACR and check the readouts for an optimal exposure.

Edited to add Bruce's & Chris' main web site links...

[MarkM]

Actually, I do know...when making any sort of profile whether DNG or ICC, the photographed (or printed) target chromaticity is evaluated and compared to the expected known target chromaticity and the algorithms calculate what color transforms need to be applied to make the photographed target chromaticity match the expected actual target chromaticity.

I use Bruce Lindbloom's ColorChecker Calculator to determine what the target chromaticity should be in ProPhoto RGB and simply read the patches in ACR set to ProPhoto RGB...pretty simple really.

This is a little confusing. It's my understanding that DNG profiles differ significantly from ICC profiles in that DNG profiles work with scene referred data.  Suggesting that it expects some RGB target or chromaticity would imply that the DNG profile editor is working with output referred data, but according to Adobe it's not.

Since it's working with linear raw data, I would think (but it's only a guess) that the profile editor would use the grey patches and normalize the exposure before creating the profile. If the data is linear, this should have almost no effect on the final result. So long as you don't clip any channels or introduce too much noise, you would be able to expose however you want.

[Schewe]

Since it's working with linear raw data, I would think (but it's only a guess) that the profile editor would use the grey patches and normalize the exposure before creating the profile. If the data is linear, this should have almost no effect on the final result. So long as you don't clip any channels or introduce too much noise, you would be able to expose however you want.

I'm talking about evaluating a raw file in ACR for optimal creation of a DNG profile in either DNG Profile Editor or Passport. The closer you are to an optimal exposure, the better the resulting DNG profile. Pretty simple really...the less transforming a profile must do, the more accurate (and smother) the transformed results. Since exposure is gonna have an impact on the photographed ColorChecker target chromaticity you should pick an optimally exposed DNG to make the profile from. Same deal for using an accurate color temp to shoot the target. Right?

[mac_paolo]

I am just asking for some numbers... I understand that a lot of art might be involved but absence of any numbers in the whole profiling thing sounds odd   ... |

Jeff linked those numbers. The same one I found years ago via a 10 seconds google search. You know, sometimes people just don't like to spend time when the speaker is the first to be lazy.

Like the recent topic = http://www.luminous-landscape.com/forum/index.php?topic=70491.msg558028#msg558028

"Today I made the usual dual illuminant camera profiles I make for each new model I test.
I find them to be much more accurate than Adobe Standard."

And when I asked how did the author actually found that they are more accurate (how dare I ?!) - I heard the following

 

I'm the "author" of that thread, which was opened to be helpful, and maybe it reached its goal, taken that almost 30 people got that profile.
I'm not the author of those two comments (which I kinda agree with, time after time), so you're mischievous, at best.

You obviously can check RGB values of the patches against given values. They are objective. What you clearly can't do is to check chromaticity values of colors in a real world scene.
What apparently anyone understood but you, is that these latter tests must be evaluated by eye, and any serious photographer is able to do it.
It doesn't take a genius to figure out which image has more realistic colors and, when the ratio is 9:1 (if not even 10:0) towards the custom DNG corrected, it's not that hard to write a thread like that.

and topic starter did not post the original raw files (he was asked), I 'd assume he was afraid to show how the target was actually shot   /"You'd see a CC24 covering almost 95% of the frame." - that is the first sign, right... fill the frame with the target/

I even lost time to explain that on that very thread an still you keep on this provocateur tone. If you really like to test it's been shot at 1,3", ƒ/5, ISO 125 for the indoor and 1/20", ƒ/5.6 for the outdoor.
Let me add a last thing, when it seems that no one replies to your [rude] requests, chances are that either no one knows the answers (I do, and certainly much more do both Mr. Chan and Mr. Schewe) or, more simply, people is ignoring you.
Have fun solving the riddle.

[bjanes]

I use Bruce Lindbloom's ColorChecker Calculator to determine what the target chromaticity should be in ProPhoto RGB and simply read the patches in ACR set to ProPhoto RGB...pretty simple really. Just set the calculator to ProPhoto RGB...

If you are lazy, you can download a synthetic ProPhoto RGB target file from Chris Murphy's web site. It gives you the RGB target readouts of ProPhoto RGB. Save it as a TIFF and you can load the photographed target brackets and the synthetic target in ACR and check the readouts for an optimal exposure.

Edited to add Bruce's & Chris' main web site links...

This is a little confusing. It's my understanding that DNG profiles differ significantly from ICC profiles in that DNG profiles work with scene referred data.  Suggesting that it expects some RGB target or chromaticity would imply that the DNG profile editor is working with output referred data, but according to Adobe it's not.

Since it's working with linear raw data, I would think (but it's only a guess) that the profile editor would use the grey patches and normalize the exposure before creating the profile. If the data is linear, this should have almost no effect on the final result. So long as you don't clip any channels or introduce too much noise, you would be able to expose however you want.

Mark,

That is a good point. For scene referred data, one should look at the raw file, which is scene referred. The problem with using ACR/LR to determine the adequacy of exposure is that the results will be affected by the tone curve and the BaselineExposure value used by those programs. For exposure, one should look at the gray values as you suggest. As those who have struggled to calibrate with the ColorChecker using Bruce Fraser's manual method know, the contrast setting will affect the gray values above and below mid-gray. The saturation will affect the color values.

The Neutral 5 patch has an optical density of 0.70. One can use Bruce Lindbloom's companding calculator to determine the normalized pixel value for this density as shown. A normalized pixel value of 0.1995 corresponds to about 3616 in a linear 14 bit space. One can check the raw values with RawDigger or a similar probram, using the green channels which have a white balance multiplier of unity.

Regards,

Bill

[RFPhotography]

I use Bruce Lindbloom's ColorChecker Calculator to determine what the target chromaticity should be in ProPhoto RGB and simply read the patches in ACR set to ProPhoto RGB...pretty simple really. Just set the calculator to ProPhoto RGB...

If you are lazy, you can download a synthetic ProPhoto RGB target file from Chris Murphy's web site. It gives you the RGB target readouts of ProPhoto RGB. Save it as a TIFF and you can load the photographed target brackets and the synthetic target in ACR and check the readouts for an optimal exposure.

Edited to add Bruce's & Chris' main web site links...

You also have to choose the illuminant type to use Bruce's calculator, correct Jeff?  One could use the table at the bottom of this page to know what the various illuminant types represent in terms of a white balance?  One of the presets in the calculator may not match up exactly with the WB used for the target.  How do you account for the differences?

[madmanchan]

The main thing with shooting charts for DNG PE is not to clip any of the patches.  Usually the tool will warn you if that's happened and refuse to proceed.  Since the tool does work off the raw data, it's not subject to rendering-related tags in the DNG format (e.g., BaselineExposure). 

If you have a severely underexposed image, the main detriment that you'll find is simply more noise in the color patches, which means the profile may be off a bit.  But overall that's not a big deal, and if you're concerned about this, the solution is bracket some exposures.   

The illuminant used to compute the numbers does matter, of course.  For the single-illuminant profile, DNG PE assumes a D50 illuminant (which is what the normal published numbers assume, too), and for the dual-illuminant profile, DNG PE assumes Standard Illuminant A for the first table, and CIE D65 for the second table (same calibration illuminants that Adobe uses for its Adobe Standard profile).

[bjanes]

The main thing with shooting charts for DNG PE is not to clip any of the patches.  Usually the tool will warn you if that's happened and refuse to proceed.  Since the tool does work off the raw data, it's not subject to rendering-related tags in the DNG format (e.g., BaselineExposure). 

If you have a severely underexposed image, the main detriment that you'll find is simply more noise in the color patches, which means the profile may be off a bit.  But overall that's not a big deal, and if you're concerned about this, the solution is bracket some exposures.   

The illuminant used to compute the numbers does matter, of course.  For the single-illuminant profile, DNG PE assumes a D50 illuminant (which is what the normal published numbers assume, too), and for the dual-illuminant profile, DNG PE assumes Standard Illuminant A for the first table, and CIE D65 for the second table (same calibration illuminants that Adobe uses for its Adobe Standard profile).

Eric,

Thanks for the definite answer. From your reply, I infer that MarkM was correct: since the profiler deals with the raw data, it can correct for minor exposure differences as long as the data are not clipped. However, one should try to have reasonable ETTR for the best signal:noise (ignoring the semantic issues). My point about using ACR/LR to judge exposure was that the tone curve and BaselineExposure do have an effect on the rendering.

As an example, here is an exposure with the Nikon D3 of the ColorChecker under Solux illumination (approx 4700K, hopefully close enough to 5000K for a single illuminant profile) with the Neutral 5 patch raw green channels near nominal exposure with a raw value of 3170, close to the value determined for this patch by Bruce Lindbloom's companding calculator.



The white patch is well short of clipping.



The image appears overexposed with ACR 7.1 Process 2012 using the default settings:



With PV2010 and a linear tone curve, the exposure appears more reasonable:



Using an exposure value of -0.5 EV to offset the BaselineExposure of +0.5 EV gives close to the correct pixel value in the Neutral 5 patch.



My conclusion is that PV2012 with the default settings is not good for evaluating exposure and, with my D3 at least, PV2010 with a linear tone curve and a BaselineExposure compensation gives better results. However, this exposure was rejected by the DNG Profile editor, which reported that the yellow patch (row 3 column 4) was overexposed. Exposing 0.5 EV less gave a valid calibration. Your comments would be welcome.

Best regards,

Bill

PS. I enjoyed very much and learned a lot from your interviews on the LuLa tutorial with Michael and Jeff. I hope they invite you back.

[Schewe]

The main thing with shooting charts for DNG PE is not to clip any of the patches.  Usually the tool will warn you if that's happened and refuse to proceed.  Since the tool does work off the raw data, it's not subject to rendering-related tags in the DNG format (e.g., BaselineExposure). 

Do you agree that getting an optimal exposure is best for making DNG profiles? Do you disagree that using a known reference chart (synthetic PPRGB) is a good way of evaluating various exposure of a bracket in ACR?

[deejjjaaaa]

Do you agree that getting an optimal exposure is best for making DNG profiles? Do you disagree that using a known reference chart (synthetic PPRGB) is a good way of evaluating various exposure of a bracket in ACR?

Jeff - that was the question - what is the optimal (a word "optimal" is not a number of any kind, it is just an adjective) exposure and what are the tolerances... I can't believe that we have a leeway on one stop more, two stops less (just make sure not to clip any raw channels) and get a profile that will be more accurate (for a regular day light or so - no exotic spectrums) than Adobe's standard... does not sound right, does it ?

Plus - can we really test that using Adobe's own tools (no rawdigger - let us stay kosher!) - I understand that DNG PE will warn - but it is clear it will warn only in case of totally unsuitable shot, but what if I want to be as close to perfection as possible...

[deejjjaaaa]

it can correct for minor exposure differences as long as the data are not clipped.

we need to define minor ? shall we... is it 1/3 EV ? is it 1/2 EV ? is it 1 EV ? is it 2 EV ?

However, one should try to have reasonable ETTR for the best signal:noise (ignoring the semantic issues). My point about using ACR/LR to judge exposure was that the tone curve and BaselineExposure do have an effect on the rendering.

but the Adobe's toolset shall be complete - we shall not use any 3rd party tools to evaluate our exposures... so naturally we need to be able to use ACR/LR to make a judgement as to how good our shot is by numbers... that means you select certain raw conversion parameters and take some RGB color readings, right ?

[Rhossydd]

If you are lazy, you can download a synthetic ProPhoto RGB target file from Chris Murphy's web site. It gives you the RGB target readouts of ProPhoto RGB.

Curiously, the numbers specified on the file don't match the actual measured values of the image.

Not disagreeing with any of your comments on this though.

[MarkM]

Sorry if I'm being dense, stubborn, or both, but I would really like to understand this.

It's my understanding that the whole point of ETTR and the reason it works is that we are able to scale the linear raw data without introducing hue/saturation shifts. In other words, there really is no difference, as far as colorimetry is concerned, between exposing and processing normally, and exposing more and bring it down in processing. My own tests have confirmed this for me, and from past reading I think most people here are comfortable with this.

Since we can move the exposure up and down without changing the color, an optimal exposure doesn't really exist. ETTR proponents will say the optimal exposure is the one the captures the most light possible, but this has more to do with shadow noise than color.

So the question is: does profile editor use the grey patches to normalize the exposure?

I would think for it to be useable it would have to. If it didn't, then Schewe's worries would be correct because even a small deviation in exposure would move the patches out of whack. In order to make a truly accurate profile you would need to know exactly what profile editor expects and exactly what your camera's meter is doing. Judging your exposure would require digging into the linear raw data. (Not just measuring processed RGB patches in ACR.)

Since the color picker has everything that profile editor needs to shift the exposure I can see no reason why it wouldn't do it. It should not twist the color and it allows for a lot of flexibility in the exposure, which means users who are not interested in digging into the raw numbers (i.e. most users) will get good results.

So long as you don't clip channels, profile editor will work.

[deejjjaaaa]

I use Bruce Lindbloom's ColorChecker Calculator to determine what the target chromaticity should be in ProPhoto RGB and simply read the patches in ACR set to ProPhoto RGB...pretty simple really. Just set the calculator to ProPhoto RGB...

ОК, so you say that ideal exposure for DNG PE profile building is when the raw file loaded in ACR (we assume v7.x), which set is to ProPhoto RGB output, matches the numbers from Bruce Lindbloom's calculator (we make sure to select the proper illuminant there and proper WB in ACR of course) when you take RGB samples in ACR itself... so what is the tolerance now - how many patches do you personally check (all 24 patches ? only one ? few of them), what are possible tolerances for RGB readings vs calculated numbers, what EV adjustment in ACR (if any) you consider to be allowable ? and are there any other settings in ACR to pay attention to (except selecting ProPhoto RGB as the output space for ACR)... and RGB readings in ACR will be in Melissa RGB, right - different gamma.

[deejjjaaaa]

Since we can move the exposure up and down without changing the color, an optimal exposure doesn't really exist. ETTR proponents will say the optimal exposure is the one the captures the most light possible, but this has more to do with shadow noise than color.

that is if you think that noise does not affect the color... or that nothing wrong happens near very saturation