More Color Checker questions WRT accurate color

[Tim Lookingbill]

By "ideal values" I meant the values for the color checker patches programmed into the DNG editor and used by the color checker profile wizard.

My understanding of what the DNG wizard does is that it reads the input values from the color patches in the loaded image, then creates a transform intended to map the input values to the values programmed into it for the patches. I'm not doing any manual tweaking of the control points so whatever is created by the wizard is what's in the created profile.

Then in ACR I apply that profile to the same input image of the color checker. I expected that the output of the transform would be very close to the values for the patches programmed into the DNG editor. Is that not the case?

As I asked earlier, are the values for the patches that the DNG editor is trying to achieve by the profile transform available anywhere? The reason I ask is that when I compare the Lab values in the ACR application in ProPhoto mode to the published values for the color checker, they don't match numerically and are visibly different.

So nothing I've posted answers this question for you? Or are you waiting for an "official" answer from Eric Chan? I think he already gave it to you somewhere in this thread.

I don't know what we can tell you that would make you understand.

[sandymc]

Then in ACR I apply that profile to the same input image of the color checker. I expected that the output of the transform would be very close to the values for the patches programmed into the DNG editor. Is that not the case?

No, not if you're doing the import with ACR's default settings. There's the various other things that ACR does by default - tone curve and contrast settings to take into account. The series of blog posts I referred to previously show how to do the reconciliation between patch values and "on screen" values. And to the point that Eric made above, even once you've eliminated those things, because of illuminant, etc it's likely to be "close" rather than "very close"

Sandy

[RFPhotography]

Sandy, if the original Raw image of the chart were converted to DNG using Adobe's DNG Converter software, would that eliminate much, or all, of the problem of the Raw converter and be a better starting point for the calibration process?

Alternatively, if you were to set up a Lightroom import preset that zero'd out all the sliders and in the Tone Curve Pane applied a curve that offset the default LR tone curve embedded in the ACR Standard profile (or whichever profile is used as the default), would that be a better starting point?

[RDoc]

So nothing I've posted answers this question for you? Or are you waiting for an "official" answer from Eric Chan? I think he already gave it to you somewhere in this thread.

I don't know what we can tell you that would make you understand.

Actually,  apart from Eric's original comment that the patch values used by the DNG editor are not the same as those published by XRite, I don't think anything you or anyone else on this thread have posted even addresses my question.

Mostly the answers here have dealt with the the difficulty of matching the actual physical patches because of lighting, tone curve matching, camera response, etc. or are completely irrelevant comments about how matching the patches doesn't guarantee that other colors will match.

None of those have anything to do with what I'm asking.

[sandymc]

Sandy, if the original Raw image of the chart were converted to DNG using Adobe's DNG Converter software, would that eliminate much, or all, of the problem of the Raw converter and be a better starting point for the calibration process?

Alternatively, if you were to set up a Lightroom import preset that zero'd out all the sliders and in the Tone Curve Pane applied a curve that offset the default LR tone curve embedded in the ACR Standard profile (or whichever profile is used as the default), would that be a better starting point?

How the raw image is converted won't (shouldn't  ) make a difference.

Yes, if you want to try to match values then a zero'd import is a far easier starting point.

If you really want to push ahead with this, then I'd suggest you download some of the synthetic GM24 images I created back when I was doing the calibration work, and first understand what you get in LR/ACR using an "ideal" image. Then go forward with real images.

You can download the synthetic images here: http://sites.google.com/site/chromasoft/referenceimages

There are also spreadsheets of patch values in different color spaces, etc on the same site.

Sandy

[madmanchan]

How does the user know what control points to add and what adjustments to make?  The user can't refer to the values published on the XRite or Lindbloom sites because the values in the DNG PE are based on a linear gamma.  So where does the user start when trying to make adjustments?  Or is it just to be assumed that the initial step of the matrix transform does the job?

Bob, the DNG Profile Editor was designed to be a visual editor.  Specifically, open an image that shows colors to be "off" in some way.  Click on the problem color in the image.  It'll create a control point.  Then you adjust the color till it looks right.  You can place additional control points to change other colors, or simply to "lock down" specific colors to prevent them from changing.

[madmanchan]

By "ideal values" I meant the values for the color checker patches programmed into the DNG editor and used by the color checker profile wizard.

My understanding of what the DNG wizard does is that it reads the input values from the color patches in the loaded image, then creates a transform intended to map the input values to the values programmed into it for the patches. I'm not doing any manual tweaking of the control points so whatever is created by the wizard is what's in the created profile.

Yes, that's right.  If you build a single-illuminant profile in DNG PE, the target values used by DNG PE are from a set of averaged ColorChecker charts with adopted illuminant of D50.  If you build a dual-illuminant profile in DNG PE, the illuminants used by DNG PE are A and D65.

Then in ACR I apply that profile to the same input image of the color checker. I expected that the output of the transform would be very close to the values for the patches programmed into the DNG editor. Is that not the case?

They should be visually close, yes, at default ACR settings. (But not numerically close. The readouts in ACR are different from the readouts in DNG PE.)

As I asked earlier, are the values for the patches that the DNG editor is trying to achieve by the profile transform available anywhere? The reason I ask is that when I compare the Lab values in the ACR application in ProPhoto mode to the published values for the color checker, they don't match numerically and are visibly different.

They'll be different numerically & visually from the ideal values because of the default tone curve rendering and black subtraction that ACR applies, which adds contrast and modifies saturation (as Sandy noted, above).  The published ideal values assume no additional rendering is done.  If you want to get a closer match, you would need to turn off the default ACR rendering.  That is done by setting Brightness, Contrast, and Blacks to 0 (instead of 50, 25, and 5) and settings the point curve to Linear instead of Medium Contrast.

[Tim Lookingbill]

If you want to get a closer match, you would need to turn off the default ACR rendering. That is done by setting Brightness, Contrast, and Blacks to 0 (instead of 50, 25, and 5) and settings the point curve to Linear instead of Medium Contrast.

Which is basically what I had to do in the posted samples above to get exact numerical Lab values to the published numbers Bjanes posted and finally measured from shown in his Imatest posting.

Except instead of zeroing out all settings, I had to modify a "linearish" tone curve and set Brightness to +60 in ACR. I built the DNG Profile used in that image sample using ACR defaults embedded in the DNG input image.

[RFPhotography]

How the raw image is converted won't (shouldn't  ) make a difference.

Yes, if you want to try to match values then a zero'd import is a far easier starting point.

If you really want to push ahead with this, then I'd suggest you download some of the synthetic GM24 images I created back when I was doing the calibration work, and first understand what you get in LR/ACR using an "ideal" image. Then go forward with real images.

You can download the synthetic images here: http://sites.google.com/site/chromasoft/referenceimages

There are also spreadsheets of patch values in different color spaces, etc on the same site.

Sandy

Thanks, Sandy.  My thought was that if the export to DNG in LR (or ACR) was applying a tone curve via the default calibration profile, using the DNG Converter may be a way to avoid that as I'd expect the DNG Converter wouldn't have any sort of default tone curve and would just use a linear curve.

Bob, the DNG Profile Editor was designed to be a visual editor.  Specifically, open an image that shows colors to be "off" in some way.  Click on the problem color in the image.  It'll create a control point.  Then you adjust the color till it looks right.  You can place additional control points to change other colors, or simply to "lock down" specific colors to prevent them from changing.

Thanks, Eric.  I understand how the control points work.  It was more a case of determining what to use a a reference point for the CC chart if the profile created from the chart didn't provide a good enough 'match' when applied back in LR and whether it would be possible to use something other than just a visual reference.  Seems not.  

[sandymc]

Thanks, Sandy.  My thought was that if the export to DNG in LR (or ACR) was applying a tone curve via the default calibration profile, using the DNG Converter may be a way to avoid that as I'd expect the DNG Converter wouldn't have any sort of default tone curve and would just use a linear curve.

ACR or LR will by default apply a tone curve to any image that it considers to be "scene referred". That's essentially any raw image, but not e.g., a JPEG or TIFF. This allows ACR/LR to load raws with the tone curve, but to load JPEGs and TIFFs "as created". So the tone curve, etc has to do with the import, not the conversion. Although, just to complicate the situation (a) DNGs can be created as either scene referred or not, and (b) DNG camera profiles can have any tone curve you want embedded in them. But neither of those complications is likely to be relevant in this situation - any DNG converted from a raw image will be tagged as scene referred, and the profile editor (so far as I know anyway) embeds the Adobe standard tone curve unless you use the control points.

Sandy

[RFPhotography]

I don't want to belabour the point, Sandy but the DNG Converter converts RAW files without using LR or ACR.  I'm wondering if that would be a better starting point if that utility doesn't encode a tone curve.

Re (b), understood.  I can modify it as I see fit.  But the 'canned' Adobe profiles have a curve built in, right?  So unless a user edited one of the Adobe profiles to alter the embedded curve that's what would be used.  That goes back to my other question about adding an offsetting curve in the Tone Curve pane of ACR/LR to bring the image back to linear.

Re (a), how would one create a DNG via ACR/LR that wasn't scene-referred?  Or is that done outside of ACR/LR? 

Re the curve used in DNG PE, control points or the Tone Curve Pane?

[sandymc]

I don't want to belabour the point, Sandy but the DNG Converter converts RAW files without using LR or ACR.  I'm wondering if that would be a better starting point if that utility doesn't encode a tone curve.

Re (b), understood.  I can modify it as I see fit.  But the 'canned' Adobe profiles have a curve built in, right?  So unless a user edited one of the Adobe profiles to alter the embedded curve that's what would be used.  That goes back to my other question about adding an offsetting curve in the Tone Curve pane of ACR/LR to bring the image back to linear.

Re (a), how would one create a DNG via ACR/LR that wasn't scene-referred?  Or is that done outside of ACR/LR? 

Re the curve used in DNG PE, control points or the Tone Curve Pane?

I don't think there's any way to create a DNG that isn't either scene referred or has a tone curve baked in already in ACR/LR. Normally you would only create a non-scene referred DNG (one with a "crICCProfilePCS" tag) if you were writing a "linear raw" DNG - aka if you were writing data that was demosaiced/color converted, etc.

Re the curve, I'd assume it could potentially be both, but that's really a question for Eric - Tone curves and tables are somewhat interchangeable. The simple approach would be to just have the tone curve set by the tone curve panel, and the lookup tables by the control points, which would give you nice simple "2D" tables, but I don't know exactly what PE might do under what circumstances.

Sandy

[deejjjaaaa]

I don't want to belabour the point, Sandy but the DNG Converter converts RAW files without using LR or ACR. 

yes, but DNG converter has full image rendering code of ACR/LR included inside (because, as you know, it can create embedded JPG preview w/ all parametric corrections that were done by ACR/LR - so it has to have that code inside).

[RFPhotography]

yes, but DNG converter has full image rendering code of ACR/LR included inside (because, as you know, it can create embedded JPG preview w/ all parametric corrections that were done by ACR/LR - so it has to have that code inside).

That's what I wasn't entirely sure of.  If it contains the full monty of the rendering code then it's a moot point.  That would seem a bit odd since it's not actually rendering the file into a visible form.  The DNG Converter does have the option to select Linear (demosaiced), so perhaps that's the answer. 

I know that the converter can create a JPEG preview.  But that doesn't have to be based on the rendering engine of LR/ACR.  Cameras create a JPEG preview as well based on the in-camera chosen colour space.  The DNG Converter could simply do the same based on, say, sRGB but that doesn't necessarily have to impact the conversion of the Raw file to a DNG, does it?  Do cameras that record in DNG have an Adobe tone curve applied?  I'd think probably not. 

[deejjjaaaa]

That would seem a bit odd since it's not actually rendering the file into a visible form.

I am not talking about user interface - I am talking about the code that does demosaicking, NR, etc

  But that doesn't have to be based on the rendering engine of LR/ACR

it does - try it yourself - store parametic edits (done w/ ACR or LR) in DNG file itself and then instruct DNG converter to update JPG preview inside (overwriting the original preview)... then open it in a 3rd party viewer that can display embedded JPG itself.

[RFPhotography]

I am not talking about user interface

  Neither am I.

it does - try it yourself - store parametic edits (done w/ ACR or LR) in DNG file itself and then instruct DNG converter to update JPG preview inside (overwriting the original preview)... then open it in a 3rd party viewer that can display embedded JPG itself.

I wasn't saying it wasn't done that way.  I was saying it didn't necessarily have to be done that way.  You're talking about edits made in LR/ACR.  I can see how those may get picked up.  I'm talking about an image that never sees LR/ACR but is loaded into the DNG Converter straight from the memory card or hard drive.  If it still applies a default tone curve, then so be it.

[deejjjaaaa]

  I'm talking about an image that never sees LR/ACR but is loaded into the DNG Converter straight from the memory card or hard drive.  If it still applies a default tone curve, then so be it.

I was only making the point that Adobe DNG Converter in fact has the full ACR/LR raw conversion code (minus UI) in it (that is also one of the reasons why it can't be open sourced) and so it can applyto raw file everything that ACR/LR can apply ("never sees LR/ACR" = open in ACR/LR w/ the default settings from Adobe "as installed")

[RFPhotography]

I was only making the point that Adobe DNG Converter in fact has the full ACR/LR raw conversion code (minus UI) in it (that is also one of the reasons why it can't be open sourced) and so it can applyto raw file everything that ACR/LR can apply ("never sees LR/ACR" = open in ACR/LR w/ the default settings from Adobe "as installed")

And that's fine.  If that's the way it works then so be it, there's no advantage.  I know the converter exists but don't use it so wasn't sure how it was set up. 

[ComputerDork]

This thread is pretty valuable so I'm posting just so I can get notifications. (I have a ColorChecker Passport and have seriously wondered about all this stuff.)

[sandymc]

yes, but DNG converter has full image rendering code of ACR/LR included inside (because, as you know, it can create embedded JPG preview w/ all parametric corrections that were done by ACR/LR - so it has to have that code inside).

Actually, not. My understanding is that while DNG converter is smart enough to understand all the raw formats that ACR/LR understands, and all of the color rendering and lens correction that can be encoded into a DNG camera profile, the demosaicing, noise reduction, etc algorithms are not as advanced as ACR/LR - they're just good enough to be able to create thumbnails and previews. So you can't in effect use the DNG converter as a no-cost substitute for LR/ACR

Sandy