Luminous Landscape Forum
Raw & Post Processing, Printing => Colour Management => Topic started by: RDoc on November 23, 2011, 02:31:37 pm
-
What I'm attempting to do is photograph paintings to produce accurate prints. As part of this I'm trying to get an accurate input profile.
I photographed the Color Checker Passport with the histogram centered in RAW using Adobe color, then converted it to a DNG with Photoshop Raw. Then with Adobe DNG Editor made some profiles using both Linear and Base Profile tone curves and no other corrections. I then used those profiles on the original RAW image of the Color Checker with no corrections. The tone cure is clearly incorrect, so I tweaked it and the exposure to get the gray patches to match the published values. Incidentally, it seems very strange to me that the DNG Editor doesn't at least have an option to match the tone curve to the color patches.
The resulting color patches are quite close in both Adobe and ProPhoto, but certainly not exact and I'm wondering why. It seems to me that if I correct the Color Patch photo with the photo itself, the results should exactly match the values. Shouldn't the software force the colors to match? If not what is it trying to do?
I did the same tests using the XRite software with similar but somewhat worse results.
-
Did you manually adjust the color corrections in the DNG Profile Editor? (The auto Chart Wizard feature may not be using the same reference values as the ones you're using.)
-
Did you manually adjust the color corrections in the DNG Profile Editor? (The auto Chart Wizard feature may not be using the same reference values as the ones you're using.)
The values I'm using are the ones at:
http://www.brucelindbloom.com/downloads/ColorCheckerSpreadsheets.zip
I only edited the exposure and tone curve in the Raw editor so the bottom row of grays match the published values. Unless I'm really missing something in the UI, I don't see how it would be possible to manually match all the points in the color checker because of the way the controls and value readouts work. I've never seen a set of values in the same space the DNG editor uses. Is there a set of its patch values available?
Also, is there an easy way to match the tone curve to the gray values in the Color Checker? I did it by manually adjusting a curve with about 6 points to get the values correct and still have a reasonably smooth curve, but that's not very rigorous.
-
Have you tried the software from Xrite instead of the adobe one?
http://xritephoto.com/ph_product_overview.aspx?action=support&id=1257
Keith
-
Lindbloom's reference numbers are different from XRite's, http://xritephoto.com/ph_product_overview.aspx?ID=1257&Action=Support&SupportID=5159&catid=28.
If I look at RGB values for various patches in ACR then export that image as a dng and open it in Adobe's DNG Profile Editor, the numbers are way off. Is there something happening in the Adobe application that's causing the issue?
-
Lindbloom's reference numbers are different from XRite's, http://xritephoto.com/ph_product_overview.aspx?ID=1257&Action=Support&SupportID=5159&catid=28.
If I look at RGB values for various patches in ACR then export that image as a dng and open it in Adobe's DNG Profile Editor, the numbers are way off. Is there something happening in the Adobe application that's causing the issue?
The RGB/Lab numbers, if I understand you correctly, as read from the DNG Profile Editor are suppose to be way off because they are read from a linear reference space.
If you're using Mac OS X you can use its DigitalColor Meter set to Lab to get the numbers according to the DNG Profile Editor preview.
But to be honest I've never had any trouble making my $500 Pentax K100D DSLR give me exactly (I mean EXACTLY) what my eyes see editing in ACR if I happen to have the subject captured right next to my display or at least close by to compare against.
I've made several different DNG profiles with the CCchart shot under various lighting and then apply each different profile to one image not shot under those lights and there's barely a difference in color shifts.
I find the most difficult colors are the ones whose weird spectral reflectance makeup gives a color that's way off from the original where it's easier and faster to fix it with ACR's HSL panel.
Paint pigments may or may not exhibit these weird spectral colors that fool the camera's sensor, but I can assure you a CCchart derived profile will only get you close to what you see overall but it's not going to fix those kind of color errors.
-
Lindbloom's reference numbers are different from XRite's, http://xritephoto.com/ph_product_overview.aspx?ID=1257&Action=Support&SupportID=5159&catid=28.
If I look at RGB values for various patches in ACR then export that image as a dng and open it in Adobe's DNG Profile Editor, the numbers are way off. Is there something happening in the Adobe application that's causing the issue?
Probably the best reference for RGB coordinates for the ColorChecker is that provided by Danny Pascale (http://www.babelcolor.com/download/RGB%20Coordinates%20of%20the%20Macbeth%20ColorChecker.pdf). Values from different sources may vary according to the illuminant (1976 MacBeth values were for Illuminant C), the target color space, whether the data are RGB or R'G'B' (the latter are gamma encoded), and the method used for chromatic adaption. The Bradford adaption is less accurate than the value obtained by using the spectral reflectance of the patches, the SPD of the illuminant, and the 2 degree standard observer when one is using D65 spaces. ProPhotoRGB is less problematic than Adobe RGB or sRGB, because the latter two are D65 and require chromatic adaption from the L*a*b D50 whereas ProPhoto is already in D50.
Danny gives the Delta E for various measurements of current ColorCheckers and spaces, and they are relatively small for charts produced after 2005. For practical work, the published values are sufficiently accurate and one does not need a spectrophotometer to measure the patches.
Regards,
Bill
-
I only edited the exposure and tone curve in the Raw editor so the bottom row of grays match the published values. Unless I'm really missing something in the UI, I don't see how it would be possible to manually match all the points in the color checker because of the way the controls and value readouts work. I've never seen a set of values in the same space the DNG editor uses. Is there a set of its patch values available?
Also, is there an easy way to match the tone curve to the gray values in the Color Checker? I did it by manually adjusting a curve with about 6 points to get the values correct and still have a reasonably smooth curve, but that's not very rigorous.
The DNG Profile Editor ignores the tone curve established in the raw converter (ACR I presume). It does use the white balance set by ACR and stored in the DNG file, but when calculating the color table, it uses its own white balance. This can be shown by setting a very wild WB in ACR and using a dramatic tone curve.
Here is the ACR screen capture with a wild WB and dramatic tone curve:
(http://bjanes.smugmug.com/Photography/WhiteBalance/i-4Ws5WhS/0/O/07WildEdit.png)
When one opens the DNG in the Profile Editor, the tone curve is ignored but the WB is preserved:
(http://bjanes.smugmug.com/Photography/WhiteBalance/i-L8MMMRm/0/L/07WildEditDNGEd-L.png)
When the color tables are calculated, the WB is adjusted by the profile editor:
(http://bjanes.smugmug.com/Photography/WhiteBalance/i-TGWQgq8/0/L/WildEditColorTable-L.png)
You may be going about your work in the wrong way. I think you might try to use the profile editor to get accurate colors and then establish a custom tone curve in ACR to render the neutral patches with the proper luminances. Hopefully, Eric can comment.
Regards,
Bill
-
The values I'm using are the ones at:
http://www.brucelindbloom.com/downloads/ColorCheckerSpreadsheets.zip
I only edited the exposure and tone curve in the Raw editor so the bottom row of grays match the published values. Unless I'm really missing something in the UI, I don't see how it would be possible to manually match all the points in the color checker because of the way the controls and value readouts work. I've never seen a set of values in the same space the DNG editor uses. Is there a set of its patch values available?
Also, is there an easy way to match the tone curve to the gray values in the Color Checker? I did it by manually adjusting a curve with about 6 points to get the values correct and still have a reasonably smooth curve, but that's not very rigorous.
I'm a bit surprised that you're seeing a major difference in tone curve. But anyway, I did a bunch of work on calibration of various raw developers versus a GM 24 chart a few years ago that might be useful to you: http://chromasoft.blogspot.com/2008/01/lightroom-aperture-and-capture-one-mini_24.html
Sandy
-
I'm a bit surprised that you're seeing a major difference in tone curve. But anyway, I did a bunch of work on calibration of various raw developers versus a GM 24 chart a few years ago that might be useful to you: http://chromasoft.blogspot.com/2008/01/lightroom-aperture-and-capture-one-mini_24.html
Sandy
Sandy,
I seem to get the best match to the nominal color checker values by using a linear tone curve in ACR. Here is the result using a Passport generated profile for my camera using a linear tone curve. Since most tone curves have an inflection point near the midtones, I adjusted exposure to give the desired value for gray patch 3 of the color checker.
(http://bjanes.smugmug.com/Photography/WhiteBalance/i-2rJRBjP/0/O/PassportACRLinear.png)
I looked at your referenced web page, but could only see page 1. Where are the rest?
Regards,
Bill
-
I looked at your referenced web page, but could only see page 1. Where are the rest?
Bill, they're separate posts. here's the whole series:
Part 1: http://chromasoft.blogspot.com/2008/01/lightroom-aperture-and-capture-one-mini_24.html
Part 2: http://chromasoft.blogspot.com/2008/01/acr-aperture-capture-one-dng-lightroom.html
Part 3: http://chromasoft.blogspot.com/2008/02/lightroom-aperture-and-capture-one-mini.html
Part 4: http://chromasoft.blogspot.com/2008/02/lightroom-aperture-and-capture-one-mini_17.html
Part 5: http://chromasoft.blogspot.com/2008/02/lightroom-aperture-and-capture-one-mini_23.html
Part 6: http://chromasoft.blogspot.com/2008/03/lightroom-aperture-and-capture-one-mini.html
Sandy
-
You may be going about your work in the wrong way. I think you might try to use the profile editor to get accurate colors and then establish a custom tone curve in ACR to render the neutral patches with the proper luminances. Hopefully, Eric can comment.
Yes, as noted in the OP that's exactly what I was trying to do.
- I photograph the passport image in even lighting, Adobe RGB color
- open it in ACR
- save it as a DNG
- use the DNG editor to create a profile with no edits
- apply that profile to the original image in ACR
- create a tone cure in ACR to match the values from the XRite site
- open the result in Lab mode in CS5 using Adobe RGB and look at the patches
Essentially I'm using the DNG editor to build a profile, then applying that profile back on the same image used to generate the profile which I would think would result in an accurate color result even if the tone curve was wrong.
The result is acceptable in most cases, delta E of around 3-6 compared to the XRite published values but several are over 10 off even if I force the L values to be the same as the published values.
I did try all this using the XRite software as well and the results were generally better although there were still errors > 9 in patches different than the ones with large errors from the DNG editor.
Overall it does look like the biggest errors may be the DNG editor using patch values that aren't the same as the XRite Passport but there is still some kind of residual error in both. Anyway, if the results in values are noticeably off, perhaps there needs to be a way to tell the DNG editor what target it's looking at.
Certainly for many uses, accuracy isn't really what's wanted, but that's not the case for some applications, and as it stands I don't see how one can generate an accurate profile with either tool. There's no editing in the XRite software, and because of the way the UI works in the DNG editor I don't see how it would be at all reasonable to correct all the patches by hand.
-
- I photograph the passport image in even lighting, Adobe RGB color
- open it in ACR
- save it as a DNG
- use the DNG editor to create a profile with no edits
- apply that profile to the original image in ACR
- create a tone cure in ACR to match the values from the XRite site
- open the result in Lab mode in CS5 using Adobe RGB and look at the patches
That is what I have been doing as well, except it is best to use ProPhotoRGB as the working space, since Photoshop would have to use the Bradford transformation to convert from the D65 of Adobe RGB to the D50 of L*a*b. A bit depth of 16 would help to prevent rounding errors. According to Danny Pascale (http://www.babelcolor.com/download/RGB%20Coordinates%20of%20the%20Macbeth%20ColorChecker.pdf), the Bradford transform produces an average ΔE of 1.4.
Essentially I'm using the DNG editor to build a profile, then applying that profile back on the same image used to generate the profile which I would think would result in an accurate color result even if the tone curve was wrong.
The result is acceptable in most cases, delta E of around 3-6 compared to the XRite published values but several are over 10 off even if I force the L values to be the same as the published values.
Since luminance is included in ΔE, failure in controlling luminance will cause ΔE to increase. BTW, how are you calculating ΔE? The CIED2000 is said to be the most accurate: i.e. correlates with observed differences in color. The various formulas are discussed by Norman Koren in the Colorcheck (http://www.imatest.com/docs/colorcheck_ref.html) documentation. I find that program to be invaluable in evaluating the accuracy of the profiles (the DigitalDog would cringe on reading this ;D). The CIED2000 usually gives smaller ΔEs and makes the data look better as well as better correlating with observed differences (which I have not checked).
Here is the best that I can do with my D3 and the DNG profiler. The results might have been better had I constructed a dual illuminant profile for my tests which used Solux illumination which is about 4700K. The Passport results are slightly worse.
(http://bjanes.smugmug.com/Photography/Calibration/i-ZvmsQfp/0/O/090904Macbeth0007DNGLumCorrcol.png)
(http://bjanes.smugmug.com/Photography/Calibration/i-qgJGnLc/0/O/090904Macbeth0007PassportLumCo.png)
I did try all this using the XRite software as well and the results were generally better although there were still errors > 9 in patches different than the ones with large errors from the DNG editor.
Overall it does look like the biggest errors may be the DNG editor using patch values that aren't the same as the XRite Passport but there is still some kind of residual error in both. Anyway, if the results in values are noticeably off, perhaps there needs to be a way to tell the DNG editor what target it's looking at.
Certainly for many uses, accuracy isn't really what's wanted, but that's not the case for some applications, and as it stands I don't see how one can generate an accurate profile with either tool. There's no editing in the XRite software, and because of the way the UI works in the DNG editor I don't see how it would be at all reasonable to correct all the patches by hand.
The Colorchecker probably is not introducing that much of an error unless it is old or improperly stored. XRite recommends replacing the chart every 2 years. Here are the ΔEs that Danny Pascale found in evaluating 20 charts.
(http://bjanes.smugmug.com/Photography/Calibration/i-KL646PF/0/O/ColorCheckerDeltaEs.png)
One must realize that perfectly accurate color is not possible with current digital cameras, since the respond differently than the human visual system to the various colors. In constructing a 3x3 matrix, the profile maker tries to minimize error, perhaps using a least squares approach, or perhaps favoring accuracy in various memory colors such as blue sky, foliage, human skin, etc. A landscape profile would be different than one optimized for portraits.
It would be interesting to learn what other users were obtaining with the DNG profiler and the Passport software.
If you need accurate reproduction under controlled conditions, it might best to use the Colorchecker SG (which uses many more patches) and make an ICC profile. This would entail considerable expense and you would have to use a raw converter (such as Capture One) that uses ICC profiles.
Regards,
Bill
-
The resulting color patches are quite close in both Adobe and ProPhoto, but certainly not exact and I'm wondering why.
You should also ask yourself, IF (big if) you got those 24 colors to be 'correct' (or what some like to call accurate, then tell you it can't be "perfectly" accurate), what would the effect be on all the other millions of solid colors that make up your painting.
If the painting is a Macbeth, you seem to be in decent shape <g>.
-
You should also ask yourself, IF (big if) you got those 24 colors to be 'correct' (or what some like to call accurate, then tell you it can't be "perfectly" accurate), what would the effect be on all the other millions of solid colors that make up your painting.
Andrew, I don't know what is the source of your hangup with "accurate". Correct (http://www.merriam-webster.com/dictionary/correct) is a more general term when used as an adjective. Accuracy (http://en.wikipedia.org/wiki/Accuracy_and_precision) is defined as follows: "In the fields of science, engineering, industry and statistics, the accuracy of a measurement system is the degree of closeness of measurements of a quantity to that quantity's actual (true) value." If I calculate a ΔE for the green patch of the ColorChecker as rendered by my camera with a given profile and raw converter, I am measuring accuracy by comparing the observed value with the correct value as per the x-rite spec or an actual measured value from a laboratory grade spectrophotometer. Do you agree? Accuracy implies a degree of error that can be quantified, whereas correct does not, at lease as I understand the definitions.
As an authority in color management, how would you recommend that the OP calibrate his camera to obtain correct results when photographing artwork? I understand that museum professionals tend to use multishot backs with high color rendering index illumination and calibrate according to laboratory standards. If one lacks such a setup, what is the best way to obtain the best results with a dSLR?
Regards,
Bill
-
The problem with using reference numbers is that they're based on an assumed illuminant (e.g., D50). So even if you average a bunch of chart values, that still doesn't account for the illuminant. It's very unlikely that you photographed under illuminant D50, since it doesn't actually exist! ;) So, you photographed under some other (real-world) illuminant, which results in different actual values for the chart. Put another way, the spectral radiances coming off the chart patches and into the camera in your test shot are not the same as the ones used to derive the reference numbers that you see posted on various sites ...
-
The problem with using reference numbers is that they're based on an assumed illuminant (e.g., D50). So even if you average a bunch of chart values, that still doesn't account for the illuminant. It's very unlikely that you photographed under illuminant D50, since it doesn't actually exist! ;) So, you photographed under some other (real-world) illuminant, which results in different actual values for the chart. Put another way, the spectral radiances coming off the chart patches and into the camera in your test shot are not the same as the ones used to derive the reference numbers that you see posted on various sites ...
Yes, I understand that, but what I'm still not getting is why creating a profile using a color chart image, then applying that profile to the same image doesn't result in the values the software thinks the chart patches have. Isn't the software essentially building a transform that converts the input image data to the ideal color patch data? If so, at least for the color components, I'd expect that if I apply that transform to the data used to build the transform I'd get the ideal values.
Perhaps what I'm seeing are the DNG editor's ideal values? Is there a published set of values for the patches that the DNG editor uses?
-
The problem with using reference numbers is that they're based on an assumed illuminant (e.g., D50). So even if you average a bunch of chart values, that still doesn't account for the illuminant. It's very unlikely that you photographed under illuminant D50, since it doesn't actually exist! ;) So, you photographed under some other (real-world) illuminant, which results in different actual values for the chart. Put another way, the spectral radiances coming off the chart patches and into the camera in your test shot are not the same as the ones used to derive the reference numbers that you see posted on various sites ...
Eric, quite true. But one can use chromatic adaption as discussed in Section 5 of Danny Pascale's RGB Coordinates of the Macbeth Color Checker (http://www.babelcolor.com/download/RGB%20Coordinates%20of%20the%20Macbeth%20ColorChecker.pdf). One can also use chromatic adaption to derive R'G'B' values for other illuminants. X-Rite publishes values for D50 CIE L*a*b and D65 sRGB. Danny and Bruce Lindbloom give values for multiple illuminants. The other factor is the actual illuminant used to take the picture. In my case, I use Solux 4700K bulbs which are not that far from D50 and rely on Camera Raw to perform a transform from the camera XYZ data using white balance and whatever else to ProPhotoRGB D50. In any case, one can compare the observed rendered values under the employed illuminant and compare them to the published or measured values of the chart and obtain a measure of the calibration of the system--illuminant, chart, camera, and raw converter. Your comments would be appreciated.
Of one had the SPD of the Solux bulbs and the spectral reflectance properties of the chart, one could calculate the D50 L*a*b values directly using the 2 degree standard observer data, but this seems to be overkill for most purposes.
Regards,
Bill
-
Yes, I understand that, but what I'm still not getting is why creating a profile using a color chart image, then applying that profile to the same image doesn't result in the values the software thinks the chart patches have. Isn't the software essentially building a transform that converts the input image data to the ideal color patch data? If so, at least for the color components, I'd expect that if I apply that transform to the data used to build the transform I'd get the ideal values.
Perhaps what I'm seeing are the DNG editor's ideal values? Is there a published set of values for the patches that the DNG editor uses?
The numbers for that chart are derived from a machine (spectrophotometer) measuring the spectral response of each color, gray and black patches under a given amount of light and color temperature illuminant as Eric indicated. What that spectro "saw" is by the numbers and doesn't take into account human's adaptive nature to brightness, contrast and saturation of colors to any given scene viewed.
This is much like the affects on perception when the lights are turned on in a darkened movie theatre where our eyes immediately see a loss in contrast and richness. A spectro if it were possible to measure from the movie screen would still see the CCchart color patches as being the same because the spectro's is using its own light source and not the movie theatre's.
The default ACR settings which establishes contrast, saturation and brightness tries to normalize the appearance as a human would perceive it, not as a spectro would. You will note when adding contrast editing an image in Photoshop or ACR/LR, saturation is greatly affected while an attempt is made by the tools in the software to maintain hue which is all the profile can possibly control due to the non-linear nature of how humans perceive any given scene along with the non-linear nature of camera sensor.
The image samples below I took with my Pentax K100D DSLR in an attempt to get the gray and black patch readouts to measure as close as possible to the published Lab numbers using a curve adjust. My illuminant and light source is direct sunlight for all shots. You can measure yourself how close I got. The last image shows the profile applied along with the settings that gave exact Lab numbers to a real scene captured under the same light intensity. Note the lack of contrast of the overall appearance of the surrounding scene. This is the same effect that happens when the lights are turned on in a darkened theatre.
I can tell you for sure my eyes saw that scene as much brighter and full of contrast than what's depicted.
-
You should also ask yourself, IF (big if) you got those 24 colors to be 'correct' (or what some like to call accurate, then tell you it can't be "perfectly" accurate), what would the effect be on all the other millions of solid colors that make up your painting.
My impression is that if your profile can reproduce these 24 colors to match published values then in theory other colors would fall more or less into place within an acceptable range. Isn't that the whole point of jumping through the hoop of making custom DNG profiles? If the only advantage of custom profiles is that you can accurately reproduce a macbeth chart (and even that seems questionable at this point) why would anyone do it? Why not just use a grey card and go from there?
While I generally opt for pleasing color, in this case it's quite easy to talk about accurate color. You have two sets of patches with known LAB values that you are comparing. This should be the one place where talking about accurate color makes perfect sense.
Also, all the talk of different spectral distributions is a bit beside the point. We're not trying to match spectral reflectance—I think we'd all be happy with a metameric match.
One question that comes to mind: are DNG profiles matrix-based? If so, it's certainly understandable that it would not be able to bring all the patches into line. If it's got a lookup-table though, I would expect the individual patches to match almost by definition.
-
Wonder what light source they used to develop and define the "Standard Observer" when coming up with the Lab color model?
Was that D50? And how bright was it? 100cd/m2? 120 cd/m2?
Does the "Standard Observer" and Lab color space model know how bright my monitor and ambient light are?
I really don't see how exact precise numbers can be maintained with all the variances involved and demonstrated reproducing an image, but considering all the billions of images online, we seem to do quite well.
-
Wonder what light source they used to develop and define the "Standard Observer" when coming up with the Lab color model?
Was that D50? And how bright was it? 100cd/m2? 120 cd/m2?
Does the "Standard Observer" and Lab color space model know how bright my monitor and ambient light are?
The CIE D illuminants like D50 did not yet exist when the standard observer color matching functions were made.
The standard observer data is based on two sets of experiments. One from John Guild and one from David Wright. Guild's experiments matched monochromatic sources to trichromatic primaries from filtered tungsten light (2900K). Wright's experiments matched the monochromatic sources to monochromatic primaries at 650, 530, and 460 nanometers. Since they are comparing emissive sources, the illuminant is not really relevant.
How the CIE went from Guild & Wright's data to 'modern' colorimetry is an interesting story which can be read in this PDF: http://www.cis.rit.edu/research/mcsl2/research/broadbent/CIE1931_RGB.pdf
LAB space, of course, came much later—1976.
-
The CIE D illuminants like D50 did not yet exist when the standard observer color matching functions were made.
The standard observer data is based on two sets of experiments. One from John Guild and one from David Wright. Guild's experiments matched monochromatic sources to trichromatic primaries from filtered tungsten light (2900K). Wright's experiments matched the monochromatic sources to monochromatic primaries at 650, 530, and 460 nanometers. Since they are comparing emissive sources, the illuminant is not really relevant.
How the CIE went from Guild & Wright's data to 'modern' colorimetry is an interesting story which can be read in this PDF: http://www.cis.rit.edu/research/mcsl2/research/broadbent/CIE1931_RGB.pdf
LAB space, of course, came much later—1976.
Oh, yeah, I got the "Standard Observer" mixed up with Lab space. How could I have done that? Now it's clear as mud. I really don't know how the hell we got this far with this much confusion.
Oh great! Another damn white paper to make it all clear as mud.
-
Oh great! Another damn white paper to make it all clear as mud.
;D
As far as I know, the current DNG spec can handle either matrix or LUT profiles. Which type either the DNG Profile Editor or the XRite Passport software create is unknown.
It seems to make sense that the XRite software is assuming a D65 source since the published values are for the sRGB space. If it's doing a transform from one illuminant (the shooting condition) to D65, that will introduce errors as has been pointed out. But what is it assuming as the shooting condition to do that transform? The WB of the image? If so, then would using an image to create the profile that isn't properly white balanced (i.e., against one of the neutral patches) potentially create a larger error factor?
Using these profiles as anything other than a decent starting point is really expecting too much. 24 patches isn't a lot. To really get something even close to 'accuracy' would require a much larger number of patches. The Colormunki reads 100 patches, if memory serves. Other profile creation tools more than that. Even then, profiles aren't perfect.
-
One question that comes to mind: are DNG profiles matrix-based? If so, it's certainly understandable that it would not be able to bring all the patches into line. If it's got a lookup-table though, I would expect the individual patches to match almost by definition.
The answer is both. See the Adobe documentation here (http://labs.adobe.com/wiki/index.php/DNG_Profiles:Editor#color_matrices_pane).
Regards,
Bill
-
As far as I know, the current DNG spec can handle either matrix or LUT profiles. Which type either the DNG Profile Editor or the XRite Passport software create is unknown.
The DNG profile editor can create both matrix and LUT profiles as per the Adobe tutorial (http://labs.adobe.com/wiki/index.php/DNG_Profiles:Editor#color_matrices_pane).
It seems to make sense that the XRite software is assuming a D65 source since the published values are for the sRGB space. If it's doing a transform from one illuminant (the shooting condition) to D65, that will introduce errors as has been pointed out. But what is it assuming as the shooting condition to do that transform? The WB of the image? If so, then would using an image to create the profile that isn't properly white balanced (i.e., against one of the neutral patches) potentially create a larger error factor?
Actually, the X-rite spec publishes values for both D65 sRGB and D50 L*a*b. One can use the Bradford chromatic adaption model to convert from one space to another with a different illuminant, and the average error for converting from D50 to D65 is about 1.4 Delta E (Danny Pascale (http://www.babelcolor.com/download/RGB%20Coordinates%20of%20the%20Macbeth%20ColorChecker.pdf)). The question arises, does this take into account the illuminant used to take the picture or merely the transformation from a D50 to D65 space when the illuminant used to take the picture was D50?
Regards,
Bill
-
My impression is that if your profile can reproduce these 24 colors to match published values then in theory other colors would fall more or less into place within an acceptable range.
More or less, maybe yes, maybe no. But bigger issue is the assumption that nailing 24 colors now in some way produces a profile or process that creates ideal, push button color. I don't know anyone working in reproduction that has found, a profile (DNG or ICC) produces this effect solely using a profile. Depending on the original, even in a fixed and controlled studio condition, there's a bit more work involved to produce a visual match.
Isn't that the whole point of jumping through the hoop of making custom DNG profiles? If the only advantage of custom profiles is that you can accurately reproduce a macbeth chart (and even that seems questionable at this point) why would anyone do it? Why not just use a grey card and go from there?
The point is to get closer to your goal. There's no question that good profiles do this and there's no question in my mind that the don't acheive any guarantees of all other colors becoming visually acceptable or "accurate".
Also, all the talk of different spectral distributions is a bit beside the point. We're not trying to match spectral reflectance—I think we'd all be happy with a metameric match.
Its not beside the point when we actually have to prove something like accurate color reproduction. Or prove the process isn't partially subjective in terms of a visual match (since the numbers don't produce a match in your example). Numbers that should provide a match don't and numbers that do provide a visual match should not (the beauty of a metameric match).
The OP asks why the numbers are not working completely. Eric beautifully described why. My initial point was, just producing a visual match of super 24 patches is useful if you want to match 24 solid colors. It doesn't guarantee all other colors will visually match (and at this point, the OP still needs to convert to an output space and print the data I assume. That too will complicate the success of the match of other colors).
Profiles ARE useful. But lets not put too much faith in what they provide in something as complex as are reproduction, a difficult process.
-
The DNG profile editor can create both matrix and LUT profiles as per the Adobe tutorial (http://labs.adobe.com/wiki/index.php/DNG_Profiles:Editor#color_matrices_pane).
Fine. That leaves the XRite then.
Actually, the X-rite spec publishes values for both D65 sRGB and D50 L*a*b. One can use the Bradford chromatic adaption model to convert from one space to another with a different illuminant, and the average error for converting from D50 to D65 is about 1.4 Delta E (Danny Pascale (http://www.babelcolor.com/download/RGB%20Coordinates%20of%20the%20Macbeth%20ColorChecker.pdf)). The question arises, does this take into account the illuminant used to take the picture or merely the transformation from a D50 to D65 space when the illuminant used to take the picture was D50?
Regards,
Bill
Yes, that's right, it does. I knew that since I posted it earlier. Forgot. Re: the highlighted section, that's basically the question I was posing. Is the illuminant of the actual image taken into account, if so how and how is the transformation being done from that to 6500 or 2850 (I don't say 5000 because 5000 isn't an option for creating the profile). Or does it simply assume the image was taken under one of the two lighting conditions that can be used to create the profile (6500 or 2850)? Either way, the potential error factor gets larger, doesn't it?
-
One question that comes to mind: are DNG profiles matrix-based? If so, it's certainly understandable that it would not be able to bring all the patches into line. If it's got a lookup-table though, I would expect the individual patches to match almost by definition.
DNG profiles must have a matrix, and can optionally have tables. So far as I know however (Eric may correct me on this), the Profile Editor does not use tables for color matching.
Sandy
-
I am not very experienced in this. I also take images of original paintings and sometimes have problems with accurate colours. If I open an image in ACR then it seems to me there is a potential problem in that when you save the file in DNG and then go to Color passport you may inadvertantly have changed the image.
There are a lot of defaults in ACR and it is easy to miss turning all of them off. If one is on and you then convert to DNG and then go to passport you are not working with the original camera colors.....
The recommended procedure I was given was to convert to DNG using the Adobe DNG Converter. Then Drop the image in to Color passport..
I might be wrong. But it is so easy to mess this up by missing a step..
cheers elo
-
See this thread on Color checker Passport..
http://forums.dpreview.com/forums/readflat.asp?forum=1006&message=38035883&changemode=1
this is not what I was told by .... the Experts? at the focus on imaging exhibition where I bought the package...
will do some more research.. ( this may be why I have the occasional problem)
my actual procedure for camera calibration is
1. take a bracketed image of the color checker
2. take all images necessary of work
3. Convert all to dng using the adobe color checker
4. then open in ACR check the color checker image that is best.... and drop the converted image(original) to the passport program...
This procedure is wrong based on the thread at dpreview
I always use the white balance patch.....
more work.....
cheers elo
-
The numbers for that chart are derived from a machine (spectrophotometer) measuring the spectral response of each color, gray and black patches under a given amount of light and color temperature illuminant as Eric indicated. What that spectro "saw" is by the numbers and doesn't take into account human's adaptive nature to brightness, contrast and saturation of colors to any given scene viewed.
If you want the scene as seen by the spectrophotometer to correlate with the photographic image, you will have to adjust the viewing conditions such that the adaption of the human visual system is the same for viewing the image as for the original scene. I downloaded your corrected image and looked at it with Imatest Colorcheck. The DeltaEs are small, indicating that your profile did quite a good job or reproducing the chart. When comparing your image on my calibrated monitor to my own ColorChecker, I see a good match even though my monitor is 6500K and my Solux viewing lamp is 4700K. My visual system can make the necessary accommodation even for this mismatch of color temperature.
(http://bjanes.smugmug.com/Photography/Calibration/i-h5wwJLG/0/O/CCchartColorMeterLabEditBcolor.png)
This is much like the affects on perception when the lights are turned on in a darkened movie theatre where our eyes immediately see a loss in contrast and richness. A spectro if it were possible to measure from the movie screen would still see the CCchart color patches as being the same because the spectro's is using its own light source and not the movie theatre's.
I don't think that is true. To measure the screen with a spectrophotometer, you would have to have an instrument that reads the actual luminance of the screen, not a reflection instrument that uses its own light source. In other words, you would need a radiance pixmap of the scene. When you turn on the theater lights, you not only change color and brightness adaption of the viewer, but the theater lights dilute the luminances on the screen, and this is most prominent in the shadows, since the luminance of the theater lights is added to the values produced by the projector, and the effect is most marked in the shadows, where the luminance produced by the theater lights is much greater than the shadow luminances produced by the projector. The effect is analogous to flare light, which washes out the shadows more than the highlights. If you wanted the image to appear good with the lights on, you would have to greatly increase the luminance of the projector so that the projected shadow luminances would not be overwhelmed by the theater lights.
If I am looking at a reflection print and increase the ambient illumination, the highlight and shadow values are increased linearly according to the change in illumination.
The image samples below I took with my Pentax K100D DSLR in an attempt to get the gray and black patch readouts to measure as close as possible to the published Lab numbers using a curve adjust. My illuminant and light source is direct sunlight for all shots. You can measure yourself how close I got. The last image shows the profile applied along with the settings that gave exact Lab numbers to a real scene captured under the same light intensity. Note the lack of contrast of the overall appearance of the surrounding scene. This is the same effect that happens when the lights are turned on in a darkened theatre.
I can tell you for sure my eyes saw that scene as much brighter and full of contrast than what's depicted.
The color checker is a low contrast scene and the luminances are easily within the range of the monitor and even a print, so the scene can be rendered without any luminance compression. However, an outdoor scene has much greater contrast and a linear rendering looks flat, and a sigmoid tone curve can help fit the luminance of the scene to that of the output medium. For a HDR scene, a global tone curve fails and local adjustments are necessary. This is the difference between scene rendering and output rendering. If our monitors could reproduce the actual luminances in the scene, no luminance compression would be necessary and we could use the scene referred image directly.
Regards,
Bill
-
Bill, thanks for going through the trouble of verifying the numbers. Nice to know my DigitalColor Meter and X-rite i1Display monitor profile isn't too far off.
Let me be clear the profile didn't do all the work to get to the numbers you measured on the last image that includes the blue doll and surrounding scene. ACR settings were all zero'ed out except for Color Temp=5000K/Tint=0, Brightness +60 and Black 1 and the adjusted curve shown.
As for the "dark theatre with the lights turned on" reference I was loosely using that to illustrate the differences between human vision and colorimetric accuracy according to spectro measured Lab numbers in the CCchart. As another analogy from my understanding, if a spectro is used to measure and profile a display in a dimly lit studio without ambient light taken into account and that calibrated/profiled display is later placed outdoors in broad daylight displaying the same CCchart, the appearance according to human perception of the display would be greatly affected but the Lab numbers would still be the same. This is what I meant.
Just to add, the blue doll/CCchart image you measured from to get the accurate Delta E numbers can't be edited to look as it did to the original scene without changing the accuracy of the CCchart Lab numbers. I've tried it and the only edit I could add was shadow definition shown in the image below. So a simple sigmoid curve applied would not work and would throw off the CCchart Lab numbers in doing so.
But really what I was trying to point out to the OP is why it's difficult to get exact CCchart numbers and still have the image look according to human vision. Spectro's aren't human.
But I can tell you the profile works quite well in my experience maintaining balance across a wide range of colors within a wide range of images of various dynamic range when trying to edit the image to get it to look according to human vision. That's really all that can be expected from a profile since we can't calibrate and profile reality which is what the digital camera is trying to capture.
-
This is how the CCchart looked according to how I perceived the overall scene. Note the simple edits applied going from Brightness +60 to +90 and Contrast 0 to +25 with the same curve that includes the shadow tweak.
Note most of the a/b readouts remain the same while the Luminance channel increased. Not all color a/b channels stayed the same. Also note the skin tone luminance went from 66L to 79L, but the Purple patch went from 31L to only 37L.
Why didn't the luminance equally increase with both patches? Non-linear behavior? I'ld like that one answered.
-
As for the "dark theatre with the lights turned on" reference I was loosely using that to illustrate the differences between human vision and colorimetric accuracy according to spectro measured Lab numbers in the CCchart. As another analogy from my understanding, if a spectro is used to measure and profile a display in a dimly lit studio without ambient light taken into account and that calibrated/profiled display is later placed outdoors in broad daylight displaying the same CCchart, the appearance according to human perception of the display would be greatly affected but the Lab numbers would still be the same. This is what I meant.
I see your point now. The viewing conditions and observer adaptions are important. With regard to the accuracy of the capture, the essential task is to have the color values in the file match those of the subject. As Eric pointed out, what the camera sees will be affected by the light used to illuminate the target. The camera can produce a metameric match for that illumination, but results may be different for another illuminant, as shown by the Metacow (http://www.cis.rit.edu/fairchild/WhyIsColor/Questions/2-8.html). Scene brightness also affects perception as shown here (http://www.cis.rit.edu/fairchild/WhyIsColor/Questions/1-6.html).
Even if the image produces a good match, the viewing conditions may affect perception, and the CIECAM02 model attempts to account for these factors. A CIECAM02 plugin is available for Photoshop and tries to take some of these factors into account. I don't know how to use it, but it is interesting to play around with.
(http://bjanes.smugmug.com/Photography/Calibration/i-9G5JHT5/0/XL/CIECAM02ScrCap-XL.png)
But really what I was trying to point out to the OP is why it's difficult to get exact CCchart numbers and still have the image look according to human vision. Spectro's aren't human.
But I can tell you the profile works quite well in my experience maintaining balance across a wide range of colors within a wide range of images of various dynamic range when trying to edit the image to get it to look according to human vision. That's really all that can be expected from a profile since we can't calibrate and profile reality which is what the digital camera is trying to capture.
This link explains how spectrophotometer data can be translated to reproduce the appearance of the object with a given illuminant. This is presumably how x-rite produced the L*a*b values for the chart for D50 illumination. Even though you used daylight illumination in your test, the camera white balance and profile was able to produce values very close to the D50 reference values. I can download a virtual ColorChecker in CIE L*a*b from Bruce Lindbloom's site and convert it to D65 AdobeRGB in Photoshop. Does this produce the same values that would be obtained by calculating the RGB values for D65 illumination? In other words, is the illuminant used for the photograph and the white point of the working space taken into account by this transformation? It appears that the profile is quite accurate for reproducing the ColorChecker, but it's performance for an actual scene with more colors might not be so good.
Regards,
Bill
-
Yes, I understand that, but what I'm still not getting is why creating a profile using a color chart image, then applying that profile to the same image doesn't result in the values the software thinks the chart patches have. Isn't the software essentially building a transform that converts the input image data to the ideal color patch data? If so, at least for the color components, I'd expect that if I apply that transform to the data used to build the transform I'd get the ideal values.
Perhaps what I'm seeing are the DNG editor's ideal values? Is there a published set of values for the patches that the DNG editor uses?
Which "ideal values" are you referring to? The RGB/Lab/HSL readouts in the DNG PE don't report the ideal values. They report the values of the input image after a color matrix transform has been applied, but before the color table transform has been applied. (The color table is what you create in DNG PE by adding control points to adjust color.) In other words, the readouts report the input (not the output) of the color transform.
-
Eric, quite true. But one can use chromatic adaption as discussed in Section 5 of Danny Pascale's RGB Coordinates of the Macbeth Color Checker (http://www.babelcolor.com/download/RGB%20Coordinates%20of%20the%20Macbeth%20ColorChecker.pdf). One can also use chromatic adaption to derive R'G'B' values for other illuminants. X-Rite publishes values for D50 CIE L*a*b and D65 sRGB. Danny and Bruce Lindbloom give values for multiple illuminants. The other factor is the actual illuminant used to take the picture. In my case, I use Solux 4700K bulbs which are not that far from D50 and rely on Camera Raw to perform a transform from the camera XYZ data using white balance and whatever else to ProPhotoRGB D50. In any case, one can compare the observed rendered values under the employed illuminant and compare them to the published or measured values of the chart and obtain a measure of the calibration of the system--illuminant, chart, camera, and raw converter. Your comments would be appreciated.
Of one had the SPD of the Solux bulbs and the spectral reflectance properties of the chart, one could calculate the D50 L*a*b values directly using the 2 degree standard observer data, but this seems to be overkill for most purposes.
Even with chromatic adaptation, the resulting numerical values are going to be quite different. For example, illuminant A adapted to D50 is very different (both visually and numerically) compared to D50 directly.
It is true that visually there will be only small differences if you're using D50-based reference values, and the actual illuminant is similar spectrally to D50.
-
My impression is that if your profile can reproduce these 24 colors to match published values then in theory other colors would fall more or less into place within an acceptable range. Isn't that the whole point of jumping through the hoop of making custom DNG profiles? If the only advantage of custom profiles is that you can accurately reproduce a macbeth chart (and even that seems questionable at this point) why would anyone do it? Why not just use a grey card and go from there?
Yes and no. It depends a lot on which set of "other colors" you're talking about (for example, blue hydrangea flower petals are notoriously difficult to reproduce accurately without throwing off other colors). In essence one has to pick what "colors" (really, spectral radiances) to optimize for.
-
Which "ideal values" are you referring to? The RGB/Lab/HSL readouts in the DNG PE don't report the ideal values. They report the values of the input image after a color matrix transform has been applied, but before the color table transform has been applied. (The color table is what you create in DNG PE by adding control points to adjust color.) In other words, the readouts report the input (not the output) of the color transform.
How does the user know what control points to add and what adjustments to make? The user can't refer to the values published on the XRite or Lindbloom sites because the values in the DNG PE are based on a linear gamma. So where does the user start when trying to make adjustments? Or is it just to be assumed that the initial step of the matrix transform does the job?
-
Which "ideal values" are you referring to? The RGB/Lab/HSL readouts in the DNG PE don't report the ideal values. They report the values of the input image after a color matrix transform has been applied, but before the color table transform has been applied. (The color table is what you create in DNG PE by adding control points to adjust color.) In other words, the readouts report the input (not the output) of the color transform.
By "ideal values" I meant the values for the color checker patches programmed into the DNG editor and used by the color checker profile wizard.
My understanding of what the DNG wizard does is that it reads the input values from the color patches in the loaded image, then creates a transform intended to map the input values to the values programmed into it for the patches. I'm not doing any manual tweaking of the control points so whatever is created by the wizard is what's in the created profile.
Then in ACR I apply that profile to the same input image of the color checker. I expected that the output of the transform would be very close to the values for the patches programmed into the DNG editor. Is that not the case?
As I asked earlier, are the values for the patches that the DNG editor is trying to achieve by the profile transform available anywhere? The reason I ask is that when I compare the Lab values in the ACR application in ProPhoto mode to the published values for the color checker, they don't match numerically and are visibly different.
-
By "ideal values" I meant the values for the color checker patches programmed into the DNG editor and used by the color checker profile wizard.
My understanding of what the DNG wizard does is that it reads the input values from the color patches in the loaded image, then creates a transform intended to map the input values to the values programmed into it for the patches. I'm not doing any manual tweaking of the control points so whatever is created by the wizard is what's in the created profile.
Then in ACR I apply that profile to the same input image of the color checker. I expected that the output of the transform would be very close to the values for the patches programmed into the DNG editor. Is that not the case?
As I asked earlier, are the values for the patches that the DNG editor is trying to achieve by the profile transform available anywhere? The reason I ask is that when I compare the Lab values in the ACR application in ProPhoto mode to the published values for the color checker, they don't match numerically and are visibly different.
So nothing I've posted answers this question for you? Or are you waiting for an "official" answer from Eric Chan? I think he already gave it to you somewhere in this thread.
I don't know what we can tell you that would make you understand.
-
Then in ACR I apply that profile to the same input image of the color checker. I expected that the output of the transform would be very close to the values for the patches programmed into the DNG editor. Is that not the case?
No, not if you're doing the import with ACR's default settings. There's the various other things that ACR does by default - tone curve and contrast settings to take into account. The series of blog posts I referred to previously show how to do the reconciliation between patch values and "on screen" values. And to the point that Eric made above, even once you've eliminated those things, because of illuminant, etc it's likely to be "close" rather than "very close"
Sandy
-
Sandy, if the original Raw image of the chart were converted to DNG using Adobe's DNG Converter software, would that eliminate much, or all, of the problem of the Raw converter and be a better starting point for the calibration process?
Alternatively, if you were to set up a Lightroom import preset that zero'd out all the sliders and in the Tone Curve Pane applied a curve that offset the default LR tone curve embedded in the ACR Standard profile (or whichever profile is used as the default), would that be a better starting point?
-
So nothing I've posted answers this question for you? Or are you waiting for an "official" answer from Eric Chan? I think he already gave it to you somewhere in this thread.
I don't know what we can tell you that would make you understand.
Actually, apart from Eric's original comment that the patch values used by the DNG editor are not the same as those published by XRite, I don't think anything you or anyone else on this thread have posted even addresses my question.
Mostly the answers here have dealt with the the difficulty of matching the actual physical patches because of lighting, tone curve matching, camera response, etc. or are completely irrelevant comments about how matching the patches doesn't guarantee that other colors will match.
None of those have anything to do with what I'm asking.
-
Sandy, if the original Raw image of the chart were converted to DNG using Adobe's DNG Converter software, would that eliminate much, or all, of the problem of the Raw converter and be a better starting point for the calibration process?
Alternatively, if you were to set up a Lightroom import preset that zero'd out all the sliders and in the Tone Curve Pane applied a curve that offset the default LR tone curve embedded in the ACR Standard profile (or whichever profile is used as the default), would that be a better starting point?
How the raw image is converted won't (shouldn't ;D) make a difference.
Yes, if you want to try to match values then a zero'd import is a far easier starting point.
If you really want to push ahead with this, then I'd suggest you download some of the synthetic GM24 images I created back when I was doing the calibration work, and first understand what you get in LR/ACR using an "ideal" image. Then go forward with real images.
You can download the synthetic images here: http://sites.google.com/site/chromasoft/referenceimages
There are also spreadsheets of patch values in different color spaces, etc on the same site.
Sandy
-
How does the user know what control points to add and what adjustments to make? The user can't refer to the values published on the XRite or Lindbloom sites because the values in the DNG PE are based on a linear gamma. So where does the user start when trying to make adjustments? Or is it just to be assumed that the initial step of the matrix transform does the job?
Bob, the DNG Profile Editor was designed to be a visual editor. Specifically, open an image that shows colors to be "off" in some way. Click on the problem color in the image. It'll create a control point. Then you adjust the color till it looks right. You can place additional control points to change other colors, or simply to "lock down" specific colors to prevent them from changing.
-
By "ideal values" I meant the values for the color checker patches programmed into the DNG editor and used by the color checker profile wizard.
My understanding of what the DNG wizard does is that it reads the input values from the color patches in the loaded image, then creates a transform intended to map the input values to the values programmed into it for the patches. I'm not doing any manual tweaking of the control points so whatever is created by the wizard is what's in the created profile.
Yes, that's right. If you build a single-illuminant profile in DNG PE, the target values used by DNG PE are from a set of averaged ColorChecker charts with adopted illuminant of D50. If you build a dual-illuminant profile in DNG PE, the illuminants used by DNG PE are A and D65.
Then in ACR I apply that profile to the same input image of the color checker. I expected that the output of the transform would be very close to the values for the patches programmed into the DNG editor. Is that not the case?
They should be visually close, yes, at default ACR settings. (But not numerically close. The readouts in ACR are different from the readouts in DNG PE.)
As I asked earlier, are the values for the patches that the DNG editor is trying to achieve by the profile transform available anywhere? The reason I ask is that when I compare the Lab values in the ACR application in ProPhoto mode to the published values for the color checker, they don't match numerically and are visibly different.
They'll be different numerically & visually from the ideal values because of the default tone curve rendering and black subtraction that ACR applies, which adds contrast and modifies saturation (as Sandy noted, above). The published ideal values assume no additional rendering is done. If you want to get a closer match, you would need to turn off the default ACR rendering. That is done by setting Brightness, Contrast, and Blacks to 0 (instead of 50, 25, and 5) and settings the point curve to Linear instead of Medium Contrast.
-
If you want to get a closer match, you would need to turn off the default ACR rendering. That is done by setting Brightness, Contrast, and Blacks to 0 (instead of 50, 25, and 5) and settings the point curve to Linear instead of Medium Contrast.
Which is basically what I had to do in the posted samples above to get exact numerical Lab values to the published numbers Bjanes posted and finally measured from shown in his Imatest posting.
Except instead of zeroing out all settings, I had to modify a "linearish" tone curve and set Brightness to +60 in ACR. I built the DNG Profile used in that image sample using ACR defaults embedded in the DNG input image.
-
How the raw image is converted won't (shouldn't ;D) make a difference.
Yes, if you want to try to match values then a zero'd import is a far easier starting point.
If you really want to push ahead with this, then I'd suggest you download some of the synthetic GM24 images I created back when I was doing the calibration work, and first understand what you get in LR/ACR using an "ideal" image. Then go forward with real images.
You can download the synthetic images here: http://sites.google.com/site/chromasoft/referenceimages
There are also spreadsheets of patch values in different color spaces, etc on the same site.
Sandy
Thanks, Sandy. My thought was that if the export to DNG in LR (or ACR) was applying a tone curve via the default calibration profile, using the DNG Converter may be a way to avoid that as I'd expect the DNG Converter wouldn't have any sort of default tone curve and would just use a linear curve.
Bob, the DNG Profile Editor was designed to be a visual editor. Specifically, open an image that shows colors to be "off" in some way. Click on the problem color in the image. It'll create a control point. Then you adjust the color till it looks right. You can place additional control points to change other colors, or simply to "lock down" specific colors to prevent them from changing.
Thanks, Eric. I understand how the control points work. It was more a case of determining what to use a a reference point for the CC chart if the profile created from the chart didn't provide a good enough 'match' when applied back in LR and whether it would be possible to use something other than just a visual reference. Seems not.
-
Thanks, Sandy. My thought was that if the export to DNG in LR (or ACR) was applying a tone curve via the default calibration profile, using the DNG Converter may be a way to avoid that as I'd expect the DNG Converter wouldn't have any sort of default tone curve and would just use a linear curve.
ACR or LR will by default apply a tone curve to any image that it considers to be "scene referred". That's essentially any raw image, but not e.g., a JPEG or TIFF. This allows ACR/LR to load raws with the tone curve, but to load JPEGs and TIFFs "as created". So the tone curve, etc has to do with the import, not the conversion. Although, just to complicate the situation (a) DNGs can be created as either scene referred or not, and (b) DNG camera profiles can have any tone curve you want embedded in them. But neither of those complications is likely to be relevant in this situation - any DNG converted from a raw image will be tagged as scene referred, and the profile editor (so far as I know anyway) embeds the Adobe standard tone curve unless you use the control points.
Sandy
-
I don't want to belabour the point, Sandy but the DNG Converter (http://www.adobe.com/support/downloads/detail.jsp?ftpID=5258) converts RAW files without using LR or ACR. I'm wondering if that would be a better starting point if that utility doesn't encode a tone curve.
Re (b), understood. I can modify it as I see fit. But the 'canned' Adobe profiles have a curve built in, right? So unless a user edited one of the Adobe profiles to alter the embedded curve that's what would be used. That goes back to my other question about adding an offsetting curve in the Tone Curve pane of ACR/LR to bring the image back to linear.
Re (a), how would one create a DNG via ACR/LR that wasn't scene-referred? Or is that done outside of ACR/LR?
Re the curve used in DNG PE, control points or the Tone Curve Pane?
-
I don't want to belabour the point, Sandy but the DNG Converter (http://www.adobe.com/support/downloads/detail.jsp?ftpID=5258) converts RAW files without using LR or ACR. I'm wondering if that would be a better starting point if that utility doesn't encode a tone curve.
Re (b), understood. I can modify it as I see fit. But the 'canned' Adobe profiles have a curve built in, right? So unless a user edited one of the Adobe profiles to alter the embedded curve that's what would be used. That goes back to my other question about adding an offsetting curve in the Tone Curve pane of ACR/LR to bring the image back to linear.
Re (a), how would one create a DNG via ACR/LR that wasn't scene-referred? Or is that done outside of ACR/LR?
Re the curve used in DNG PE, control points or the Tone Curve Pane?
I don't think there's any way to create a DNG that isn't either scene referred or has a tone curve baked in already in ACR/LR. Normally you would only create a non-scene referred DNG (one with a "crICCProfilePCS" tag) if you were writing a "linear raw" DNG - aka if you were writing data that was demosaiced/color converted, etc.
Re the curve, I'd assume it could potentially be both, but that's really a question for Eric - Tone curves and tables are somewhat interchangeable. The simple approach would be to just have the tone curve set by the tone curve panel, and the lookup tables by the control points, which would give you nice simple "2D" tables, but I don't know exactly what PE might do under what circumstances.
Sandy
-
I don't want to belabour the point, Sandy but the DNG Converter (http://www.adobe.com/support/downloads/detail.jsp?ftpID=5258) converts RAW files without using LR or ACR.
yes, but DNG converter has full image rendering code of ACR/LR included inside (because, as you know, it can create embedded JPG preview w/ all parametric corrections that were done by ACR/LR - so it has to have that code inside).
-
yes, but DNG converter has full image rendering code of ACR/LR included inside (because, as you know, it can create embedded JPG preview w/ all parametric corrections that were done by ACR/LR - so it has to have that code inside).
That's what I wasn't entirely sure of. If it contains the full monty of the rendering code then it's a moot point. That would seem a bit odd since it's not actually rendering the file into a visible form. The DNG Converter does have the option to select Linear (demosaiced), so perhaps that's the answer.
I know that the converter can create a JPEG preview. But that doesn't have to be based on the rendering engine of LR/ACR. Cameras create a JPEG preview as well based on the in-camera chosen colour space. The DNG Converter could simply do the same based on, say, sRGB but that doesn't necessarily have to impact the conversion of the Raw file to a DNG, does it? Do cameras that record in DNG have an Adobe tone curve applied? I'd think probably not.
-
That would seem a bit odd since it's not actually rendering the file into a visible form.
I am not talking about user interface - I am talking about the code that does demosaicking, NR, etc
But that doesn't have to be based on the rendering engine of LR/ACR
it does - try it yourself - store parametic edits (done w/ ACR or LR) in DNG file itself and then instruct DNG converter to update JPG preview inside (overwriting the original preview)... then open it in a 3rd party viewer that can display embedded JPG itself.
-
I am not talking about user interface
Neither am I.
it does - try it yourself - store parametic edits (done w/ ACR or LR) in DNG file itself and then instruct DNG converter to update JPG preview inside (overwriting the original preview)... then open it in a 3rd party viewer that can display embedded JPG itself.
I wasn't saying it wasn't done that way. I was saying it didn't necessarily have to be done that way. You're talking about edits made in LR/ACR. I can see how those may get picked up. I'm talking about an image that never sees LR/ACR but is loaded into the DNG Converter straight from the memory card or hard drive. If it still applies a default tone curve, then so be it.
-
I'm talking about an image that never sees LR/ACR but is loaded into the DNG Converter straight from the memory card or hard drive. If it still applies a default tone curve, then so be it.
I was only making the point that Adobe DNG Converter in fact has the full ACR/LR raw conversion code (minus UI) in it (that is also one of the reasons why it can't be open sourced) and so it can applyto raw file everything that ACR/LR can apply ("never sees LR/ACR" = open in ACR/LR w/ the default settings from Adobe "as installed")
-
I was only making the point that Adobe DNG Converter in fact has the full ACR/LR raw conversion code (minus UI) in it (that is also one of the reasons why it can't be open sourced) and so it can applyto raw file everything that ACR/LR can apply ("never sees LR/ACR" = open in ACR/LR w/ the default settings from Adobe "as installed")
And that's fine. If that's the way it works then so be it, there's no advantage. I know the converter exists but don't use it so wasn't sure how it was set up.
-
This thread is pretty valuable so I'm posting just so I can get notifications. (I have a ColorChecker Passport and have seriously wondered about all this stuff.)
-
yes, but DNG converter has full image rendering code of ACR/LR included inside (because, as you know, it can create embedded JPG preview w/ all parametric corrections that were done by ACR/LR - so it has to have that code inside).
Actually, not. My understanding is that while DNG converter is smart enough to understand all the raw formats that ACR/LR understands, and all of the color rendering and lens correction that can be encoded into a DNG camera profile, the demosaicing, noise reduction, etc algorithms are not as advanced as ACR/LR - they're just good enough to be able to create thumbnails and previews. So you can't in effect use the DNG converter as a no-cost substitute for LR/ACR
Sandy
-
Actually, not. My understanding is that while DNG converter is smart enough to understand all the raw formats that ACR/LR understands, and all of the color rendering and lens correction that can be encoded into a DNG camera profile, the demosaicing, noise reduction, etc algorithms are not as advanced as ACR/LR - they're just good enough to be able to create thumbnails and previews. So you can't in effect use the DNG converter as a no-cost substitute for LR/ACR
Sandy
certainly it creates just highly compressed JPGs and not 16bit TIFFs (= "So you can't in effect use the DNG converter as a no-cost substitute for LR/ACR"), but think again - why 'd Adobe developers maintain a separate code for that part of DNG converter functionality vs ACR/LR ;D ? the code in Adobe DNG converter even supports all the local edits w/ brush that you can do in ACR/LR...
-
certainly it creates just highly compressed JPGs and not 16bit TIFFs (= "So you can't in effect use the DNG converter as a no-cost substitute for LR/ACR"), but think again - why 'd Adobe developers maintain a separate code for that part of DNG converter functionality vs ACR/LR ;D ? the code in Adobe DNG converter even supports all the local edits w/ brush that you can do in ACR/LR...
They don't need to - there's a low performance (very very low ;D) demosaicing module in the DNG SDK. Which sure isn't what's in ACR/LR.
Sandy
-
They don't need to - there's a low performance (very very low ;D) demosaicing module in the DNG SDK. Which sure isn't what's in ACR/LR.
Sandy
well, we can ask Eric Chan directly or we can do a test... let us see... new sharpening technology in recent versions of ACR uses some form of deconvolution, so we can push the sliders to make it prominent, run DNG converter and see if the JPG thumbnail will display the same effect as seen in ACR... OK ? and we can also test NR the same way - push sliders so that effect in ACR will be very-very noticeable to everybody's eye and then check if the same is seen in JPG thumbnail... that low performance demosaicing module - has it any NR there at all ;) ?
-
Which, in the end, is all essentially moot anyway. The DNG PE backs out edits made in LR/ACR except for WB. The only thing I wasn't sure of was whether the same default tone curve was applied via the DNG Converter vs via an export out of LR/ACR.
-
well, we can ask Eric Chan directly or we can do a test... let us see... new sharpening technology in recent versions of ACR uses some form of deconvolution, so we can push the sliders to make it prominent, run DNG converter and see if the JPG thumbnail will display the same effect as seen in ACR... OK ? and we can also test NR the same way - push sliders so that effect in ACR will be very-very noticeable to everybody's eye and then check if the same is seen in JPG thumbnail... that low performance demosaicing module - has it any NR there at all ;) ?
Well, I did ask Eric a while ago, and he said that's what's in the DNG converter isn't what's in ACR/LR. But's that quite a long time ago, so things may have changed.
Sandy
-
Well, I did ask Eric a while ago, and he said that's what's in the DNG converter isn't what's in ACR/LR. But's that quite a long time ago, so things may have changed.
Sandy
I think I remember that topic, but what is the exact quote of your question/his answer... let me try to find it...
here it is :
"...Hi Sandy, yes, you are right: the DNG SDK only has a simple bilinear interpolation demosaic routine implemented. (We have been meaning to update it with something more serviceable, but have not worked that into the already rather-packed dev schedule ..."
Eric is talking about available in source code form SDK, not closed source Adobe DNG Converter... and that is the point - Adobe DNG converter has the full blown ACR/LR raw conversion/adjustment interpretation code inside, DNG SDK being available for a general public as a source code does not have that functionality.
-
Does this exchange between Eric and Nat Coalson change anyone's opinion on what's in a DNG and what isn't?
http://www.natcoalson.com/blog/2011/11/29/my-adobe-dng-chat-with-eric-chan/
-
Does this exchange between Eric and Nat Coalson change anyone's opinion on what's in a DNG and what isn't?
Nope...it's a standardized file format wrapper, just like Eric said. There are technical issues and political issues, Eric addressed the technical issues. The politics of DNG are considerably murkier than the technical issues because so many people have incorrect assumptions based on faulty technical understanding.
-
Nope...it's a standardized file format wrapper, just like Eric said. There are technical issues and political issues, Eric addressed the technical issues. The politics of DNG are considerably murkier than the technical issues because so many people have incorrect assumptions based on faulty technical understanding.
LOL. There's a shock, Jeff. I was really posing that question to deejjjaaaa because Eric's explanation seems to conflict with deejjjaaaa's understanding.
Intuitively, to me, it seems odd that conversion to DNG would apply a default tone curve based on an Adobe profile (e.g., Adobe Standard). It would seem that would cause a difference in the appearance of two images, one original Raw and one DNG, in any other program that was able to read DNG files, like Photomatix wouldn't it? I just tried it. Converted 5 Raw files to DNG, loaded the DNG files into PM, tonemapped and saved then loaded the same 5 Raw files into PM, tonemapped and saved and the two tonemapped files are identical. I loaded the files into PM directly, not via the LR plugin, so PM is doing the conversion. Doesn't that indicate that the DNG doesn't have a default Adobe tone curve applied to it? Exporting those same 5 files out of LR to PM produces a different result. I've got an import preset that zeroes out everything (incl Blacks, Brightness, Contrast) and sets the curve to linear so the only thing that should be different on export is the application of the Adobe Standard tone curve. That tonemapped image is different.
Even within an Adobe program, unless LR or ACR handled DNG files differently from how it handles other files, there'd be a doubling of the tone curve, once for what was embedded in the DNG file and again when the DNG file was opened in LR/ACR based on whatever profile was chosen in the Camera Calibration function. Wouldn't there?
-
Intuitively, to me, it seems odd that conversion to DNG would apply a default tone curve based on an Adobe profile (e.g., Adobe Standard).
It does...based on the "Default" ACR/LR settings YOU have set as default. If you haven't changed your ACR/LR defaults then they will be the same "Defaults" Thomas and Eric decided should be default and pretty much all other applications that support DNGs would see the raw DNG the same way. And it will currently be with the "medium" tone curve applied–unless you've changed the Defaults...
It's important to note that Camera Raw, DNG Converter and Camera Raw all share a per machine Default settings...if you change your Defaults in ACR or LR then DNG Converter's Defaults will also change. There is no UI to alter the DNG Converter's Defaults though.
-
Yes, that's right. If you build a single-illuminant profile in DNG PE, the target values used by DNG PE are from a set of averaged ColorChecker charts with adopted illuminant of D50. If you build a dual-illuminant profile in DNG PE, the illuminants used by DNG PE are A and D65.
They should be visually close, yes, at default ACR settings. (But not numerically close. The readouts in ACR are different from the readouts in DNG PE.)
They'll be different numerically & visually from the ideal values because of the default tone curve rendering and black subtraction that ACR applies, which adds contrast and modifies saturation (as Sandy noted, above). The published ideal values assume no additional rendering is done. If you want to get a closer match, you would need to turn off the default ACR rendering. That is done by setting Brightness, Contrast, and Blacks to 0 (instead of 50, 25, and 5) and settings the point curve to Linear instead of Medium Contrast.
Which is basically what I had to do in the posted samples above to get exact numerical Lab values to the published numbers Bjanes posted and finally measured from shown in his Imatest posting.
Except instead of zeroing out all settings, I had to modify a "linearish" tone curve and set Brightness to +60 in ACR. I built the DNG Profile used in that image sample using ACR defaults embedded in the DNG input image.
I've been trying that all along with OK but far from perfect results. If I shut off all the ACR edits the results are very far off from the published values. If I adjust the tone curve to match the gray patch published values, with or without changing the exposure or brightness first to get it close the values are OK but not perfect.
My original question was what was happening that made the profile transform not produce the published values. The assumption being that the DNG created profile is a transform designed to map from an image to the color checker. If that were true then applying that transform to the original image should produce something very close to the values programmed into the DNG editor.
Is the major issue here that because the profile doesn't produce a full tone correction, the transform isn't complete? That is, the DNG PE profile doesn't attempt to really reproduce the color checker in the Prophoto color space because it's not fully dealing with the tone curve? Since attempting to later reproduce the tone curve in ACR is both inexact and modifies all 3 Lab components of all the patches it's pretty much impossible to get back to the published values. Every tweak to the tone of the gray patches is also moving all 3 of the the color patch values around in a non-linear manner.
I did try playing with the tone curve in the DNG Editor to see if it was possible to get the profile to reproduce the tone curve. I started both from the base curve and from linear but both produced quite unexpected results when applied to the image in ACR. It's not clear to me what this is doing or what the input/output numbers for the control points mean, but that's really a separate topic.
-
My original question was what was happening that made the profile transform not produce the published values.
You're expecting too much out of an image editing program designed for photographers who rely on visual judgement in determining what looks good to them. Exact numerical color matching is almost impossible to pull off by most photographers due mostly to the fact they have to rely on the limitations of their memory of the actual scene. Let me reemphasize the Eric Chan quote you posted...
They should be visually close, yes, at default ACR settings. (But not numerically close. The readouts in ACR are different from the readouts in DNG PE.)
If you want an exact visual match of your painting viewed on your display in ACR/LR to match the actual painting you see with your eyes, you're better off doing this using your eyes to guide your edits along with a custom camera profile. The main purpose of a camera profile using the DNG PE Wizard is to get the hues='a/b' Lab channel relationships to remain the same when applying contrast, brightness which in turn affect saturation levels. Human vision sees hue errors more so over any other of the HSL color channels which is why ACR/LR includes an HSL panel.
Believe me, it's easier than you think to get an accurate reproduction using ACR/LR. I've done it myself countless times photographing objects having a wide range of colors lit by various types of continuous lights, placed next to my display and editing to get an exact match. It's uncanny how close I get, but I have to use not just the profile but the HSL, WB sliders, SplitTone panel to get a match.
Sometimes I can just use Adobe Standard or ACR 4.4 default especially if I use neutral-ish fluorescent lights which I have several different brands. And due to these light's spiky spectra HSL panel and WB sliders really fix a lot of the hue errors.
This is why you need to shoot your paintings under full spectrum light source like Solux 4700K bulbs, strobes or flash. It will make editing a whole lot easier.
-
Oh, and I found an added bonus converting my Raw PEF's to DNG by saving out of ACR in that format. I went into ACR's (4.6) Preferences and checked both boxes in the DNG file handling section to "ignore xmp files" and "Update Jpeg Previews" set to "Full Size".
Now when I save out of ACR to DNG format I select Jpeg Preview=Full Size and now sharpening shows up in Bridge CS3's Preview Pane of ONLY DNG previews of my Raw PEF's. I could never get my PEFs to show applied sharpening in the image preview even when I have selected "Use High Quality Previews" in Bridge Preferences. I even purged the cache and it doesn't work.
I don't know if choosing Full Size JPEG previews or Update JPEG previews caused this to happen.
-
based on faulty technical understanding.
based on non disclosure by Adobe
-
I was really posing that question to deejjjaaaa because Eric's explanation seems to conflict with deejjjaaaa's understanding.
I was only stating that Adobe DNG converter has the same __code__ as ACR/LR to deal w/ raw conversion/sharpening/NR and various adjustment interpretation (like local edits = adjustment brush, etc) and that Adobe DNG converter is perfectly capable to generate JPG output (degraded by compression of course) natuarally matching what ACR/LR will produce (again taking into consideration that it will be a highly compressed JPG intended for preview purposes and stored inside DNG file - but you can extract it from there w/ various tools)... and I did not see anything posted by Eric to contradict that (Adobe DNG converter != DNG SDK).
-
Eric is talking about available in source code form SDK, not closed source Adobe DNG Converter... and that is the point - Adobe DNG converter has the full blown ACR/LR raw conversion/adjustment interpretation code inside, DNG SDK being available for a general public as a source code does not have that functionality.
Well, sort of. This is actually the relevant part of my post that Eric was responding to:
Umm, DNG converter has the full ACR demosaicing algorithm embedded in it, rather than a simplified, lower performance version? That would surprise me. But then, I get surprised on a regular basis.....
But subsequently Eric and I have also had several "off-forum" conversations - while those conversation were not about the DNG converter, I can only repeat that my understanding is that the DNG converter doesn't have the full ACR/LR conversion engine in it. But as I said in the quote above, in the field of color management, I get surprised regularly :o
Sandy
Sandy
-
It does...based on the "Default" ACR/LR settings YOU have set as default. If you haven't changed your ACR/LR defaults then they will be the same "Defaults" Thomas and Eric decided should be default and pretty much all other applications that support DNGs would see the raw DNG the same way. And it will currently be with the "medium" tone curve applied–unless you've changed the Defaults...
It's important to note that Camera Raw, DNG Converter and Camera Raw all share a per machine Default settings...if you change your Defaults in ACR or LR then DNG Converter's Defaults will also change. There is no UI to alter the DNG Converter's Defaults though.
Yep, I get that, Jeff. But the one thing I can't change in ACR/LR is the application of a profile via the Camera Calibration pane. That's the only aspect I'm uncertain about. Whether it's Adobe Standard, Camera Faithful or whatever one of those Adobe profiles is used, there's a tone curve baked in. Does that profile get taken into account when converting to a DNG? If it does then I don't believe that's the best starting point for trying to create a custom DNG camera profile (maybe my belief is wrong). My thinking is that the best starting point is the linear raw data captured by the camera (or as close to it as possible). If there's a tone curve in the DNG as a result of the profile used in the Calibration pane (or if there's one baked into the DNG Converter) then it's not the linear camera data anymore. Now, DNG PE does back out any user edits (except WB) when the DNG is opened in that application. If PE also backs out the tone curve of the calibration profile, no problem. But I don't know if it does. Does it?
-
Well, sort of. This is actually the relevant part of my post that Eric was responding to:
1) you asked about Adobe DNG converter (closed source) and Eric answered you about Adobe DNG SDK (open source)...
why ? may be because :
2) the issue discussed there was "original raw file" -> "linear DNG file" conversion, which is different from what we (you and me - not the original theme of the topic) are discussing now (generation of embedded JPG previews).
-
then it's not the linear camera data anymore.
some cameras already make the data "non linear" before writing their raw files and then raw converters make the reverse application of a "curve" to the data when reading it.
for example : http://translate.google.com/translate?sl=ru&tl=en&js=n&prev=_t&hl=en&ie=UTF-8&layout=2&eotf=1&u=http%3A%2F%2Fblog.lexa.ru%2F2011%2F11%2F10%2Fo_lineinosti_raw_nikon_d5x00.html
-
The translation on that link is useless. But that still doesn't answer the question.
-
Wow! Why does that Nikon D5xx curve look so familiar? Looks pretty close to my attempt at getting exact Lab numbers for the sample image posted above.
Bob said:
If there's a tone curve in the DNG as a result of the profile used in the Calibration pane (or if there's one baked into the DNG Converter) then it's not the linear camera data anymore.
After the voltage readings off each sensor pixel site goes through the A/D converter, all sensor data is interpreted by software at that point. Remember it's just grayscales assigned RGB colorants. Even demosaicing is an interpretation. Even if you were to work with linear data each converter will deliver its own flavor/interpretation of what that looks like or how it's calculated.
I've compared three previews of "linear" Raw data from what was claimed by each Raw converter to be a linear setting based preview and they were all different from each other.
In the end it's really just individually colored tiny little squares crammed tightly next to each other. It's the display preview that's really controlling how you see and edit the severely anti-aliased preview depending on zoom level. Who cares what each individual pixel looks like. They all look the same at 1600% zoom level in Photoshop.
-
Wow! Why does that Nikon D5xx curve look so familiar? Looks pretty close to my attempt at getting exact Lab numbers for the sample image posted above.
Bob said:
After the voltage readings off each sensor pixel site goes through the A/D converter, all sensor data is interpreted by software at that point. Remember it's just grayscales assigned RGB colorants. Even demosaicing is an interpretation. Even if you were to work with linear data each converter will deliver its own flavor/interpretation of what that looks like or how it's calculated.
I've compared three previews of "linear" Raw data from what was claimed by each Raw converter to be a linear setting based preview and they were all different from each other.
In the end it's really just individually colored tiny little squares crammed tightly next to each other. It's the display preview that's really controlling how you see and edit the severely anti-aliased preview depending on zoom level. Who cares what each individual pixel looks like. They all look the same at 1600% zoom level in Photoshop.
No quibble with any of that, Tim. What I'm trying to figure out - and as yet can't get an answer - is how to get the best starting point for creating the camera profile. It would seem that the linear (or near linear) data out of the camera would be that best starting point. But a file with a tone curve baked in seemingly wouldn't be. If that (near) linear data is the best starting point and if the tone curve from the camera calibration pane of LR/ACR is getting baked into the DNG during conversion because that part of the rendering can't be turned off and if that default tone curve isn't backed out of the file when it's opened up in DNG PE or run through something like XRite's Passport software to bring the file back to linear then it seems like a fruitless exercise. But no one's yet said if that (near) linear camera data is the best starting point and why or why not. No one's yet said whether the tone curve of the profile used as the default in the camera calibration pane gets baked in or not. No one's yet said whether that tone curve gets backed out along with any other user edits in DNG PE or the Passport software (user edits get backed out in the Passport software too because I loaded a b&w DNG and it opened in colour). And I'm fully on board with the fact that the camera profile isn't going to be perfect (I've said that all along) and that further 'tweaking' will be necessary. But it almost seems as though it's coming down to the point that going through the step of creating a camera profile is essentially useless.
-
Hi Bob,
Conversion to DNG doesn't involve any application of a tone curve. The tone curve gets applied when rendering a DNG to an output (e.g., for display or print purposes).
There is a very significant difference between "converting" and "rendering." As an example, consider opening a PSD file in Photoshop and re-saving it as a TIFF. All you've done is a conversion (of file format). There is no change in rendering. Now, consider taking that same image, and applying a tone curve to it (e.g., using Curves panel). In that case, you're changing the rendering.
Converting a raw file (like CR2, NEF, etc.) to DNG isn't rendering. It's just converting to another file format / container for the purposes of holding the image & associated metadata. The image data is still unrendered (scene-referred) e.g., no white balance, no color profile, no tone curve, etc. The rendering takes place when you open that DNG into a raw converter software and then save it out as something else (e.g., JPEG, TIFF), in a so-called output-referred format.
-
No quibble with any of that, Tim. What I'm trying to figure out - and as yet can't get an answer - is how to get the best starting point for creating the camera profile. It would seem that the linear (or near linear) data out of the camera would be that best starting point.
Correct. And that's how the DNG Profile Editor works (same with X-Rite's ColorChecker Passport software). The color tables that are created using DNG PE and Passport transform the linear scene-referred RGB values. You don't need to do anything special to make this happen. Raw DNG files are (by definition) scene-referred and linear.
The part that may be confusing is the fact that the default preview of the image as shown in DNG PE (and in ACR/LR) is not scene-referred. Similarly, if you try to take RGB readouts in ACR (or open the image into Ps and take Lab readouts), you'll be reading values that aren't scene-referred. That's because of the default tone curve and black subtraction, as I explained earlier. You can turn all that stuff off, if you want to check the numerical readings against published values (with the practical caveats that I mentioned earlier about illumination, and various other factors).
-
Sandy, deejjjaaaa,
It is true that DNG Converter doesn't have all of ACR/LR engine in it.
It is also true that DNG Converter needs enough of it to build thumbnails & embedded previews (and of course, to read non-DNG raw files).
The public DNG SDK does contain a lot of the essentials, such as linearization, tone curves, white balance, and color profiles. Both DNG Converter and ACR/LR heavily use the DNG SDK. (That is, the DNG SDK isn't just some separate piece of code we put out there! ;) )
In terms of the file format, the DNG SDK contains everything needed to read & write DNGs. So a software developer could, for example, take the dcraw source code and DNG SDK and essentially write their own "DNG converter software" if desired. The bit about rendering thumbnails & embedded previews is often convenient for users, but totally optional.
-
Thanks Eric. That answers the questions. I do understand the difference between rendering and converting/scene-referred and output-referred and I know I used 'render' in my last remark but if a tone curve were applied that would be rendering and not just converting.
Understood about the comparison between sampled and published values. I'm not trying to do that. But if one were going to do that, the best way to try and compare the values would be to set the curve to Linear in the Tone Curve tab of PE. Right?
-
Understood about the comparison between sampled and published values. I'm not trying to do that. But if one were going to do that, the best way to try and compare the values would be to set the curve to Linear in the Tone Curve tab of PE. Right?
There are a couple of ways to do this.
One way is to leave the Tone Curve tab of PE alone (unchanged), and instead set ACR's Brightness Contrast, and Blacks (in Basic panel) to zero, and also set ACR's Point Curve to Linear. Thus, when you apply your color profile, you are performing the color adjustments, but turning off ACR's default tone curve rendering.
Another way is to do as you mentioned, i.e., set the curve to Linear in DNG PE's Tone Curve tab. Then in ACR, you would need to set Blacks to zero, and leave Brightness, Contrast, and the Point Curve at their respective defaults of 50, 25, and Medium Contrast.
In case you're wondering why: ACR is set up so that ACR will produce the profile's embedded tone curve, at ACR's default curve settings. ACR's default curve settings are Brightness, Contrast, and Point Curve = 50, 25, and Medium Contrast (respectively). If you've set your profile's embedded tone curve (via DNG PE's Tone Curve panel) to Linear, then that's exactly what you'll get at these ACR defaults.
-
Got it. Thanks, Eric.
-
Sandy, deejjjaaaa,
It is true that DNG Converter doesn't have all of ACR/LR engine in it.
It is also true that DNG Converter needs enough of it to build thumbnails & embedded previews
thank you for clarifications ! noted.
PS: now we need to find out what is missing though ;)
-
thank you for clarifications ! noted.
PS: now we need to find out what is missing though ;)
What I suspect is missing is based on examination of that Nikon D5xx tone curve and the similarities to the one I created to get exact Lab numbers of a perfectly exposed (within specs) CCchart.
I suspect we're seeing the effects of sensor gain on linearity brought about by the nature of electronics in response to a REAL D50-ish light source. Just throwing it out there.
Hopefully Eric Chan will set me straight on that suspicion.
-
I suspect we're seeing the effects of sensor gain on linearity brought about by the nature of electronics in response to a REAL D50-ish light source. Just throwing it out there.
How would the electronics respond differently to different light sources with respect to sensor gain?
-
How would the electronics respond differently to different light sources with respect to sensor gain?
I'm basing my suspicion on the increased amount of electrons bombarding a sensor just before peak saturation which looks to be around 200RGB region. The pulled down section in both mine and the Nikon D5xx curve hint at a need to correct in that area as a way to compensate for the non-linearity caused by this electrical phenomenon.
I doubt the electronics of Bayer sensors built into a digital camera system are as high quality with regard to electronic tolerances as the electronics built into a spectrophotometer used to measure the CCchart.
I'm only guessing on this based on my observing how some rheostat switches in home lighting (LCD brightness adjusts) and volume attenuators in home audio amplifiers behave as loudness or lighting is increased. The behavior of increase starts out smooth but suddenly becomes less smooth and eratic the higher it goes just at the point before maxing out. There's variances between manufacturers in the quality of these switches and how they behave to attenuation.
It's just a guess.
Got to remember a digital camera was never meant to be a copier of reality by the numbers, only an interpreter. The electronics has to be the weak link that prevents it from being that precise.
At least it's a lot more precise than shooting and scanning film.
Here's a speech given by the inventor of the CMOS sensor...
http://www.dpreview.com/news/2011/10/28/ericfossumspeech