Pages: [1] 2   Go Down

Author Topic: Adobe DNG Profile Editor Does not work?  (Read 10631 times)

solarj

  • Newbie
  • *
  • Offline Offline
  • Posts: 25
Adobe DNG Profile Editor Does not work?
« on: January 26, 2013, 08:19:46 pm »

I'm trying to use DNG profile editor to generate a camera profile for my new NEX-5R camera and I got some strange result, confusing

The camera profile generated by DNG PE have large errors. But there is a built-in NEX-5R profile in ACR, it works fine with small errors.

Take blue patch for example, ACR's built-in NEX-5R camera standard profile give me following reading in DNG PE:
R:  7
G: 12
B: 27
 
L:  24
a: -15
b: -25
 
While the DNG profile built by me using the chart function:
R:7
G:7
B:30
 
L:19
a:5
b:-38
 
They differ a lot, and that explained why the generated dcp file does not work.

sRGB reference value for blue patch is 56,61,150, and the ACR built-in camera profile gives me exactly 56,61,150 on blue patch spot on, but the DNG profile built by me using the chart function gives me 0,46,142, out of gamut, it is not acceptable!

I guess that should has something to do with color space, since I'm using sRGB color space everywhere, but how could I correct the situation and generate the right color profile?  ???
Logged

risedal

  • Newbie
  • *
  • Offline Offline
  • Posts: 29
Re: Adobe DNG Profile Editor Does not work?, try qp-card
« Reply #1 on: January 31, 2013, 12:29:24 pm »

Buy the qp-card solution,  easy, and cheap . I have generated my own profiles to my Canon and Nikon cameras for over a year
www.qpcard.com
here are some info I got from the company yesterday
http://www.qpcard.com/media/applications/press/PublicreleaseQPC_201301.pdf?utm_campaign=cmp_176543&utm_source=getanewsletter
Logged

solarj

  • Newbie
  • *
  • Offline Offline
  • Posts: 25
Re: Adobe DNG Profile Editor Does not work?
« Reply #2 on: February 26, 2013, 08:54:27 pm »

QPcard seems also come out with a software, but still no possibility to see the matrix, I'd better get a dng.sdk to change the matrix, but that is a lot of time required
Logged

samueljohnchia

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 498
Re: Adobe DNG Profile Editor Does not work?
« Reply #3 on: February 26, 2013, 10:18:14 pm »

I have noticed that the ColorChecker is sensitive to small changes in the illumination's incident angle to the target, and whether the light source is diffused or directional. The placement of the control points in the DNG PE is also very sensitive, and I have on many occasions had varying blues when I move the control points around slightly. It does seem to be a sensitive color. Are you shooting the target illuminated by sunlight or by some artificial light source?

The sRGB reference from Danny Pascale is 43,63,147. I can get reasonable close for my camera, for daylight.

Can you upload a copy of the raw file you used in the DNG PE? I can try and give it a go and see if I can get better results.
Logged

solarj

  • Newbie
  • *
  • Offline Offline
  • Posts: 25
Re: Adobe DNG Profile Editor Does not work?
« Reply #4 on: February 27, 2013, 05:01:45 pm »

I have noticed that the ColorChecker is sensitive to small changes in the illumination's incident angle to the target, and whether the light source is diffused or directional. The placement of the control points in the DNG PE is also very sensitive, and I have on many occasions had varying blues when I move the control points around slightly. It does seem to be a sensitive color. Are you shooting the target illuminated by sunlight or by some artificial light source?

The sRGB reference from Danny Pascale is 43,63,147. I can get reasonable close for my camera, for daylight.

Can you upload a copy of the raw file you used in the DNG PE? I can try and give it a go and see if I can get better results.

That will be great, the colorchecker photo was shot in a cloudy day outside, it was shot with custom white balance with a grey card showed 6600K color temperature in camera

http://www.energain.se/photo/dsc00179acrdng.zip

And here is a standard NEX-5R profile included with ACR, it works so that blue patch becomes 56,61,150 (that is xrite reference value), and I think this is the best profile I can find so far
http://www.energain.se/photo/nex-5r_standard.zip

The most strange thing with DNG PE: If I open that colorchecker dng photo, and load this standard NEX-5R profile in DNG PE, do nothing, just export it to a new profile name, the newly generated profile will not provide the same color performance as the original one... I'm very confused by how many internal processing it did behind the scene
Logged

samueljohnchia

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 498
Re: Adobe DNG Profile Editor Does not work?
« Reply #5 on: February 27, 2013, 09:45:44 pm »

Firstly, doing custom white balance in camera is not doing to make any difference to the outcome. The DNG PE will white balance the image from the second gray patch in the CC and then perform the mapping calculations, all done internally and the user cannot control how this works, like say maybe do WB from the larger gray patch in a Passport instead (theoretically more accurate WB). So, don't waste your time on this step. UNIWB for ETTR is also not going to help because if the white patch gets clipped after multiplying the R and B channels to give correct WB, the DNG PE will also reject the file.

Basically your image is 1 and 1/3 stop underexposed. That is causing the issue that you are facing. Elsewhere in this forum, it was discovered that the DNG PE builds profiles that maps colors in a significantly less saturated manner, if the shot used to build the CC is underexposed. I had the same under-saturated problems myself when I first started.

Try to bracket a series of exposures with 1/3 stop exposure increments and then feed the DNE PE the brightest one that it doesn't reject. You should have much better results. Oh btw, try to stay as far away from that white wall in your shot as possible, and if you wern't already doing so, wear a black shirt when shooting the target.  :)

The most strange thing with DNG PE: If I open that colorchecker dng photo, and load this standard NEX-5R profile in DNG PE, do nothing, just export it to a new profile name, the newly generated profile will not provide the same color performance as the original one... I'm very confused by how many internal processing it did behind the scene

The Adobe Standard dcp profiles have value (luminosity adjustments) in the HueSatDelta LUTs already and don't require an internal tone curve, but the custom built profiles from the DNG PE chart function don't and need a generic curve (okay since most cameras are quite linear in their raw state). Doing what you did, would cause a base profile tone curve to be embedded in the profile, which was not present initially with the Standard series of profiles. The base tone curve applied on top of the LUT mapping is what is causing the slight color differences. They shouldn't be too far off - I mostly note a luminosity differece, particularly in the shadow regions. The DNG PE is not doing any other adjustments to the color mapping, as far as I can tell.
Logged

solarj

  • Newbie
  • *
  • Offline Offline
  • Posts: 25
Re: Adobe DNG Profile Editor Does not work?
« Reply #6 on: February 27, 2013, 10:12:23 pm »

Firstly, doing custom white balance in camera is not doing to make any difference to the outcome. The DNG PE will white balance the image from the second gray patch in the CC and then perform the mapping calculations, all done internally and the user cannot control how this works, like say maybe do WB from the larger gray patch in a Passport instead (theoretically more accurate WB). So, don't waste your time on this step. UNIWB for ETTR is also not going to help because if the white patch gets clipped after multiplying the R and B channels to give correct WB, the DNG PE will also reject the file.

Basically your image is 1 and 1/3 stop underexposed. That is causing the issue that you are facing. Elsewhere in this forum, it was discovered that the DNG PE builds profiles that maps colors in a significantly less saturated manner, if the shot used to build the CC is underexposed. I had the same under-saturated problems myself when I first started.

Try to bracket a series of exposures with 1/3 stop exposure increments and then feed the DNE PE the brightest one that it doesn't reject. You should have much better results. Oh btw, try to stay as far away from that white wall in your shot as possible, and if you wern't already doing so, wear a black shirt when shooting the target.  :)

The Adobe Standard dcp profiles have value (luminosity adjustments) in the HueSatDelta LUTs already and don't require an internal tone curve, but the custom built profiles from the DNG PE chart function don't and need a generic curve (okay since most cameras are quite linear in their raw state). Doing what you did, would cause a base profile tone curve to be embedded in the profile, which was not present initially with the Standard series of profiles. The base tone curve applied on top of the LUT mapping is what is causing the slight color differences. They shouldn't be too far off - I mostly note a luminosity differece, particularly in the shadow regions. The DNG PE is not doing any other adjustments to the color mapping, as far as I can tell.

Thanks, I just discovered that there is a tool called dcptool, now I can manually modify the dcp profile so all these questions are solved  :D :D

Logged

samueljohnchia

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 498
Re: Adobe DNG Profile Editor Does not work?
« Reply #7 on: February 27, 2013, 10:20:34 pm »

Thanks, I just discovered that there is a tool called dcptool, now I can manually modify the dcp profile so all these questions are solved  :D :D



Great. Just wondering - how would you know what new matrix values to use? Or how to modify the HueSatDelta LUTs to hit your target reference values?
Logged

solarj

  • Newbie
  • *
  • Offline Offline
  • Posts: 25
Re: Adobe DNG Profile Editor Does not work?
« Reply #8 on: February 27, 2013, 11:04:45 pm »

Great. Just wondering - how would you know what new matrix values to use? Or how to modify the HueSatDelta LUTs to hit your target reference values?

In order to calculate the matrix, I must have the white balance data, but that is not inside the dcp file, do you know where can I find it in dng file?

Logged

solarj

  • Newbie
  • *
  • Offline Offline
  • Posts: 25
Re: Adobe DNG Profile Editor Does not work?
« Reply #9 on: February 27, 2013, 11:47:35 pm »

Some findings:

If I open the ACR Standard profile in DNG PE and export it without doing any change, it does several things:
1. Keep all the matrices intact
1. Change ProfileLookTableEncoding from 1 to 0 (sRGB encoding to linear encoding)
2. Change the tone curve in ACR standard profile to one of the built-in tone curve of NDG PE
3. Set BaselineExposureOffset to 0
5. Set DefaultBlackRender from 1 to 0 (disabled to auto)

Personaly I don't like LUT based approach, so I will try to find a way to get rid of that LUT and see what it the result
« Last Edit: February 28, 2013, 12:28:45 am by solarj »
Logged

samueljohnchia

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 498
Re: Adobe DNG Profile Editor Does not work?
« Reply #10 on: February 28, 2013, 03:18:43 am »

Some findings:

If I open the ACR Standard profile in DNG PE and export it without doing any change, it does several things:
1. Keep all the matrices intact
1. Change ProfileLookTableEncoding from 1 to 0 (sRGB encoding to linear encoding)
2. Change the tone curve in ACR standard profile to one of the built-in tone curve of NDG PE
3. Set BaselineExposureOffset to 0
5. Set DefaultBlackRender from 1 to 0 (disabled to auto)

Personaly I don't like LUT based approach, so I will try to find a way to get rid of that LUT and see what it the result

Very interesting. I cannot see the ProfileLookTableEncoding, BaselineExposureOffset and DefaultBlackRender in the xml version of the decompiled profile with Sandy's dcpTool. May I know how you are able to locate these tags?
Logged

samueljohnchia

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 498
Re: Adobe DNG Profile Editor Does not work?
« Reply #11 on: February 28, 2013, 05:46:29 am »

In order to calculate the matrix, I must have the white balance data, but that is not inside the dcp file, do you know where can I find it in dng file?

I don't know either. I think the white balance values are calculated on the fly from the gray patch in the CC by the DNG PE, but not stored in the profile. The profile only contains the illuminant and bradford transform. I'm not entirely sure and may be wrong.
Logged

solarj

  • Newbie
  • *
  • Offline Offline
  • Posts: 25
Re: Adobe DNG Profile Editor Does not work?
« Reply #12 on: February 28, 2013, 03:40:37 pm »

Very interesting. I cannot see the ProfileLookTableEncoding, BaselineExposureOffset and DefaultBlackRender in the xml version of the decompiled profile with Sandy's dcpTool. May I know how you are able to locate these tags?

you can analyze the standard 5R dcp profile I provided above in the link, it is included by ACR, not generated by DNG PE or Xrite software, all of them have different content  :-\

Now I'm testing the simplest dcp profile generated by xrite software, they use only one light and one colormatrix, with 6x6x3 HSD table, but strangely the whitebalance reading (as shot) changed  from 6600k to 6450k after applying their dcp profile

But the whitebalance reading (as shot) do not change when I apply the standard ACR dcp profile, is there any settings hidden? Still lot's of confusion here...
Logged

samueljohnchia

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 498
Re: Adobe DNG Profile Editor Does not work?
« Reply #13 on: February 28, 2013, 09:08:37 pm »

you can analyze the standard 5R dcp profile I provided above in the link, it is included by ACR, not generated by DNG PE or Xrite software, all of them have different content  :-\

Now I'm testing the simplest dcp profile generated by xrite software, they use only one light and one colormatrix, with 6x6x3 HSD table, but strangely the whitebalance reading (as shot) changed  from 6600k to 6450k after applying their dcp profile

But the whitebalance reading (as shot) do not change when I apply the standard ACR dcp profile, is there any settings hidden? Still lot's of confusion here...

I had the old version of the dcpTool. Now I see what you are talking about. Different profiles are treated differently? Hmm my 5D mark II profiles are ProfileLookTableEncoding 0 and DefaultBlackRender 0 to begin with, and the DNG PE does no change to that. The X-rite software profile does not change my as shot white balance to something else. Something is amiss and I don't know what...yeah its not entirely clear to me either.
Logged

solarj

  • Newbie
  • *
  • Offline Offline
  • Posts: 25
Re: Adobe DNG Profile Editor Does not work?
« Reply #14 on: March 01, 2013, 12:36:18 am »

Finally generated a nice profile  ;D ;D ;D


This is so far the most precise color checker photo I get from a outdoor shot, all the channels for all the color patch can get as close to babel color reference value as possible, typical channel error is 3-5, only one dark skin patch diviate more than 5, and I think that has something to do with the curve

Normally human eyes will not notice a color difference if the channel error is less than 20, so this gave me lots of room for after processing without generate observable change in color appearance

And the solution is super stupid easy, use dcptool to remove all the tables in a given profile and in DNG PE load that profile and create color table, and use a third party color meter (colorpic) inspect the on screen srgb value when adjust color tables

But whitebalance value still changed, I don't know why it does not keep the original white balance value, this might not be the final solution, but anyway I know there is a way to generate 99% precise profile

Logged

samueljohnchia

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 498
Re: Adobe DNG Profile Editor Does not work?
« Reply #15 on: March 01, 2013, 02:02:38 am »

I noticed that while all the color patches are nearly spot on, the grayscale patches are not.

Try again using process 2010 in ACR, and set the tone curve to linear, brightness to 0, contrast to 0 and blacks to 0. This is ACR's "linear" mode. Then raise the exposure slider to about +1.33 to match the white patches. You would likely find that the rest of the grayscale patches will follow and be almost matching in tone. You'll probably find then that your color patches are off.
Logged

solarj

  • Newbie
  • *
  • Offline Offline
  • Posts: 25
Re: Adobe DNG Profile Editor Does not work?
« Reply #16 on: March 01, 2013, 02:29:38 am »

I noticed that while all the color patches are nearly spot on, the grayscale patches are not.

Try again using process 2010 in ACR, and set the tone curve to linear, brightness to 0, contrast to 0 and blacks to 0. This is ACR's "linear" mode. Then raise the exposure slider to about +1.33 to match the white patches. You would likely find that the rest of the grayscale patches will follow and be almost matching in tone. You'll probably find then that your color patches are off.

Yes, I have noticed this long time ago, and this is just a first test. Since the curves in profile normally consists of more than 90 points and the color checker only have 6 points, it is not very easy to get such a smooth curve in DNG PE, but I will try later

Anyway the tone curve will not change the scaling of 3 channels of each color patch, it is not as critical as the color mapping
Logged

samueljohnchia

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 498
Re: Adobe DNG Profile Editor Does not work?
« Reply #17 on: March 01, 2013, 02:44:33 am »

Yes, I have noticed this long time ago, and this is just a first test. Since the curves in profile normally consists of more than 90 points and the color checker only have 6 points, it is not very easy to get such a smooth curve in DNG PE, but I will try later

Anyway the tone curve will not change the scaling of 3 channels of each color patch, it is not as critical as the color mapping

The tone curve embedded in the profile - yes. But the tone curve and other sliders like contrast in ACR are going to introduce hue and saturation shifts.

I would just take another shot of the CC but properly exposed, and run it through the DNG PE. I get as close as you do to the color patches, but also have the gray patches match the reference values without all this fussing around with editing.
Logged

solarj

  • Newbie
  • *
  • Offline Offline
  • Posts: 25
Re: Adobe DNG Profile Editor Does not work?
« Reply #18 on: March 01, 2013, 10:05:16 am »

The tone curve embedded in the profile - yes. But the tone curve and other sliders like contrast in ACR are going to introduce hue and saturation shifts.

I would just take another shot of the CC but properly exposed, and run it through the DNG PE. I get as close as you do to the color patches, but also have the gray patches match the reference values without all this fussing around with editing.

My exposure standard is to have the green chanel of the green patch to get a value close to 150. And I had multiple different exposed photo sent into DNG PE and xrite software, almost all of them generated same profile, so I think exposure do not affect the process, the calibration software will automatically lower or higher the exposure when it process the photo, as long as it does not over expose too much

I think it is impossible to generate a 99% matching profile without LUT, the sensor will never be as perfect as human eyes. I also use a foveon sensor DP1 with customized matrix change, I can get most of the channel error below 6 with a good matrix, but some patches will have an error as high as 20, since there is no way to use LUT

If the sensor and matrix are good enough, the changes in LUT should be small (less than 5 change on each slider). Currently for some of the patch I need to change the slider by as much as 10, that indicated an imperfection of the sensor / matrix combination, I need to test with other matrices
Logged

solarj

  • Newbie
  • *
  • Offline Offline
  • Posts: 25
Re: Adobe DNG Profile Editor Does not work?
« Reply #19 on: March 01, 2013, 01:06:05 pm »

I noticed that while all the color patches are nearly spot on, the grayscale patches are not.

Try again using process 2010 in ACR, and set the tone curve to linear, brightness to 0, contrast to 0 and blacks to 0. This is ACR's "linear" mode. Then raise the exposure slider to about +1.33 to match the white patches. You would likely find that the rest of the grayscale patches will follow and be almost matching in tone. You'll probably find then that your color patches are off.

Just tried this method, as you said, without any curve applied, the grayscale patches will match more or less correct, and color patch now performs much better with profiles generated by DNG PE or xrite software chart wizard, but there are still big errors on blue patch, and channel values of some color patches are just too high, by almost 20

And this is a question: If I could achieve a perfect reproduction of colorchecker photo at linear mode, but all the photos looks very dull at linear mode, then maybe it is better to calibrate the colorchecker against a curve adjusted mode which is more frequently used daily?





« Last Edit: March 01, 2013, 01:08:21 pm by solarj »
Logged
Pages: [1] 2   Go Up