DNG Profile Editor confusion with Sony RX10.

[Redcrown]

Got a new camera for my wife, a Sony RX10. That's a high end P&S (now called a "Bridge" camera). So I'm profiling it with the Adobe DNG Profile Editor (DNGpe). I've had a lot of experience with profiling, but it's been a few years since the last time. All prior experience was with Canon (1Ds, 5D2, 5D3).

I'm getting some strange results with the Sony that I don't understand. I'm hoping someone can help clear up that confusion.

I made bracketed shots of my ColorChecker under different lights. I generated 3 DNGpe profiles, one for each shot in the bracketed set. When I apply these profiles to any one image in the set I get different results. I'm used to minor differences between theoretically identical profiles, but in this case the differences are significant. That's my confusion. Why do profiles made from slightly different exposures generate significantly different results?

The attached image shows different results from different profiles applied to the same image.

When I bring my three bracketed shots into ACR using the Adobe Standard profile and Adobe98 colorspace I see RGB values for the White and Black squares of 240/48, 228/36, and 200/25. Those should all be good exposures. No clipping on either end. The light was even, 5600K. I generated single illuminant profiles using Adobe Standard as the base profile and the 6500K color table. I closed and relaunched DNGpe between each run to make sure I didn't carry over any previous settings.

On this sample, the base image is the one exposed at 228 with the 228 profile applied. I then converted the same image under the 200 profile and the 240 profile, layered them on top the base, and masked them in with circles. The circles on the top left of each square are the results of the 240 profile (applied to the 228 image). The bottom right circles are the result of applying the 200 profile.

I used my color sampler in HSB mode to look at the differences. I noticed that the Hue and Brightness values are about the same. Only 1 or 2 point differences. But the Saturation values move 5% to 15%. The brighter 240 profile increases saturation while the darker 200 profile lowers saturation. On the blue square, the 240 profile drives the saturation to 100%.

Can anyone offer and explanation, or critique my process?

P.S. I did everything in Adobe98 without remembering I had to convert to sRGB for posting. So my Adobe98 numbers above won't match the sRGB sample below. Sorry about that.

[ErikKaffehr]

Hi,

I would think about hue twists, as described here: http://chromasoft.blogspot.se/2009/02/visualizing-dng-camera-profiles-part-1.html

Now, this article indicates that DNG Profile Editor may not yield "hue twists": https://www.hansvaneijsden.com/colorchecker-perfect-skin-colors/

But we don't know what DNG Profile Editor does. I can use DcpTool to look into that, but I don't really have the time to do it right now.

This write up may be of some interest (it is a long and uncomplete): http://echophoto.dnsalias.net/ekr/Articles/OLS_OnColor/OnColor.pdf

Update:

I tested "untwisting" two of my DNG PE profiles with dcpTool and it says they have no hue twists.

What about posting your DNG images and the resulting dcp profiles?

Best regards
Erik

Got a new camera for my wife, a Sony RX10. That's a high end P&S (now called a "Bridge" camera). So I'm profiling it with the Adobe DNG Profile Editor (DNGpe). I've had a lot of experience with profiling, but it's been a few years since the last time. All prior experience was with Canon (1Ds, 5D2, 5D3).

I'm getting some strange results with the Sony that I don't understand. I'm hoping someone can help clear up that confusion.

I made bracketed shots of my ColorChecker under different lights. I generated 3 DNGpe profiles, one for each shot in the bracketed set. When I apply these profiles to any one image in the set I get different results. I'm used to minor differences between theoretically identical profiles, but in this case the differences are significant. That's my confusion. Why do profiles made from slightly different exposures generate significantly different results?

The attached image shows different results from different profiles applied to the same image.

When I bring my three bracketed shots into ACR using the Adobe Standard profile and Adobe98 colorspace I see RGB values for the White and Black squares of 240/48, 228/36, and 200/25. Those should all be good exposures. No clipping on either end. The light was even, 5600K. I generated single illuminant profiles using Adobe Standard as the base profile and the 6500K color table. I closed and relaunched DNGpe between each run to make sure I didn't carry over any previous settings.

On this sample, the base image is the one exposed at 228 with the 228 profile applied. I then converted the same image under the 200 profile and the 240 profile, layered them on top the base, and masked them in with circles. The circles on the top left of each square are the results of the 240 profile (applied to the 228 image). The bottom right circles are the result of applying the 200 profile.

I used my color sampler in HSB mode to look at the differences. I noticed that the Hue and Brightness values are about the same. Only 1 or 2 point differences. But the Saturation values move 5% to 15%. The brighter 240 profile increases saturation while the darker 200 profile lowers saturation. On the blue square, the 240 profile drives the saturation to 100%.

Can anyone offer and explanation, or critique my process?

P.S. I did everything in Adobe98 without remembering I had to convert to sRGB for posting. So my Adobe98 numbers above won't match the sRGB sample below. Sorry about that.

[Redcrown]

Thanks for the thoughts,

I tried DcpTool conversions and, like you, I found that DcpTool claimed that DNGpe profiles were already untwisted.

I had seen similar issues when profiling a Canon 5D2 and 5D3 a few years ago. Profiles generated from different exposures gave different results. But back then, the differences were barely visible thus hardly noticeable.

I never understood that. And now, with the Sony RX10 images, I'm baffled. Having such significant differences from exposures that are less than 1 stop apart seems to cast big doubts on the quality of DNGpe profiles.

When DNGpe first came out, people asked how to best expose the chart. I recall Adobe's Eric Chan saying that exposure was not critical as long as there was no clipping. He said it was more important to make sure your light was even across the chart. Since then, I've read somewhere in the DNGpe documentation that the recommended exposure is one with a LAB L value of 96% on the white square. That would equate to Adobe98 RGB values of 243. My shot at 240 (L=95%) is very close, yet it's the profile from that exposure that gives high saturation, and even clips the red value on the blue square.

So, if anybody else is listening, which of my three profiles do you think is most correct?

[ErikKaffehr]

Hi,

The Lab value for the white patch on the ColorChecker is 96,0,0  next one is 81,0,0 if I recall correctly.

I have noticed that different converters and profiles push around colours and saturations a lot. I guess I will look a bit more into this. I made an experiment comparing Adobe Standard Profiles with untwisted profiles and could see some shifts between exposures. But your case is different.

Adobes Process version 2012 is making a few odd things. PV 2010 can make a linear conversion. With black levél set to zero it reproduces color checker gray values very well. PV 2014 can hide a lot of overexposure.

Best regards
Erik

Thanks for the thoughts,

I tried DcpTool conversions and, like you, I found that DcpTool claimed that DNGpe profiles were already untwisted.

I had seen similar issues when profiling a Canon 5D2 and 5D3 a few years ago. Profiles generated from different exposures gave different results. But back then, the differences were barely visible thus hardly noticeable.

I never understood that. And now, with the Sony RX10 images, I'm baffled. Having such significant differences from exposures that are less than 1 stop apart seems to cast big doubts on the quality of DNGpe profiles.

When DNGpe first came out, people asked how to best expose the chart. I recall Adobe's Eric Chan saying that exposure was not critical as long as there was no clipping. He said it was more important to make sure your light was even across the chart. Since then, I've read somewhere in the DNGpe documentation that the recommended exposure is one with a LAB L value of 96% on the white square. That would equate to Adobe98 RGB values of 243. My shot at 240 (L=95%) is very close, yet it's the profile from that exposure that gives high saturation, and even clips the red value on the blue square.

So, if anybody else is listening, which of my three profiles do you think is most correct?

[Peter_DL]

I made bracketed shots of my ColorChecker under different lights. I generated 3 DNGpe profiles, one for each shot in the bracketed set. When I apply these profiles to any one image in the set I get different results. I'm used to minor differences between theoretically identical profiles, but in this case the differences are significant.

Similar finding here.

From what I see from the screenshot above, the profile from the lower exposed shot delivers less saturated colors
(see my post #11).

--

[AlterEgo]

I recall Adobe's Eric Chan saying that exposure was not critical as long as there was no clipping.

he did and later in the same thread he noted

Eric Chan :: ( http://www.luminous-landscape.com/forum/index.php?topic=70762.msg561908#msg561908 ) @  September 26, 2012 :

    

The DNG PE needs to be somewhat conservative in its chart wizard with regards to clipping because color tables are always applied after white balance.  Your raw capture may not have clipped data in its native coordinate system (native RGB, without WB applied), but that data can become clipped after WB is applied.  That leads to problems with the color mapping.  DNG PE will generally detect this case and prevent you from proceeding. My recommendation is simply to bracket exposures and then pick the brightest one that DNG PE will accept without giving you an error.

[Redcrown]

Thanks to Peter_DL and AlterEgo for pointing to older threads on this issue. I probably read those in the past, but needed the reminder. I've spent the past few days reviewing those and others, plus expanding, repeating, and reviewing my own tests.

I remain curious why this issue has received so little attention, and even seems to be ignored or considered trivial by the more famous gurus (Schew, Chan, Rodney, etc.). To me, it appears that how you expose a colorchecker is a critical factor.

My tests repeatedly show that higher exposures of the colorchecker generate higher saturation when applied to raw images.

I made fresh exposures of the colorchecker with my Canon 5D3, bracketing so I had LAB-L values off the white patch that ranged in close increments from 81 to 99. The DNG Profile editor rejected the one with a value of 98 and accepted all others. I verified with RawDigger that the 98 version had clipping. It was only 0.3% in the green channels. The 96 version showed no clipping in RawDigger.

So, following Eric Chan's advice, my best exposure - the highest exposure accpeted by the profile editor - would be the 96 version. When I apply that version to a wide variety of real world images I find that sometimes I like it better, sometimes not. The difference depends on whether the image has naturally bright colors or not.

For example, on a dull sky with weark blues, the 96 version is great. Brings out more blue. But on a great sky with strong blues, the 96 version pushes them too far and gives a garish, un-natural look. Ditto for reds. If they start out weak, the 96 version makes them stronger, which is good. But if they start out strong, the 96 version goes too far.

The 96 version often clips some RGB values to 0 within the Adobe98 colorspace. Strong blues have a 0 red channel. Strong reds have a 0 green channel. No problem in ProPhoto, of course.

Another curious observation - my 92 version matches the Adobe Standard profile almost exactly. I'm wondering if that means Adobe created the standard profile from a "middle" exposure of the colorchecker. 

Here is a table and graph I made to map the saturation values from some real world images under DNGPE profiles made at various exposures. The blue was from a blue sky, the red from a very red shirt, the green from some grass.



Look at the slopes of those curves. The blues ramp up the most, followed by reds, then greens last.

My "temporary" conclusion: I'm better off using profiles made from a colorchecker exposure with a white patch LAB-L value of 92 to 94. That profile gives good saturation without pushing some values to the edge, and leaving a little wiggle room for other post production adjustments.

I'm at a stage in life where I often shoot a pack of grandkids. Their parents dress them in the American style, which means they are always wearing bright colors. When they show up, I tell my wife to get my sun glasses because the "day-glo" gang is here.

They give me the most difficult color management challenges I face. I usually need to back off the colors. Using a profile that pushes those colors to the max before I can get my hands on them is not a good thing.

[AlterEgo]

...

so have 2 (two) profiles because you start to talk about your taste and that's where when somebody might as well start liking profiles made from severely underexposed raws.. because it's about taste.