Pages: [1] 2   Go Down

Author Topic: Same DcamProf-ile, Different Converter Colors?  (Read 6254 times)

Jack Hogan

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 798
    • Hikes -more than strolls- with my dog
Same DcamProf-ile, Different Converter Colors?
« on: December 15, 2016, 06:34:51 am »

Hello folks,

I've been playing with DcamProf single illuminant, noon daylight.  Any suggestions as to why the colors produced by ACR look so different from RawTherapee's when using the same single matrix .dcp neutral+ 'look' profile (obtained from a cc24 captured at the same time as the desired image), same white balance and everything (possible) turned off?

Here is a rough example, screen capture of 200% AdobeRGB image, left to right:
  • matlab matrix+HSmap+lookTable from DNG converted raw file;
  • ACR CC directly from raw file
  • RawTherapee 4.2.1.375 (Base and Look table checked, directly from raw file)
  • matlab matrix only, from DNG converted raw file.



I was trying to figure out why my routine (1) produces results that look quite different from RT's (3) after the application of HSmap and lookTable - when I realized that RT looks quite different from ACR.  Which can be used as a reference?

Jack
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: Same DcamProf-ile, Different Converter Colors?
« Reply #1 on: December 15, 2016, 11:01:25 am »

I'm not sure I understand what I'm looking at, are you using the same DCamProf-generated DCP in ACR CC and RawTherapee?

A signature difference between Adobe's default profiles and a DCamProf profile with the neutral tone reproduction operator is that Adobe's skies are a bit under-saturated while DcamProf keeps more of the blue. So from the image it seems like you're not using the same DCP.

If you are using the same DCP I can't really explain the difference. RawTherapee doesn't have black level subtraction like Adobe has, which causes lighter shadows in RT if the DCP doesn't have the blackrender none flag set (default for a DCamProf DCP), but otherwise the rendering should be the same for normally exposed images. For overexposed images ACR does some extra tricks to simulate overexposed film, which RT doesn't do.

Looking at the wooden details it seems like the ACR is warmer, that the white balance is still not the same. That could be the reason, but saturation shouldn't differ so much from that, it's a bit hard to tell, I'm not at a calibrated screen for the moment. A different white balance can be due to the white balance shift issue with DCP, discussed here:
https://www.ludd.ltu.se/~torger/dcamprof.html#dcp_wb_shift
If you have the white balance setting at "as shot" or if you have used a wb picker in ACR *after* you changed profile, you shouldn't have WB shift problems, or if you have copied the color matrices from the Adobe Default profile, which I recommend to ACR users.
Logged

Jack Hogan

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 798
    • Hikes -more than strolls- with my dog
Re: Same DcamProf-ile, Different Converter Colors?
« Reply #2 on: December 15, 2016, 12:35:55 pm »

I'm not sure I understand what I'm looking at, are you using the same DCamProf-generated DCP in ACR CC and RawTherapee?

Hi Torger,

Yes, the first three images are all supposedly applying the same .dcp profile generated by DcamProf (off cc24, with neutral+ operator).

Quote
A signature difference between Adobe's default profiles and a DCamProf profile with the neutral tone reproduction operator is that Adobe's skies are a bit under-saturated while DcamProf keeps more of the blue. So from the image it seems like you're not using the same DCP.

Hmm, everything is possible (especially when I am involved:-) but I did select the same profile in both programs.

Quote
If you are using the same DCP I can't really explain the difference. RawTherapee doesn't have black level subtraction like Adobe has, which causes lighter shadows in RT if the DCP doesn't have the blackrender none flag set (default for a DCamProf DCP), but otherwise the rendering should be the same for normally exposed images. For overexposed images ACR does some extra tricks to simulate overexposed film, which RT doesn't do.

Thanks for that, if the two images are supposed to look the same I must have something wrong with the setup since I don't think this image should be classified as overexposed.

Quote
Looking at the wooden details it seems like the ACR is warmer, that the white balance is still not the same. That could be the reason, but saturation shouldn't differ so much from that, it's a bit hard to tell, I'm not at a calibrated screen for the moment. A different white balance can be due to the white balance shift issue with DCP, discussed here:
https://www.ludd.ltu.se/~torger/dcamprof.html#dcp_wb_shift
If you have the white balance setting at "as shot" or if you have used a wb picker in ACR *after* you changed profile, you shouldn't have WB shift problems, or if you have copied the color matrices from the Adobe Default profile, which I recommend to ACR users.

Ah, I think I did pick WB on the cc24 image and synchronized before changing profile (though I don't understand why that would make a difference).  Here they are now, having changed profile before picking WB, both sRGB for simplicity.  Strangely ACR now indicates 5100K vs 5350K before (which agreed with RT).  Colors are much closer now, with obvious exceptions the blues and the handling of out of gamut red (see flowers at the bottom), RT on left and ACR on right:



Is this correct or should they look the same now?

Thanks for your input.
Jack
Logged

Jack Hogan

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 798
    • Hikes -more than strolls- with my dog
Re: Same DcamProf-ile, Different Converter Colors?
« Reply #3 on: December 15, 2016, 12:53:47 pm »

Just in case anyone is interested the following link downloads two NEFs (cc24 and image) and the .dcp profile used above. 58MB.

Jack
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: Same DcamProf-ile, Different Converter Colors?
« Reply #4 on: December 15, 2016, 02:20:26 pm »

"True" white balance for a digital camera is multipliers for R, G and B. If you have the exact same multipliers in two software you get the exact same white balance. Few softwares allow you to set the multipliers directly though, as it would be a too technical interface.

Instead you usually set white balance with "temperature and tint". Some raw converters convert those to multipliers and store those, and some, like ACR, store the temperature and tint rather than multipliers. The problem is that there is no standard formula to convert multipliers to temp/tint, so 5000/+10 in one raw converter may not be the same as 5000/+10 in another.

With ACR it's even "worse", as temp/tint is derived using the color matrices in the DNG profile. So if you change profile the temp/tint derivation will change, as color matrices are not exactly the same. As ACR store temp/tint rather than the multipliers when you change profile the multipliers change, but temp/tint stays the same. However, if you have white balance "As Shot" ACR actually use the *multipliers* stored in the raw file, and thus when you change profile white balance stays the same, but temp/tint change. And of course if you use a wb picker the white balance will be set after that patch, but if you change profile after that, the wb is set from temp/tint and you will have a shift so you need to use the wb picker again.

Perhaps one would think as the profile is calculated for the same camera the color matrices would be "close enough" that this would not be an issue. But cameras are far from exact color temperature measurement devices, and algorithms of how to calculate the color matrices differ between software so you can easily get a few hundred degrees difference, and certainly a visible difference.

A work-around for ACR users is to copy the ACR color matrices into DCamProf profiles, which you can do using the -m parameter to make-dcp command.

As the color matrices are only used for temp/tint calculation and not the actual color rendering (forward matrices are used for that) you won't hurt color.

If you ask me I think it would have been better design by Adobe if they'd always store the multipliers instead so changing profile never could cause a white balance shift, but it is the way it is.

To make sure RT and ACR renders the same you need to make a DNG and let RT use that, as black levels and white levels may differ a bit between ACR's and RT's raw format translation. I don't think it should be an issue for the NEF though, but it can be good to do as a precaution. Black render we have already discussed, then there's the output profile and clip handling which you have noted.

I still think the difference in color of the blue sky is a bit strange, and the flat colors of the rock and the closest door doesn't seem right, it's as if the ACR screenshot doesn't reach to full clipping. I would suspect some issue with output profile. Note the yellowy detail in the center of the bottom edge of the image, it's clearly clipped in RT, but just flat gray in ACR.

Also note that when making screenshots the use or not use of display profile can cause unexpected results.

I haven't done any recent comparison between ACR and RT, but my memory tells me that they should be much closer than your images show when the same normally exposed DNG is used with the same DCP (with blackrender=none flag), using the same white balance, and the same output profile.
Logged

sankos

  • Newbie
  • *
  • Offline Offline
  • Posts: 36
  • Why matters
Re: Same DcamProf-ile, Different Converter Colors?
« Reply #5 on: December 15, 2016, 04:21:41 pm »

I noticed some differences between LR and RT when using the same dcp profile some time ago. First it was with DCamprof profiles that I made, but then I used the same Adobe Standard dcp to eliminate any mistakes I could have made when creating the profiles. In my case the RT photos are undersaturated when compared with LR, so I need to compensate for that in my post-processing. I couldn't find an explanation for this other than it must be something in RT's processing pipeline -- maybe the AMaZe demosaicing? because that's different between the two apps but I really don't know.
« Last Edit: December 15, 2016, 04:45:08 pm by sankos »
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: Same DcamProf-ile, Different Converter Colors?
« Reply #6 on: December 16, 2016, 03:42:15 am »

I noticed some differences between LR and RT when using the same dcp profile some time ago. First it was with DCamprof profiles that I made, but then I used the same Adobe Standard dcp to eliminate any mistakes I could have made when creating the profiles. In my case the RT photos are undersaturated when compared with LR, so I need to compensate for that in my post-processing. I couldn't find an explanation for this other than it must be something in RT's processing pipeline -- maybe the AMaZe demosaicing? because that's different between the two apps but I really don't know.

I guess I need to make some comparisons myself again, because I don't think there should be such differences. I have made A/B comparisons in the past but it's more than a year ago. I was deeply involved developing RT's DCP support, but I'm not as active in RT's development now and much has happened since...

If the profile lacks the black render=none flag, and lacks a curve, the difference will be huge as RT doesn't apply a default curve and no black subtraction like LR does. But otherwise it should look same when no adjustments are applied, other than really minor things such as the white point may differ 0.1-0.2 in exposure or so.
Logged

Jack Hogan

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 798
    • Hikes -more than strolls- with my dog
Re: Same DcamProf-ile, Different Converter Colors?
« Reply #7 on: December 16, 2016, 04:49:44 am »

Ok, thanks for that explanation and suggestion Torger.  I will try the dng on RT when I have some time this weekend.  BTW, the image was captured by a D610 and Nikon subtracts blacks from it before writing data to the raw file.

Jack
Logged

sankos

  • Newbie
  • *
  • Offline Offline
  • Posts: 36
  • Why matters
Re: Same DcamProf-ile, Different Converter Colors?
« Reply #8 on: December 16, 2016, 06:29:33 am »

I've done a fresh comparison, this time both with a nef and a dng converted from it. It looks like checking the DCP look table in RT desaturates my photo (it compresses the blue channel in the 3/4 of the histogram mainly). Other than that, as Anders said, RT produces lighter shadows (less contrast) as compared with ACR. Colours look the same between ACR and RT with the DCP tone curve checked, look table unchecked and Camera WB (I've chosen the RelCol intent in RT output options -- I'm not sure what rendering intent is used by ACR?).
Logged

sankos

  • Newbie
  • *
  • Offline Offline
  • Posts: 36
  • Why matters
Re: Same DcamProf-ile, Different Converter Colors?
« Reply #9 on: December 16, 2016, 07:44:19 am »

Concerning the WB differences -- I've noticed that if you switch on the demosaicing CA auto-correction in RT (in the Raw tab, not in the Transform) the image gets slightly less warm, so maybe the OP had this option turned on when doing the comparison?
Logged

Jack Hogan

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 798
    • Hikes -more than strolls- with my dog
Re: Same DcamProf-ile, Different Converter Colors?
« Reply #10 on: December 16, 2016, 02:48:15 pm »

Thanks for checking that sankos.  I am pretty sure that CA was unchecked.   I don't think I have the tone curve turned on (it looks like Adobe's base curve which is way too bright so I removed it with DNG Profile Editor), I wonder if this could be the problem?
« Last Edit: December 16, 2016, 02:56:57 pm by Jack Hogan »
Logged

Tim Lookingbill

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2436
Re: Same DcamProf-ile, Different Converter Colors?
« Reply #11 on: December 16, 2016, 07:55:27 pm »

Jack, again, your png screenshot doesn't have an embedded profile. Screenshot png color is heavily influenced by the gamut of your display recorded into the display's ICC profile which needs to be assigned and converted to sRGB for online display. My Mac OS automatically tags/assigns my display profile to my png screenshots which I have a choice to convert to sRGB or just leave as is in hope it is used by online browsers for color managed previews. I don't know what OS you're using so this is just FYI.

Now, aside from that, just based on my observation and speculation have you considered ACR's color engine is different in the way it interprets demosaiced sensor color data as the cause for the differences you see over other Raw converters who use different demosaicing algorithms and color definers with the added complexity of how each integrate ICC color managed Raw previews?

I also noticed a change in color when I switched to PV2012 in LR4 and rebuilt my camera profiles using updated DNG converter (and updated DNG Profile Wizard) from profiles I made from old DNG converters/Wizard in CS3 PV2003. What I noticed in the newer profile generated previews in LR4 is they closely resembled the more neutral, less saturated look (more pinkish Caucasian skin tones) of Adobe Standard with minor tweaks to hue/sat color tables. It wasn't a desirable result especially in skin tones so I went back to the old profiles which of course made flowers over saturated.
« Last Edit: December 16, 2016, 08:27:59 pm by Tim Lookingbill »
Logged

Jack Hogan

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 798
    • Hikes -more than strolls- with my dog
Re: Same DcamProf-ile, Different Converter Colors?
« Reply #12 on: December 17, 2016, 03:32:52 am »

Jack, again, your png screenshot doesn't have an embedded profile. Screenshot png color is heavily influenced by the gamut of your display recorded into the display's ICC profile which needs to be assigned and converted to sRGB for online display. My Mac OS automatically tags/assigns my display profile to my png screenshots which I have a choice to convert to sRGB or just leave as is in hope it is used by online browsers for color managed previews. I don't know what OS you're using so this is just FYI.

Hi Tim

I am using Win 7 and I am aware of the effect of the ICC profile, which however I think should affect color from all images in the same way when displayed on and 'cut' from the same monitor.

Quote
Now, aside from that, just based on my observation and speculation have you considered ACR's color engine is different in the way it interprets demosaiced sensor color data as the cause for the differences you see over other Raw converters who use different demosaicing algorithms and color definers with the added complexity of how each integrate ICC color managed Raw previews?

I also noticed a change in color when I switched to PV2012 in LR4 and rebuilt my camera profiles using updated DNG converter (and updated DNG Profile Wizard) from profiles I made from old DNG converters/Wizard in CS3 PV2003. What I noticed in the newer profile generated previews in LR4 is they closely resembled the more neutral, less saturated look (more pinkish Caucasian skin tones) of Adobe Standard with minor tweaks to hue/sat color tables. It wasn't a desirable result especially in skin tones so I went back to the old profiles which of course made flowers over saturated.

From what I understand of demosaicing, global tones should not be affected.  Process Versions are instead clearly right in the thick of it because they are based on the changing DNG spec, which in fact substantially changed color handling at each step of the way (v1.1, v1.2, and the current v1.4 referring to the various PVs, the last one being PV2012).

Thanks for your input,
Jack
Logged

Jack Hogan

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 798
    • Hikes -more than strolls- with my dog
Re: Same DcamProf-ile, Different Converter Colors?
« Reply #13 on: December 17, 2016, 11:02:16 am »

Ok, got it.  It was indeed the fact that I disabled the tone curve.  Here are RT (left) and ACR (right), both first converted to dng, with negative EC of 0.75 and 1.05 respectively to compensate for the aggressive tone curve.  They are now very, very similar:



Thanks all for your help.

Jack
Logged

Jack Hogan

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 798
    • Hikes -more than strolls- with my dog
DNG Converted Embedded Profile?
« Reply #14 on: December 17, 2016, 11:10:19 am »

While I have your attention, I have noticed an annoying habit of DNG Converter 9.8 (but the 'feature' was present in earlier versions as well): it loads in the converted DNG file matrix and profile information from a custom .dcp profile in the ...\CameraRaw\CameraProfiles\ directory, I can't tell how it chooses which one to use if more than one is present.  The only way that I have found to have it load default profile information (I assume Adobe Standard) is to erase all .dcp profiles from that directory.  However that is a real pain, because one then looses them for ACR/LR use.

Any idea how to tell DNG Converter to go with its default profile and not use one of the custom ones?  Or even better, how to tell it if and which custom profile to load?  The Adobe forum has not been helpful on this issue so far.

Jack
Logged

scyth

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 584
Re: DNG Converted Embedded Profile?
« Reply #15 on: December 17, 2016, 12:19:20 pm »

While I have your attention, I have noticed an annoying habit of DNG Converter 9.8 (but the 'feature' was present in earlier versions as well): it loads in the converted DNG file matrix and profile information from a custom .dcp profile in the ...\CameraRaw\CameraProfiles\ directory, I can't tell how it chooses which one to use if more than one is present.  The only way that I have found to have it load default profile information (I assume Adobe Standard) is to erase all .dcp profiles from that directory.  However that is a real pain, because one then looses them for ACR/LR use.

Any idea how to tell DNG Converter to go with its default profile and not use one of the custom ones?  Or even better, how to tell it if and which custom profile to load?  The Adobe forum has not been helpful on this issue so far.

Jack

you can simply use ACR to convert to DNG instead of DNG converter.
Logged

Jack Hogan

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 798
    • Hikes -more than strolls- with my dog
Re: Same DcamProf-ile, Different Converter Colors?
« Reply #16 on: December 17, 2016, 01:18:48 pm »

Without demosaicing?  How does one do that? I still need the data to be raw, just in DNG format so that matlab will read it.
Logged

scyth

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 584
Re: Same DcamProf-ile, Different Converter Colors?
« Reply #17 on: December 17, 2016, 02:13:28 pm »

Without demosaicing?

of course... DNG is DNG - so if you do not select it to be a linear DNG it will not be a linear DNG ...
Logged

scyth

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 584
Re: Same DcamProf-ile, Different Converter Colors?
« Reply #18 on: December 17, 2016, 02:14:31 pm »

How does one do that?

don't you see the "Save Image..." button
Logged

Jack Hogan

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 798
    • Hikes -more than strolls- with my dog
Re: Same DcamProf-ile, Different Converter Colors?
« Reply #19 on: December 17, 2016, 03:20:28 pm »

Who knew, always learning something new. I'll try that, thanks scyth.

Jack
Logged
Pages: [1] 2   Go Up