Luminous Landscape Forum
Raw & Post Processing, Printing => Colour Management => Topic started by: Jack Hogan on December 15, 2016, 06:34:51 am
-
Hello folks,
I've been playing with DcamProf single illuminant, noon daylight. Any suggestions as to why the colors produced by ACR look so different from RawTherapee's when using the same single matrix .dcp neutral+ 'look' profile (obtained from a cc24 captured at the same time as the desired image), same white balance and everything (possible) turned off?
Here is a rough example, screen capture of 200% AdobeRGB image, left to right:
- matlab matrix+HSmap+lookTable from DNG converted raw file;
- ACR CC directly from raw file
- RawTherapee 4.2.1.375 (Base and Look table checked, directly from raw file)
- matlab matrix only, from DNG converted raw file.
(http://i.imgur.com/695uEGf.png)
I was trying to figure out why my routine (1) produces results that look quite different from RT's (3) after the application of HSmap and lookTable - when I realized that RT looks quite different from ACR. Which can be used as a reference?
Jack
-
I'm not sure I understand what I'm looking at, are you using the same DCamProf-generated DCP in ACR CC and RawTherapee?
A signature difference between Adobe's default profiles and a DCamProf profile with the neutral tone reproduction operator is that Adobe's skies are a bit under-saturated while DcamProf keeps more of the blue. So from the image it seems like you're not using the same DCP.
If you are using the same DCP I can't really explain the difference. RawTherapee doesn't have black level subtraction like Adobe has, which causes lighter shadows in RT if the DCP doesn't have the blackrender none flag set (default for a DCamProf DCP), but otherwise the rendering should be the same for normally exposed images. For overexposed images ACR does some extra tricks to simulate overexposed film, which RT doesn't do.
Looking at the wooden details it seems like the ACR is warmer, that the white balance is still not the same. That could be the reason, but saturation shouldn't differ so much from that, it's a bit hard to tell, I'm not at a calibrated screen for the moment. A different white balance can be due to the white balance shift issue with DCP, discussed here:
https://www.ludd.ltu.se/~torger/dcamprof.html#dcp_wb_shift
If you have the white balance setting at "as shot" or if you have used a wb picker in ACR *after* you changed profile, you shouldn't have WB shift problems, or if you have copied the color matrices from the Adobe Default profile, which I recommend to ACR users.
-
I'm not sure I understand what I'm looking at, are you using the same DCamProf-generated DCP in ACR CC and RawTherapee?
Hi Torger,
Yes, the first three images are all supposedly applying the same .dcp profile generated by DcamProf (off cc24, with neutral+ operator).
A signature difference between Adobe's default profiles and a DCamProf profile with the neutral tone reproduction operator is that Adobe's skies are a bit under-saturated while DcamProf keeps more of the blue. So from the image it seems like you're not using the same DCP.
Hmm, everything is possible (especially when I am involved:-) but I did select the same profile in both programs.
If you are using the same DCP I can't really explain the difference. RawTherapee doesn't have black level subtraction like Adobe has, which causes lighter shadows in RT if the DCP doesn't have the blackrender none flag set (default for a DCamProf DCP), but otherwise the rendering should be the same for normally exposed images. For overexposed images ACR does some extra tricks to simulate overexposed film, which RT doesn't do.
Thanks for that, if the two images are supposed to look the same I must have something wrong with the setup since I don't think this image should be classified as overexposed.
Looking at the wooden details it seems like the ACR is warmer, that the white balance is still not the same. That could be the reason, but saturation shouldn't differ so much from that, it's a bit hard to tell, I'm not at a calibrated screen for the moment. A different white balance can be due to the white balance shift issue with DCP, discussed here:
https://www.ludd.ltu.se/~torger/dcamprof.html#dcp_wb_shift
If you have the white balance setting at "as shot" or if you have used a wb picker in ACR *after* you changed profile, you shouldn't have WB shift problems, or if you have copied the color matrices from the Adobe Default profile, which I recommend to ACR users.
Ah, I think I did pick WB on the cc24 image and synchronized before changing profile (though I don't understand why that would make a difference). Here they are now, having changed profile before picking WB, both sRGB for simplicity. Strangely ACR now indicates 5100K vs 5350K before (which agreed with RT). Colors are much closer now, with obvious exceptions the blues and the handling of out of gamut red (see flowers at the bottom), RT on left and ACR on right:
(http://i.imgur.com/lnG0wIc.png)
Is this correct or should they look the same now?
Thanks for your input.
Jack
-
Just in case anyone is interested the following link (https://drive.google.com/file/d/0BzGif7iQ1sgldVpXVzhOYkxFSXc/view?usp=sharing) downloads two NEFs (cc24 and image) and the .dcp profile used above. 58MB.
Jack
-
"True" white balance for a digital camera is multipliers for R, G and B. If you have the exact same multipliers in two software you get the exact same white balance. Few softwares allow you to set the multipliers directly though, as it would be a too technical interface.
Instead you usually set white balance with "temperature and tint". Some raw converters convert those to multipliers and store those, and some, like ACR, store the temperature and tint rather than multipliers. The problem is that there is no standard formula to convert multipliers to temp/tint, so 5000/+10 in one raw converter may not be the same as 5000/+10 in another.
With ACR it's even "worse", as temp/tint is derived using the color matrices in the DNG profile. So if you change profile the temp/tint derivation will change, as color matrices are not exactly the same. As ACR store temp/tint rather than the multipliers when you change profile the multipliers change, but temp/tint stays the same. However, if you have white balance "As Shot" ACR actually use the *multipliers* stored in the raw file, and thus when you change profile white balance stays the same, but temp/tint change. And of course if you use a wb picker the white balance will be set after that patch, but if you change profile after that, the wb is set from temp/tint and you will have a shift so you need to use the wb picker again.
Perhaps one would think as the profile is calculated for the same camera the color matrices would be "close enough" that this would not be an issue. But cameras are far from exact color temperature measurement devices, and algorithms of how to calculate the color matrices differ between software so you can easily get a few hundred degrees difference, and certainly a visible difference.
A work-around for ACR users is to copy the ACR color matrices into DCamProf profiles, which you can do using the -m parameter to make-dcp command.
As the color matrices are only used for temp/tint calculation and not the actual color rendering (forward matrices are used for that) you won't hurt color.
If you ask me I think it would have been better design by Adobe if they'd always store the multipliers instead so changing profile never could cause a white balance shift, but it is the way it is.
To make sure RT and ACR renders the same you need to make a DNG and let RT use that, as black levels and white levels may differ a bit between ACR's and RT's raw format translation. I don't think it should be an issue for the NEF though, but it can be good to do as a precaution. Black render we have already discussed, then there's the output profile and clip handling which you have noted.
I still think the difference in color of the blue sky is a bit strange, and the flat colors of the rock and the closest door doesn't seem right, it's as if the ACR screenshot doesn't reach to full clipping. I would suspect some issue with output profile. Note the yellowy detail in the center of the bottom edge of the image, it's clearly clipped in RT, but just flat gray in ACR.
Also note that when making screenshots the use or not use of display profile can cause unexpected results.
I haven't done any recent comparison between ACR and RT, but my memory tells me that they should be much closer than your images show when the same normally exposed DNG is used with the same DCP (with blackrender=none flag), using the same white balance, and the same output profile.
-
I noticed some differences between LR and RT when using the same dcp profile some time ago. First it was with DCamprof profiles that I made, but then I used the same Adobe Standard dcp to eliminate any mistakes I could have made when creating the profiles. In my case the RT photos are undersaturated when compared with LR, so I need to compensate for that in my post-processing. I couldn't find an explanation for this other than it must be something in RT's processing pipeline -- maybe the AMaZe demosaicing? because that's different between the two apps but I really don't know.
-
I noticed some differences between LR and RT when using the same dcp profile some time ago. First it was with DCamprof profiles that I made, but then I used the same Adobe Standard dcp to eliminate any mistakes I could have made when creating the profiles. In my case the RT photos are undersaturated when compared with LR, so I need to compensate for that in my post-processing. I couldn't find an explanation for this other than it must be something in RT's processing pipeline -- maybe the AMaZe demosaicing? because that's different between the two apps but I really don't know.
I guess I need to make some comparisons myself again, because I don't think there should be such differences. I have made A/B comparisons in the past but it's more than a year ago. I was deeply involved developing RT's DCP support, but I'm not as active in RT's development now and much has happened since...
If the profile lacks the black render=none flag, and lacks a curve, the difference will be huge as RT doesn't apply a default curve and no black subtraction like LR does. But otherwise it should look same when no adjustments are applied, other than really minor things such as the white point may differ 0.1-0.2 in exposure or so.
-
Ok, thanks for that explanation and suggestion Torger. I will try the dng on RT when I have some time this weekend. BTW, the image was captured by a D610 and Nikon subtracts blacks from it before writing data to the raw file.
Jack
-
I've done a fresh comparison, this time both with a nef and a dng converted from it. It looks like checking the DCP look table in RT desaturates my photo (it compresses the blue channel in the 3/4 of the histogram mainly). Other than that, as Anders said, RT produces lighter shadows (less contrast) as compared with ACR. Colours look the same between ACR and RT with the DCP tone curve checked, look table unchecked and Camera WB (I've chosen the RelCol intent in RT output options -- I'm not sure what rendering intent is used by ACR?).
-
Concerning the WB differences -- I've noticed that if you switch on the demosaicing CA auto-correction in RT (in the Raw tab, not in the Transform) the image gets slightly less warm, so maybe the OP had this option turned on when doing the comparison?
-
Thanks for checking that sankos. I am pretty sure that CA was unchecked. I don't think I have the tone curve turned on (it looks like Adobe's base curve which is way too bright so I removed it with DNG Profile Editor), I wonder if this could be the problem?
-
Jack, again, your png screenshot doesn't have an embedded profile. Screenshot png color is heavily influenced by the gamut of your display recorded into the display's ICC profile which needs to be assigned and converted to sRGB for online display. My Mac OS automatically tags/assigns my display profile to my png screenshots which I have a choice to convert to sRGB or just leave as is in hope it is used by online browsers for color managed previews. I don't know what OS you're using so this is just FYI.
Now, aside from that, just based on my observation and speculation have you considered ACR's color engine is different in the way it interprets demosaiced sensor color data as the cause for the differences you see over other Raw converters who use different demosaicing algorithms and color definers with the added complexity of how each integrate ICC color managed Raw previews?
I also noticed a change in color when I switched to PV2012 in LR4 and rebuilt my camera profiles using updated DNG converter (and updated DNG Profile Wizard) from profiles I made from old DNG converters/Wizard in CS3 PV2003. What I noticed in the newer profile generated previews in LR4 is they closely resembled the more neutral, less saturated look (more pinkish Caucasian skin tones) of Adobe Standard with minor tweaks to hue/sat color tables. It wasn't a desirable result especially in skin tones so I went back to the old profiles which of course made flowers over saturated.
-
Jack, again, your png screenshot doesn't have an embedded profile. Screenshot png color is heavily influenced by the gamut of your display recorded into the display's ICC profile which needs to be assigned and converted to sRGB for online display. My Mac OS automatically tags/assigns my display profile to my png screenshots which I have a choice to convert to sRGB or just leave as is in hope it is used by online browsers for color managed previews. I don't know what OS you're using so this is just FYI.
Hi Tim
I am using Win 7 and I am aware of the effect of the ICC profile, which however I think should affect color from all images in the same way when displayed on and 'cut' from the same monitor.
Now, aside from that, just based on my observation and speculation have you considered ACR's color engine is different in the way it interprets demosaiced sensor color data as the cause for the differences you see over other Raw converters who use different demosaicing algorithms and color definers with the added complexity of how each integrate ICC color managed Raw previews?
I also noticed a change in color when I switched to PV2012 in LR4 and rebuilt my camera profiles using updated DNG converter (and updated DNG Profile Wizard) from profiles I made from old DNG converters/Wizard in CS3 PV2003. What I noticed in the newer profile generated previews in LR4 is they closely resembled the more neutral, less saturated look (more pinkish Caucasian skin tones) of Adobe Standard with minor tweaks to hue/sat color tables. It wasn't a desirable result especially in skin tones so I went back to the old profiles which of course made flowers over saturated.
From what I understand of demosaicing, global tones should not be affected. Process Versions are instead clearly right in the thick of it because they are based on the changing DNG spec, which in fact substantially changed color handling at each step of the way (v1.1, v1.2, and the current v1.4 referring to the various PVs, the last one being PV2012).
Thanks for your input,
Jack
-
Ok, got it. It was indeed the fact that I disabled the tone curve. Here are RT (left) and ACR (right), both first converted to dng, with negative EC of 0.75 and 1.05 respectively to compensate for the aggressive tone curve. They are now very, very similar:
(http://i.imgur.com/ZvPVtdk.png)
Thanks all for your help.
Jack
-
While I have your attention, I have noticed an annoying habit of DNG Converter 9.8 (but the 'feature' was present in earlier versions as well): it loads in the converted DNG file matrix and profile information from a custom .dcp profile in the ...\CameraRaw\CameraProfiles\ directory, I can't tell how it chooses which one to use if more than one is present. The only way that I have found to have it load default profile information (I assume Adobe Standard) is to erase all .dcp profiles from that directory. However that is a real pain, because one then looses them for ACR/LR use.
Any idea how to tell DNG Converter to go with its default profile and not use one of the custom ones? Or even better, how to tell it if and which custom profile to load? The Adobe forum has not been helpful on this issue so far.
Jack
-
While I have your attention, I have noticed an annoying habit of DNG Converter 9.8 (but the 'feature' was present in earlier versions as well): it loads in the converted DNG file matrix and profile information from a custom .dcp profile in the ...\CameraRaw\CameraProfiles\ directory, I can't tell how it chooses which one to use if more than one is present. The only way that I have found to have it load default profile information (I assume Adobe Standard) is to erase all .dcp profiles from that directory. However that is a real pain, because one then looses them for ACR/LR use.
Any idea how to tell DNG Converter to go with its default profile and not use one of the custom ones? Or even better, how to tell it if and which custom profile to load? The Adobe forum has not been helpful on this issue so far.
Jack
you can simply use ACR to convert to DNG instead of DNG converter.
-
Without demosaicing? How does one do that? I still need the data to be raw, just in DNG format so that matlab will read it.
-
Without demosaicing?
of course... DNG is DNG - so if you do not select it to be a linear DNG it will not be a linear DNG ...
-
How does one do that?
don't you see the "Save Image..." button
-
Who knew, always learning something new. I'll try that, thanks scyth.
Jack
-
With ACR it's even "worse", as temp/tint is derived using the color matrices in the DNG profile. So if you change profile the temp/tint derivation will change, as color matrices are not exactly the same. As ACR store temp/tint rather than the multipliers when you change profile the multipliers change, but temp/tint stays the same.
There is a way to cater for this though. In a DCP profile with forward matrix as well as colour matrix, the latter is only used to convert from XYZ to camera space and thus only to do the whitebalancing by temperature/tint. The actual profile transform uses forward matrix. This allows to share the same colour matrix across several profiles for the same camera for example and get repeatable WB across profiles. This can be changed after profile has been built by using dcptool for example - convert profiles to XML, copy the colour matrix (but not the forward one) from one of them to all and convert the XML to DCP with dcptool again. It would be very nice if Adobe procucts would actually store WB in the multipliers as opposed to temperature since it will be profile independent WB but I don't think it will happen.
For myself and dealing with multiple Kodak cameras I had a little tool written to convert between temperature/tint and camera WB multipliers for a given profile. Example:
c:>dcrdcpwb.exe Kodak_SLRn.dcp tuscany.DCR
Profile name: Kodak DCS Pro SLR/n - RPP
Camera Model:Kodak DCS Pro SLR/n
WB presets:
Daylight: Temperature = 4962.92K, Tint = -15.8262 for camera neutral (0.814638,1,0.646778)
Tungsten: Temperature = 3073.79K, Tint = -38.7456 for camera neutral (1,0.961426,0.51123)
Fluorescent: Temperature = 5222.13K, Tint = 7.5804 for camera neutral (0.778411,1,0.625618)
Flash: Temperature = 5213.96K, Tint = -4.44951 for camera neutral (0.789819,1,0.644813)
Auto: Temperature = 5670.06K, Tint = -31.5093 for camera neutral (0.793308,1,0.714674)
What makes situation even worse is that ACR/Lightroom WB settings are interegers and for some WB settings the conversion of them to multipliers can be quite crude.