Camera Profiling - DNG, ICC and alternative methods

[samueljohnchia]

but then you do not need to intentionally use an artificial light w/ spectrum like sodium vapor or fluorescent light, do you ? why 'd you equip a studio w/ such light instead of a something w/ proper spectrum ? for some special effects ? but then you might be exactly looking for a bad color reproduction for that purpose 

I think you have misread my intentions. I was specifically told when I wrote in to QPcard that I would see clear advantages when profiling for fluorescent light sources, and that I should try that, so I decided to include that with my initial testing for the QPcard vs CC (some additional comments in the other thread), and I just dropped a simple comment on fluorescent light performance that's all. Eric Chan also mentioned elsewhere that folks will probably be much happier with custom camera profiles for unique spiky light sources as the Adobe Standard profiles are really meant for tungsten-daylight images, that's part of the reason why the DNG PE was made available. I'm not surprised with the results that I got, except for the fact that I could come reasonably close to the real deal with camera profiling from such a limited set of color patches in a target, made from limited materials that Ernst pointed out in my other thread, is most probably because of sheer luck than any camera profiling wizardry has got to do with it.

If it pleases you I'm sure that we will not be bothering to further test profiles for fluorescent light sources in this thread. No need to get worked up over this small issue.

you can manually set baseline exposure to zero in your .dcp profiles - so that is not a reason at all

I must be mistaken. I know I can remove the Base Tone Curve for dcp profiles, not the BaselineExposure offset. Bjanes has documented that issue elsewhere on this forum. If you know how to offset that in the dcp profile manually, I'm all ears.

[digitaldog]

?? In your quote in the Shooting Color Targets thread, I think he said Iridient Developer did NOT support DNG profiles other than embedded in DNG image files. So how would you embed them unless in ANOTHER raw converter that supports this? Or do you have new information?

An update! From Brian:

The Iridient Developer 2.0.1 update is now available here and adds full support for DNG Camera Profiles from the very latest DNG v1.4 spec:
http://www.iridientdigital.com/products/rawdeveloper_download.html

A couple notes:
1) There is still the option to color render using the older (baseline, most camera makers still use the old spec too) DNG v1.1 color matrix rendering too, even when loaded from full DCP files.
2) If you load a DNG you will now have 3 Input Profile options available (at least).
   a) ICC profile --> generally my default --> uses my default camera tone curve
   b) DNG Matrix Color --> this is the old v1.1 DNG color spec. Lacks support for tone curves, color lookup tables, etc. --> uses my default camera tone curve
   c) DNG Camera Profile --> this is the very latest v1.4 DNG color spec. --> by default uses either embedded tone curve or DNG SDK default (based on old ACR3 rendering style, I think Adobe 2010 process?) --> however optionally can disable use of DCP tone curve and use my camera tone curve.

2) Sometimes you'll see a fairly big difference between the DNG Matrix Color and the DNG Camera Profile, even with Adobe standard profiles and/or DNG image files that may only have support for the older DNG v1.1 specification. Why is this?

This is almost always largely due to differences in default tone curves. In the case of say a camera maker's DNG file that only specifies the original DNG v1.1 color information there will be no camera tone curve. So the DNG SDK v1.4 defaults to the old ACR3 style tone curve, obviously this may not match up exactly with the latest ACR7/LR4 style rendering. On the other hand (in theory) if the DNG Camera Profile includes a specific tone curve then both Adobe software and Iridient Developer will use the same DCP tone curve for rendering and the match should be quite good?

Haven't had a chance to mess around any with Color Checker Passport profiles and this latest release yet and I can't remember if they included their own tone curves or not? If they don't this could be an issue in color matching between Iridient Developer rendering and the latest Adobe software which I believe uses a newer default tone curve setup?? Anyway something to be aware of.

This is one of the continuing issues with the DNG color spec or for that matter RAW image color matching in general. If the most common option is to fall back to the Adobe default tone curve and Adobe doesn't publish their default tone curve spec then you'll get different results from different RAW processors. Same case with ICC profiles too, in order to use say Capture One ICC profiles with Iridient Developer you need to use the Capture One camera tone curves too to get a close match.

3) DNG Camera Profiles are much slower than other color rendering options. About 5x slower than my implementation of the original DNG v1.1 color spec! Adobe DNG SDK code is nice to look at, but extremely (often unusably!!) slow. I did some multi-threading and other quick and easy optimizations to the DNG SDK v1.4 giving about a 4-12x speedup, but it's still relatively slow compared to well optimized code that takes advantage of vector processing, etc.

That said "slow" on a mid-range to slow Mac when exporting a full size 16MP image is maybe about 0.5 sec versus something like 0.08 sec for my DNG matrix processing (written before DNG SDK even existed so all custom Iridient code) or ICC profiles which use Apple's ColorSync which is very well optimized for multi-core processing and vector support. With Adobe lens profiles my final implementation ended up being about 8-40x faster depending on processor than the DNG SDK code so if the DNG camera profiles are a popular option it will likely get much, much faster in the future as I move away from use of the DNG SDK for this color processing...




I also added an option in my Camera Curve pane to "Use Embedded Curve". If this option is disabled then the Iridient Developer or user specified tone curve will be used in place of the DCP tone curve (or lacking one the DNG SDK, ACR3 fall back default).

If you disable the "Use Embedded Curve"  checkbox with the standard DNG Camera Profiles or v1.1 spec DNG images then the DNG Matrix Color and DNG Camera Profile color renderings should be virtually identical (and in many cases quite similar to my default ICC profile too...).

Anyway let me know what you think. Look forward to your feedback! Supports using embedded DNG image data or loading camera profile data from standalone profile files (DCP) or other DNG images.

Adobe's Camera Profiles as shipped with ACR or DNG Converter or LR are located here:
/Library/Application\ Support/Adobe/CameraRaw/CameraProfiles

Best regards,
Brian Griffith
Iridient Digital

[Vladimirovich]

>  I must be mistaken. I know I can remove the Base Tone Curve for dcp profiles, not the BaselineExposure offset.


that is what I do (w/ all profiles, ACR7.x, to use w/ "Process 2012") for .xml dump from dcptool (and compile back to .dcp) :

<ToneCurve Size="2">
<Element v="0.000000" h="0.000000" N="0"/>
<Element v="1.000000" h="1.000000" N="1"/>
</ToneCurve>

<BaselineExposureOffset>0.000000</BaselineExposureOffset>

[digitaldog]

When opening a DNG with embedded and custom profile in ID, the preview looks very close but as expected, not identical to the same image in Lightroom. This leads me to believe that one can create custom DNG profiles and they work as expected in ID 2.0.1 as well as Adobe converters of course.

To select the DNG profile, use DNG file metadata in ID where you'd select an ICC profile.

[Hening Bettermann]

digitaldog:
> An update! From Brian

Look what His Masters Voice can achieve! :-)
and thanks for the instruction.

[solarj]

>  I must be mistaken. I know I can remove the Base Tone Curve for dcp profiles, not the BaselineExposure offset.


that is what I do (w/ all profiles, ACR7.x, to use w/ "Process 2012") for .xml dump from dcptool (and compile back to .dcp) :

<ToneCurve Size="2">
<Element v="0.000000" h="0.000000" N="0"/>
<Element v="1.000000" h="1.000000" N="1"/>
</ToneCurve>

<BaselineExposureOffset>0.000000</BaselineExposureOffset>

Thanks for letting me know about the dcptool, now I know all the answers of my questions regarding DNG PE 

BTW, what is the meaning of this BaselineExposureOffset?

[samueljohnchia]

>  I must be mistaken. I know I can remove the Base Tone Curve for dcp profiles, not the BaselineExposure offset.


that is what I do (w/ all profiles, ACR7.x, to use w/ "Process 2012") for .xml dump from dcptool (and compile back to .dcp) :

<ToneCurve Size="2">
<Element v="0.000000" h="0.000000" N="0"/>
<Element v="1.000000" h="1.000000" N="1"/>
</ToneCurve>

<BaselineExposureOffset>0.000000</BaselineExposureOffset>

Sorry, inserting the BaselineExposureOffset tag does not work for me. Only removing the tone curve does.

[samueljohnchia]

BTW, what is the meaning of this BaselineExposureOffset?

Take a look at this: http://www.luminous-landscape.com/forum/index.php?topic=30794.0

[samueljohnchia]

When opening a DNG with embedded and custom profile in ID, the preview looks very close but as expected, not identical to the same image in Lightroom. This leads me to believe that one can create custom DNG profiles and they work as expected in ID 2.0.1 as well as Adobe converters of course.

To select the DNG profile, use DNG file metadata in ID where you'd select an ICC profile.

Awesome stuff! Thank you Andrew, for nudging Brian to support the latest 1.4v spec, and also thanks to Brian for implementing it so quickly. This is wonderful. My experience with Raw Therapee is also similar and we can add that to the list of raw converters that handle can DNG profiles outside of Adobe's.

Have you tried the generic ICC profiles against your custom DNG profiles yet?

[Vladimirovich]

Sorry, inserting the BaselineExposureOffset tag does not work for me.

may I ask you how did you test that ? for example decompile a profile, make baseline exposure = 3.0 (or something like this, to see for sure), compile it back (w/ different name of course), open your raw file and switch between those 2 profiles where only baseline exposure differs by 3 EV (or whatever you put there)... do you see no changes at all ? are you sure ?

[samueljohnchia]

may I ask you how did you test that ? for example decompile a profile, make baseline exposure = 3.0 (or something like this, to see for sure), compile it back (w/ different name of course), open your raw file and switch between those 2 profiles where only baseline exposure differs by 3 EV (or whatever you put there)... do you see no changes at all ? are you sure ?

Yes, I did that. I started with a custom profile built using the DNG PE. I created two versions, one with a <BaselineExposureOffset>0.000000</BaselineExposureOffset> tag and one with a <BaselineExposureOffset>3.000000</BaselineExposureOffset> tag. All three give me exactly the same result.

I tried it on a Adobe Standard profile for my camera and the same thing happened.

Of course eliminating the tone curve did give me a linear profile, but the BaselineExposureOffset didn't offset any exposure.

[Vladimirovich]

Yes, I did that. I started with a custom profile built using the DNG PE. I created two versions, one with a <BaselineExposureOffset>0.000000</BaselineExposureOffset> tag and one with a <BaselineExposureOffset>3.000000</BaselineExposureOffset> tag. All three give me exactly the same result.

it is very strange because in my case it does ( ACR 7.4 RC + Process 2012 ) ...

[digitaldog]

Have you tried the generic ICC profiles against your custom DNG profiles yet?

I have to look at a lot more images but so far, the differences seem rather subtle. Here's one example with the only differences being the profile and everything else ID Defaults. (do we like the blue patch?):

[samueljohnchia]

it is very strange because in my case it does ( ACR 7.4 RC + Process 2012 ) ...

Do you mind looking at my xml file? It is at this link.

I added the baseline tag after the tone curve (which I left there to just test the effect of that tag). If I have formatted it wrongly please let me know.

[samueljohnchia]

I have to look at a lot more images but so far, the differences seem rather subtle. Here's one example with the only differences being the profile and everything else ID Defaults. (do we like the blue patch?):

The icc profile also has a slight green cast at this WB setting. May have to use a different WB for comparison. Hmm blue patch. I prefer the top image. Purple patch goes to the bottom image. But both are rather washed out!

[digitaldog]

The icc profile also has a slight green cast at this WB setting. May have to use a different WB for comparison. Hmm blue patch. I prefer the top image. Purple patch goes to the bottom image. But both are rather washed out!

I might but what I wanted to do was simply see the differences between the two profiles, nothing else. And that said, I too prefer the top image. The top has a tad more contrast too and I guess I could 'equalize' the two. But right off the bat, I like the custom DNG profile a bit better.

[samueljohnchia]

I might but what I wanted to do was simply see the differences between the two profiles, nothing else. And that said, I too prefer the top image. The top has a tad more contrast too and I guess I could 'equalize' the two. But right off the bat, I like the custom DNG profile a bit better.

I'm ok with re-white balancing, but I'm not sure about equalizing the contrast - depending on ID's implementation, it may introduce hue and saturation shifts and it will be difficult to tell if its the profile twisting the colors or the adjustment. Hence the earlier discovery to find linear renderings from ACR, ID and RT.

We need to look at custom ICC profiles too. I apologise for the delay in testing on my side. I've been rather swamped of late. I think I would have some time later this week to do the testing again.

[Vladimirovich]

Do you mind looking at my xml file? It is at this link.

I added the baseline tag after the tone curve (which I left there to just test the effect of that tag). If I have formatted it wrongly please let me know.

you certainly do something wrong...

I took 5DmkII Adobe Standard profile, decompiled it, removed LUT (not that it matters, just I can't stand 'em)... baseline exposure tag was there w/ value = 0... I changed the name of profile to BE0, left tag @ 0, compiled to BE0.dcp, moved to the proper custom profiles location, then changed the name of profile to BE3, changed tag to 3, moved again to the proper custom profiles location... now when I open a raw from 5DmkII I can see two new profiles BE0 and BE3 and switching between them changes the exposure... so I 'd assume you did something wrong in your setup - are you sure you are even using the proper profiles with different baseline exposure tags in your setup ?

[Hening Bettermann]

Here's another quickie. Canon 5D2, 2 Solux lamps run at 14 Volt, custom WB of the day.
All displayed in ID, working and output profile set to ProPhoto, rendering intent to rel.col. Otherwise ID defaults except the profiles:

1-ID default ICC
2-QP ICC
3-QP dcp

My view:
2 is visually closest to the physical chart in the pale yellow at bottom right (E7) and its neighbours, which are more reddish in particular in 3.
2 is also best in showing the slight differences in the reds (B2-4). None of them hits the purplish blue in B 4, but 1 is slightly closer than the others. 3 looks overall muddy, which is confirmed by the DigitalColorMeter: the white patch in A7 has gone from 214/211/208 in 1 and 2 to 195/192/189.

Good light - and true colors ;-)

[samueljohnchia]

you certainly do something wrong...

I took 5DmkII Adobe Standard profile, decompiled it, removed LUT (not that it matters, just I can't stand 'em)... baseline exposure tag was there w/ value = 0... I changed the name of profile to BE0, left tag @ 0, compiled to BE0.dcp, moved to the proper custom profiles location, then changed the name of profile to BE3, changed tag to 3, moved again to the proper custom profiles location... now when I open a raw from 5DmkII I can see two new profiles BE0 and BE3 and switching between them changes the exposure... so I 'd assume you did something wrong in your setup - are you sure you are even using the proper profiles with different baseline exposure tags in your setup ?

Ahh I missed the dcpTool december update to DNG v1.4 spec. My bad. Otherwise everything went as you said, as expected. I can also see the other tags ProfileLookTableEncoding and DefaultBlackRender now.