State of the art camera profiling software?

[torger]

I make my custom software. Actually all calculations does in Matlab plus sort of handmade works (eg matrix fitting, whitepoints iteration etc).

Schema is: CMF + spectral "paths" (~50K items) + spectra of selected illuminants (A, D50, D65 etc) -> sensor gamuts under selected illuminants -> warping (Shepard's method) sensor gamuts to human gamuts under selected illuminants -> LUT's for every illuminant -> resulting profiles.

Thanks! I think I understand.

I've had a look at the profile for Canon 5Dmk2, what differs the "GM" from the "NOGM"?

I've noted that the GM has a LookTable and the NOGM has not, otherwise they are equivalent. The GM version looks slightly more saturated.

I made a quick comparison of the same ~D50 scene using matrix only, Adobe default, Adobe DNG Profile Editor on the color checker in the scene, Trantor's GM and NOGM, plus Capture One's 5Dmk2 generic profile, all rendered in RawTherapee which supports both DNG and Capture One icc.

The scene is not particularly good only a colorchecker on a chair (an old shot from another occasion, I don't have the camera here now so I can't reshoot), used that because I have the chair and light here so I can see it in reality.

Making a casual evaluation of this scene I'd say the matrix profile is quite obviously off, Adobe Standard is not too bad but have some color shifts, easier detected on the subtle colors of the wooden chair than on the colorchecker itself, the DNG PE generated profile is clearly over-saturated and got yellow too orange, Trantor's profiles and Capture One's are all quite similar and looks most correct to my eyes.

Trantor's profile seems to have some issue with the dark blue patch on the color checker, it's a little bit too low saturation, and the bright yellow is too orange. Overall Capture One's profile is the most correct to my eyes.

As the light is a florescent print viewing light it's not as good as true D50 though. I was surprised that DNG PE did not make a better result which actually got the specific color checker in the scene under the specific light to work with, but it could make all sorts of tradeoffs against accuracy.

I then tested the same five profiles side by on a couple of landscape scenes with sunset type of light. I don't have the color memory to know which one that looks most realistic, but the same type of differences as on the colorchecker scene could be noted, quite large differences between matrix-only, adobe standard, DNG PE which all had their own unique looks, and then quite small (but detectable) differences between Trantor's two and Capture One's.

It seems to me from this brief test that Trantor's method works well, but maybe can be improved upon, and that DNG PE is not really that good, but it can be the software's fault rather than the color checker.

[TRANTOR]

what differs the "GM" from the "NOGM"?

Profiles marked with "GM" contains gamut mapping LUT that maps ProPhoto gamut into sRGB gamut. Marked with "NOGM" do not have gamut mapping table respectively.

[torger]

Profiles marked with "GM" contains gamut mapping LUT that maps ProPhoto gamut into sRGB gamut. Marked with "NOGM" do not have gamut mapping table respectively.

Ah... why would one want to map from Prophoto to sRGB? As DNG's color space is defined as Prophoto the raw converter should take care of gamut mapping, right?

[TRANTOR]

raw converter should take care of gamut mapping, right?

I don't like how it works in LR, Photoshop etc in some cases. Example in attachment. LR relative gamut mapping vs my gamut mapping.

[Tim Lookingbill]

So you've figured out a way to track luminance and mate it to hue/sat and preserve that relationship with linear increase in luminance or overall brightness. Those are really good results.

I've had to resort to creating huge shepherd hook shaped point curves and HSL adjusts in ACR/LR combined with a DNG profile to get those results, so you're saying you can now do it all in a profile?

[TRANTOR]

so you're saying you can now do it all in a profile?

Yeah, thats right. Algorithm is pretty smpile in Lch (lightness/chroma/hue) space:

L = const, h = const
c_in -> source gamut by chroma
c_out -> destination gamut by chroma
c_in_max - > source gamut by chroma maximum
c_out_max -> destination gamut by chroma maximum

[0 < c_in < c_in_max]
[0 < c_out < c_out_max]

if c_in_max > c_out_max
{c_result = sigmoidal compression function(c_in, c_out) }
else { c_result = c_in }

Compression function looks like this:

[Tim Lookingbill]

Some nice complex work I could never rap my head around, but I'm glad there's someone who can.

Can you track this compression and trace it/or keep it tied to measurements of the actual scene or are you going about measuring how the sensor responds to the scene.

How does that profile make normal gamut scenes with less saturated objects appear? Less saturated overall?

[TRANTOR]

In my case this is static compression/mapping from ProPhoto to sRGB. For DCP-profiles I use two first LUT's for correction (by warping) sensor gamut to the human gamut and second LUT for gamut mapping (ProPhoto -> sRGB). LR and other RAW-processors do not use spatial gamut mapping that depends on scene gamut.

[Tim Lookingbill]

Can you post another image with not so saturated color using the same profile you applied to the rose? I'm trying to see if the profile renders a wide range of scenes with a consistent balance.

Lack of color balance is the main issue current profiles have a hard time with across a wide range of scenes depending on the level of saturation of objects and illuminant in the scene. A lot of this lack of balance can be exacerbated by the quality of UV/IR filtering on the sensor which can be different between camera brands & models.

In my experience with my own camera I never know when these saturated objects are going to go off the charts while others don't. I don't know if it's possible a profile can fix that. Yours might.

[torger]

Worth noting is that if you print the gamut mapping can be controlled by the ICC profile, via the perceptual intent. However I typically reduce gamut in the raw converter to fit the printer's gamut and then print relative colorimetric with black point compensation, and then all gamut mapping will take place in the raw converter.

Tools to reduce gamut in a raw converter manually is just things like decreasing overall saturation, reduce saturation (or value) for specific problematic colors etc.

There is no native support in DNG Profile to make gamut mapping, but of course you can do it in Trantor's way, it means that you should only use the gamut mapping version if you have issues with out of gamut colors.

Personally I don't think gamut mapping is a big issue, for my own work I'm satisfied with doing manual adjustments of problem colors. I generally don't have very saturated colors in my pictures though. sRGB does not cover Pointer's gamut so there can be issues with flowers etc, AdobeRGB covers most of it but still not all.

[papa v2.0]

I believe I read Iliah Borg on the colorsync list commenting about how camera profiles should be a simple matrix profile, built from monochromator data, with no LUTs involved. That is how Adobe does it to derive their color transform matrices.

I though they used a target based system? I might be wrong.
iain

[Tim Lookingbill]

From visual observation gamut mapping requires luminance to be factored in with the luminance of the display which also has gamut limits. Luminance seems to be the black hole for defining exactly how it should be rendered on a ICC calibrated/profiled display.

It's trying to figure out if the failure to preserve HSL relationships that crush flower detail as in the rose example happens at the capture stage or processing stage with software and how it maps it to the luminance of the display.

[TRANTOR]

Can you post another image with not so saturated color using the same profile you applied to the rose?

Yeah. Sure.

GM:



NOGM:



GM:



NOGM:



GM:



NOGM:



GM:



NOGM:



GM:



NOGM:



All pictures by Canon 6D.

[Tim Lookingbill]

Thanks, Trantor.

I can barely tell the NOGM from the GM which I'm guessing is as it should seeing that the differences look similar to assigning an sRGB-ish gamut monitor profile to sRGB encoded image. Their gamuts are so close that there are only subtle differences.

I had to download the CCchart examples and layer one on top of the other in Photoshop turning the top layer off/on to see the subtle changes.

The rose's scene gamut must be beyond the Canon 6D's gamut capture capability by a considerable degree compared to the 5 you've posted.

Very informative demonstration, Trantor. That's some really good work figuring all that out.

[Czornyj]

It's not beyond 6D'2 gamut - it's compressed from 6D's gamut (rendered to ProPhoto) to sRGB. RelCol rendering intent in LR et al. flattens all OOG colors to the most saturated, colormetric consistent colors available in sRGB.

[Tim Lookingbill]

It's not beyond 6D'2 gamut - it's compressed from 6D's gamut (rendered to ProPhoto) to sRGB. RelCol rendering intent in LR et al. flattens all OOG colors to the most saturated, colormetric consistent colors available in sRGB.

This word "compression" as it is used in these discussions appears to not render the same across a wide range of apps that apply it to the data with regard to rendering definition and detail as in Trantor's rose example vs how it's rendered like say in ACR/LR.

Now whether this detail and definition that is revealed through compression can be attributed to gamut mapping still doesn't indicate that detail was within or outside any gamut description model or construct unless the rose detail was actually measured with a spectro. This could just be on account of the limits of the slider tools within each software that are suppose to bring out this detail. I saw this kind of blown detail recovered with a severe shepherd's hook curve edit in cloud highlights on jpegs derived from minilab scan years back but I could never say that was on account of any gamut mapping.

Anyone can assign LMS cone response numbers, XYZ, HSL, HSV etc., pick any color description model of your choice but to attribute it to the gamut capture of the device and prove it are two different things. All I'm trying to do is understand and locate where this loss of detail comes from on a wide range of cameras that some lab outfit I've never heard of now says all have a particular 3D gamut shape and size.

No one is explaining the differences seen in previews between apps by applying their own version of "compression" to bring out detail.

[torger]

It's not unlikely that the red rose had saturated enough colors to be outside the sRGB gamut, but probably not by much. The image below left shows a comparison between Pointer's gamut (gamut of all possible reflected colors, the irregular shape), and sRGB (the triangle). Red can come outside but not that much. The right shows a comparison with the huuuge Prophoto.



(sorry image links seem to break, but you can look at the article at TFT central: http://www.tftcentral.co.uk/articles/pointers_gamut.htm)

Trantor's gamut mapping is a static mapping though as it's in the profile, ie it doesn't know how little it needs to compress, it compresses from the much larger prophoto space and will thus lower saturation more than needed. It would be interesting with a gamut mapping from Pointer's gamut to sRGB, it would compress considerably less and it would handle all reflected colors (in theory at least).

Anyway, as far as I understand Trantor's profiles are designed for accuracy as good as possible, ie no intentional saturation increase. How accurate Trantor's 6D profile actually captures the saturated red tones in that light condition is hard to know of course. Of all things I've read so far I think it's quite safe to say that a modern digital camera with a profile made for the purpose can capture fairly accurate colors a far bit outside sRGB.

I'm not sure this is true, but I suspect that a virtually generated profile from measured camera response (like Trantor's profile) can be more exact for saturated colors than a profile made the traditional shoot-a-colorchecker-way, but probably a colorchecker can achieve better accuracy for colors close to its actual patches.

[Bart_van_der_Wolf]

Trantor's gamut mapping is a static mapping though as it's in the profile, ie it doesn't know how little it needs to compress, it compresses from the much larger prophoto space and will thus lower saturation more than needed. It would be interesting with a gamut mapping from Pointer's gamut to sRGB, it would compress considerably less and it would handle all reflected colors (in theory at least).

Hi Anders,

I agree, although Trantor's compression is non-linear as I understand it. Bruce Lindbloom already took a similar approach by designing his Beta RGB colorspace, which takes into account the three dimensional gamut hull of many important colors that needed to be accurately encoded, yet with maximum efficiency for creating as small a gamut hull as possible (to minimize the quantization step size of the integer number encoding precision). I'm not sure about Pointer's gamut in 3D, how well it covers very dark and very bright colors.

Cheers,
Bart

[Tim Lookingbill]

Color balance is the best that we can achieve over exact accuracy which it appears from Trantor's examples requires various levels of compression depending on the scene gamut to achieve consistency across a wide range of scene gamuts. I would have to say though that Trantor's examples overall are not accurate to what those scenes should look like given the light characteristcs which appears beefs up contrast which can have a major influence not only in the rendering of saturated colors but also viewer perception.

As a side note I conducted an experiment today by photographing my calibrated 6500K, 100cd/m2, 2.2gamma, sRGB-ish display showing an image of my own CCchart whose Lab numbers for each patch are exact including the gray patches read in Photoshop. I built a single illuminant profile using the most current DNG PE Wizard and the resulting profile didn't make much of a difference between the normally produced profile. Blue skies in landscapes were slightly magenta and brighter and greens were a bit cool. Cyan's didn't change at all understandably since it's an sRGB display.

The neat thing about doing it this way is that the actual CCchart patches could be edited to achieve a result with other colored (wider gamut) test objects in the scene (instead of editing the Table & Calibration panel through DNG PE which tends to create artifacts if you go too far). Those with AdobeRGB gamut displays would have wider range of colors such as cyan's to tweak and bring out applying the profile to scenes that have these hues. It will also give some indication of the gamut capture capability of the camera.

I have very richly intense cyan objects that my camera can't reproduce. I'm not saying this is the ultimate in achieving accuracy, just a way to creating and editing the results of a DNG profile that doesn't require using the DNG PE editor or any software presets that have to tag along with the custom profile.

It's worth a try and not much of a hassle conducting this experiment but it did give me an idea of what the Wizard actually corrects for if you screw up the colors.

[Peter_DL]

The efficiency of pure matrix profiles are indeed an interesting subject.

DxO provides a Metamerism Index as part of their sensor measurements and KPIs,
it indicates the degree of compliance with the Luther-Ives condition, so that in the ideal case the "sensor gamut" could be described by one simple matrix profile.

A brief description is given here, however, it is also stated that the value is not particularly meaningful to discriminate DSLR sensors which are typically all in the 75 to 85 range (as opposed to low-end cameras such as camera phones).
Also, it does not allow a direct conclusion if matrix-to-matrix-interpolation would be good enough for "ordinary photographic purposes" (as opposed to reproduction tasks), means to have a reasonably accurate starting point in order to work to a pleasing rendition.

From my experience, this matrix approach has become significantly better over the past 10 years or so and with today’s cameras. In the past there were cameras which could not be reasonably calibrated in ACR by tweaking the matrix primaries with the corresponding sliders. There often was an unconquerable dualism of e.g. getting the red hues right but making the skin hues worse, and vice versa. Later on, the Nikon D5100 which was mentioned somewhere along this thread was my first camera where I concluded that "matrix" (matrix-to-matrix-interpolation) is good enough for me.

However, Adobe went in different direction with the Adobe Standard profiles which replaced the former ACR x.y matrix profiles.  Initially there was a LookTable added, however, since 2 or 3 years there is also a HueSatDelta table (actually 2 of it, one per illuminant) which I understand are part of the accuracy compartment of the profile next to the matrices.

Peter