Camera Profiling - DNG, ICC and alternative methods

[Tim Lookingbill]

Here's a study you may find of interest on this subject:

"Space Of Spectral Sensitivity Function For Digital Color Cameras"... http://www.cis.rit.edu/~jxj1770/camSpec/

[Hening Bettermann]

Hi Tim,
thank you so much for your extensive answer - I have something to study...

[Hening Bettermann]

Hi Tim

> I'm not seeing such a "gross" difference between the first shot (exposed under the Solux) second from the Lindbloom's synth version.

No, the gross difference was between the screen image (non-Adobe) and the physical CC24.

> I had to zero out contrast, black point, switch to Linear curve and adjust exposure to get the white patch to read 96L Lab readings to get near perfect Lab numbers on every patch.

My ICC profile is linear in the first place.

> you're WAY within specs

That's good news, and I see the same as you do: see grab 1. So my custom display profile seems to do fine. (I did not check numbers myself, thinking that as long as I could see the difference with my bare eyes, it was to early to get into numbers)

> In fact I'ld like to know why your png screengrabs have "Display" as the name of the embedded profile when I open them in Photoshop CS3. Your custom display profile should be embedded or you should've converted to sRGB.

The png's were taken using the thumb button of my Logitech mouse.  Converting to my custom monitor profile does not seem to improve color accuracy, see grab 2 - because the profile is applied 2 times on top of each other?

> Here's a study you may find of interest on this subject:

"Space Of Spectral Sensitivity Function For Digital Color Cameras"... http://www.cis.rit.edu/~jxj1770/camSpec/

> Thanks for that link! It looks interesting, but it's 03:45 (a.m.) in my part of the woods now, and study will have to wait a little.

Thanks again for your detailed posts! - Hening

[samueljohnchia]

Hi

Samuel, thank you, too, for your reply.

I have fiddlet a little more with this and have learned a thing or two.
1-The comparison to the physical chart is so inconsistent depending on viewing light that I dropped it. It was just the Adobe Profile that confused me.
2- I made a quick and dirty shot of the CC24 in daylight (16:15 h winter time, clear sunshine), with a custom white balance taken right before the shot, based on the second-whitest patch of the CC. When displayed on the monitor, WB as shot, the orange looked even more reddish than before. Then I had the bright idea of using only the blue-yellow balance of the as-shot-WB, and set the Tint to 0. And look! some difference! Thus inspired, I did the same to my Solux shot, and look, that improved, too, if not by that much.

What can I do to come even closer - flash?

Here is a comparison; left to right:
1-Synthetic, 2-Daylight Tint 0, 3-Solux Tint 0, 4-Solux Tint 0 5-Solux as shot.
In case you're interested to see the whole charts, here is a zip.

Hening, although zeroing the tint value has gotten you a closer match in the orange patch, your neutral patches are definitely not neutral anymore. "2-Daylight Tint 0" exhibits a very strange cyan tint in the white patch. A more optimal camera profile will help you get a good match of the color patches, while maintaining good neutrality in the neutral patches. I have demonstrated that this is possible with many of my earlier examples, both for DNG profiling as well as ICC profiling. I highly doubt that changing the light source is going to solve your issues. Lighting is not something we can always control, especially when photographing under natural/available light. We need to look at how the camera data is transformed also. Tim's link to that excellent article makes me hopeful that we will eventually have a better way to render color out of our cameras. Hopefully we all care enough to make it happen. Just look at how many folks were interested in this thread - not many by my reckoning. I think that many others are already happy with what they can achieve, or has developed some editing trick for a color handicap (I used to do that a lot), or are simply satisfied with what is possible with today's tools. Early on in the thread Stephen G said that he can already achieve good results with current tools already. There may not be enough market pressure to up the ante on good color mapping for average users. There are so many other issues that a photographer today has to grapple with.

[Hening Bettermann]

> Converting to my custom monitor profile does not seem to improve color accuracy, see grab 2 - because the profile is applied 2 times on top of each other?

No! Because, it turns out, my custom profile was not in fact used by the monitor - the default profile was! Some software must have changed that under the table. OTOH, the profile seems quite good...but my own is a little better, the orange is visually closer to the synthetic target.

to samueljohnchia:
> Hening, although zeroing the tint value has gotten you a closer match in the orange patch, your neutral patches are definitely not neutral anymore. "2-Daylight Tint 0" exhibits a very strange cyan tint in the white patch. A more optimal camera profile will help you get a good match of the color patches, while maintaining good neutrality in the neutral patches.

Yes indeed to all of your post. -
And if Tim says that the orange is well within specs on the "Solux as shot", even my current camera profile doesn't seem too bad after all, because that one has near neutral grays all the way.

Maybe I'm ready to explore ColorPerfect now...David's theory may just be the turn of the century wrt color processing...

[samueljohnchia]

Hi Hening, I look forward to any new findings in your exploration of ColorPerfect. Feel free to add your discoveries to this thread. Unfortunately due to other work I cannot continue my investigation into this area at the moment. Additionally I have been turned off by several things that I have pointed out earlier. Iliah has also questioned David's statements in your other thread. Good luck!

P.S.
I have just received confirmation from Michael Ezra that Raw Therapee can work with dual table DNG camera profiles, but cannot interpolate between the two tables. It is as I suspected, resulting in the differences I reported earlier.

[Hening Bettermann]

So here is my first step exploring ColorPerfect.

The ColorChecker was shot in Solux light with custom WB.

For ColorPerfect, the shot was processed with MakeTiff, opened in PhotoLine with ColorPerfect, Mode ColorPos, Perfect Raw checked. ProPhoto chosen as output space. The canned profile for the camera is automatically selected. There is no option for custom profiles at this time. The CC correction was left at the default..The color mode was left in RGB, because CP does not work with Lab (strangely, since the author states that Lab preserves what he calls Color Integrity).

In Iridient Developer, CA correction was enabled. Custom camera profile in, out and as working profile. The resulting TIF was opened in PhotoLine and converted to Lab.

In RawTherapee, CA correction was enabled. Custom camera profile in and out, ProPhoto as working profile (no choice of custom here). The Tif converted to Lab in PL.

For the screen shots, the monitor profile was: 5800K, WP 100 cd, BP 0.2 cd, gamma 1.8

Mesaurements: (with the DigitalColorMeter on my screen shots, RGB spreads disregarded/averaged)
Black patch: synth. 35, CP 34, ID 23, RT 26
White patch: synth. 238, CP 255, ID 236, RT 244
Middle Gray: synth. 101, CP 100, ID 93, RT 95

My view:
RT is somewhat pale overall. Both CP and RD seem close. CP seems "clearer", but in some cases (B1, B6, C5) clearer/brighter than the synth (!) - because it pushes the highlights? Because of this, I think my nod goes to ID.
This is my view with daylight in my room - I'll check tonight...

Next post: a real image.

[Hening Bettermann]

2 oaks.
1- like above, except that the default CC correction resulted in an image with a gross blue-green cast, and was set to zero.
2- Since I still find the rendering too bluish, I tried to manually adjust the CC filter pack to visually match the ID rendering, more or less.
3-Iridient, like above. This is the closest match to my memory
4-RawTherapee, like above.

[samueljohnchia]

Thanks for sharing your results, Hening. Most of what you observe is similar to my own findings. I feel that CP tends to clip highlights a tad more with the "Black" adjustment (exposure, in the common tongue) at zero. It also has the tendency to render yellows brighter and more saturated, too much, IMO.

Do you find it more difficult to set the correct "white balance" in CP? I find it less intuitive than the conventional methods. And there is also one slider (green) that always resets to zero, with its offsets recalculated to visually equivalent red and blue offsets! Weird.

Hmm I'm surprised that your ID and RT results are this different with custom profiles. I think that they should be very close. Something could have gone wrong in the profiling process - perhaps the wrong settings in RT were used?

While the RT result looks worse in comparison, I also think that the ID results are a little unfair. The white point has been set too low, and correspondingly the black patch is too dark. The scale of 0-100 for Lab values are imprecise, so a two value difference (238 and 236) is represented with a greater difference in 8 bit notation, and even more in 16 bits. After correcting your ID screenshot for the white point, it seems to be the most accurate of the 3, colormetrically speaking. The darker patches are still a tad too dark - that may have something to do with the profile...one wouldn't want to loose precious shadow information/dynamic range to a profile with badly made curves at the shadow end! Your real world image of the tree shows the same shadow issue too. While we have wonderful shadow recovery tools to undo this effect, the adjustment curves may not be perfectly matched to perfectly eliminate the issue, or even worse, it might greater quantization errors than necessary.

I would suggest to stick in the ProPhoto RGB output space of the raw converters, and you might be interested in using Danny Pascale's averaged ProPhoto RGB target (slightly more representative than Bruce's statistically speaking, and also since Bruce was using GretagMacbeth's original values, which are different with the newly formulated versions of the CC). Assuming the color space conversions were in 16 bit precision or higher, converting to Lab for comparison is also ok. While I trust that you have an excellent display profile, that could also throw off, albeit slightly, the results that you posted here.

[Hening Bettermann]

Hi Samuel

Thank you for your comments.

> perhaps the wrong settings in RT were used?
I think I used my camera profile as input profile. Below is a shot of the CC24 with a new RT profile, with No input profile selected, ProPhoto working and out. I looks indeed different - but by no means better! The first one looks like it could be cured by a little saturation boost, but this one... see below

> I also think that the ID results are a little unfair. The white point has been set too low, and correspondingly the black patch is too dark. The scale of 0-100 for Lab values are imprecise, so a two value difference (238 and 236) is represented with a greater difference in 8 bit notation, and even more in 16 bits.

Sorry I did not specify that: the figures I gave are 8-bit RGB, not L*. On my screen, the DigitalColorMeter readings of these 2 white patches are:
synth. (Lab): 239-238-234, ID: 236-237-234 - that's not much to be adjusted? I doubt that this little difference can account for the drop in the black patch from 35 to 23.

> I would suggest to stick in the ProPhoto RGB output space of the raw converters, and you might be interested in using Danny Pascale's averaged ProPhoto RGB target

Below are screen shots of
1-the Babel average 16 bit TIF in ProPhoto
2-the ID TIF, ProPhoto all the way
3-the RT TIF with new profile, in ProPhoto all the way
The CP image was in ProPhoto in the first place.

> display profile, that could also throw off, albeit slightly, the results that you posted here.

Of course. But the original TIFs would be 126 MB the piece, even as pngs they would still be 35 MB, whereas the screen shots are only 900 kB. And my focus was on the relative differences rather than absolute values. - The weak point of my current monitor profile are indeed the blacks, where the ColorNavigator profile has a ΔE of 4.0-3.3-1.7 for RGB 0-16-32, respectively. (average ΔE: 0.6).

Best regards - Hening.

[samueljohnchia]

Sorry I did not specify that: the figures I gave are 8-bit RGB, not L*.

Goodness, sorry! It was a foolish mistake on my part - obviously L* is usually represented on a scale of 0-100 and a value of 238 or 236 is immediately obvious that it is in standard 8 bit notation - and you did indicate so in your post!

On my screen, the DigitalColorMeter readings of these 2 white patches are:
synth. (Lab): 239-238-234, ID: 236-237-234 - that's not much to be adjusted? I doubt that this little difference can account for the drop in the black patch from 35 to 23.

I measured the readouts in Photoshop - you should get 242, 243, 237 in ProPhoto RGB from the Babelcolor average chart. So the ID values are too dark, and could be adjusted slightly brighter.

Interestingly, the new screenshot that you posted below shows no problems in the shadows - maybe something affecting the shadows/contrast in ID has been set back to zero but was not in your initial screenshot example? I think this is a far more pressing issue than any of the others, since you probably will stick to this workflow? One must be certain that the profile is not the cause of this issue...

I think I used my camera profile as input profile. Below is a shot of the CC24 with a new RT profile, with No input profile selected, ProPhoto working and out. I looks indeed different - but by no means better! The first one looks like it could be cured by a little saturation boost, but this one... see below

I was referring to the possibility that wrong settings were dialed into RT to produce a sub-optimal rendering of a CC image to be fed to the profiling software. Or perhaps the wrong settings were used to render the CC image with a custom camera profile? Yeah, the colors look pretty off. I presume that the "new" profile is "no input profile"? That would certainly result in a poor match to the synthetic CC. The custom profile route can give almost exactly the same look you get from ID, as long as you get the WB to match (and that requires different WB settings, a real pain) and also setting the white point correctly. Though if you aren't planning to switch to RT, this shouldn't bother you too much?

Below are screen shots of
1-the Babel average 16 bit TIF in ProPhoto
2-the ID TIF, ProPhoto all the way
3-the RT TIF with new profile, in ProPhoto all the way
The CP image was in ProPhoto in the first place.

> display profile, that could also throw off, albeit slightly, the results that you posted here.

Of course. But the original TIFs would be 126 MB the piece, even as pngs they would still be 35 MB, whereas the screen shots are only 900 kB. And my focus was on the relative differences rather than absolute values. - The weak point of my current monitor profile are indeed the blacks, where the ColorNavigator profile has a ΔE of 4.0-3.3-1.7 for RGB 0-16-32, respectively. (average ΔE: 0.6).

But at the end it is still a PNG screenshot that you provided, converted to your display profile, which is the (small) issue I raised. Saving out a downsampled PNG of the same pixel dimensions, but in ProPhoto RGB, instead of doing a screenshot is one possibility - I'm not requesting for those full size tiffs!  No big deal though.

[Hening Bettermann]

Hi Samuel

thank you for your detailed answer.

Yes the new input profile for RT was "no profile".
I have now prepared a new RT with custom camera profile input, and indeed, the look is very close to the others.  To me, they look all great. CP may became even better if and when custom camera profiles will be supported. Or if I can use ID output with my custom proifle, but otherwise without CM (which should be possible).

I think will chose CP for now. One thing is the colors of a CC shot under controlled conditions, out of the box with no further edits; another question is what will happen to these colors when we apply curves. I remember discussions about hue shifts, I tried Tindemann's Tonability plug-in, but could not make it work. (Also, it's Windows...).  

Of course, the differences may be small in real life, where tonal edits (mine) are usually moderate. But I prefer something that seems to be theoretically correct at the basis.  

A problem will be to fit CP into a worflow with multiple stacking, since it requires completely unadultered raw files. This MAY become a show stoppper, at least in part. But that's another story.

Best regards - Hening.

Edit:
I see that the Babel is al off in the rendering here. I have seen the same with some images when opened in Preview. Opened in PhotoLine on my Mac it looks OK. ??

[Hening Bettermann]

The problem with the Babel is solved: I had not assigned ProPhoto (PL does this automatically without dialog). Here is the profiled version. (I see no option to remove attachments).

Edit: no, it did not help. Then I don't know.
Edit: I'll try 'Convert' instead of 'Assign'
Edit: Nee, that didn't help either. Then I really don't know.

[Hening Bettermann]

And here cometh what really sells me to CP: scanned roll film images as 3f files.

The film was Kodak Ektachrome Prof. Daylight 200 ISO 70 mm. The scans were done on an Imacon X5 by digitalcopy24.de.

The files are opened in PhotoLine without a profile. Assigning the scanner profile is no improvement, on the contrary.

1 and 3 are test charts shot in sunny mid-daylight, 4 and 5 is a scene in Swedish Lappland. For the latter, it is impossible (for me) to achieve anything trustworthy using color balance controls in PL.

CP pushes the highlights, but that could be handled.

Edit: Sorry, something has gone wrong with #1 and 3. I'll try to fix it and send them in a new post.  

[Hening Bettermann]

the message body requires text

[Hening Bettermann]

And here is my best effort on this image, before sharpening. The exposure ('Black' in CP) adjusted to retrieve the highlights, the color balance adjusted guided by the original transparency. No tone curve.
For comparison: the default rendering in PhotoLine, with the scanner profile applied. (I was wrong, the scanner profile is not worse than the PL default, it is better, but no way near CP).

[Ernst Dinkla]

Related to this subject, on the Apple Colorsync list a discussion is going on after Ben Goren wrote an article on exposure, threadname:
"Primer on photographic exposure, etc."

Ben Goren's  article:
http://trumpetpower.com/photos/Exposure

The thread is as interesting as the article.

--
Met vriendelijke groet, Ernst

http://www.pigment-print.com/spectralplots/spectrumviz_1.htm
December 2012, 500+ inkjet media white spectral plots.

[Schewe]

Ben Goren's  article:

Yeah, ya know, this guy is a jazz trumpeter with a lot of time on his hands (cause he doesn't blow his horn enough) and finds interesting ways of dissing Camera Raw...

Yeah I read it...kinda like Dan M. where first you need to make your image look like crap before magically fixing it in the end in Photoshop. (it's "magic").

One is better off learning the ins/outs of the hot new raw processor de jure (that you want to use) and actually set the optimal settings for your raw image...

Learn to make your images look the way you want them to look...regardless of what you have to do to get there! There is no magic bullet! There is skill and judgement involved...

[Ernst Dinkla]

Learn to make your images look the way you want them to look...regardless of what you have to do to get there! There is no magic bullet! There is skill and judgement involved...

It was not my intention to encourage further exercises into camera profiling but more to frighten the ones that would like to face that complexity. The linked thread adds to that. Within the limited scope of art reproduction it could work however and is more needed. I expressed my doubts on the general use in a similar thread before this one. So my thoughts are not that much different from yours but I expressed them otherwise :-)

--
Met vriendelijke groet, Ernst

http://www.pigment-print.com/spectralplots/spectrumviz_1.htm
December 2012, 500+ inkjet media white spectral plots.

[Vladimirovich]

There is no magic bullet!

but the sentiment of 99% of $99 (?) ColorChecker users is like - I can buy a cheap 24patch target, shoot it craplike in a daylight (sic), use PE (which Adobe itself is not using) and always get a "better" "color" right out of the box than when using Adobe's supplied profiles...  just like that... placebo effect or snake oil...