DCamProf - a new camera profiling tool

[howardm]

Version 0.10.1 for Mac OSX now available.

[AlterEgo]

0.10.1 build for Windows ( mingw = dcamprof.exe + libgomp_64-1.dll + both manual & tutorial / = copies of Torger's web pages / in 3 formats : IE archive .mht, Mozilla archive .maff and regular .pdf ) : https://app.box.com/DCamProf

[StephK]

Thanks Torger for Version 0.10.1 and AlterEgo for the Windows build.

I had noticed some clipping in the previous version but otherwise has been excellent.

[torger]

I'm now working on refining high saturation color handling of the neutral tone reproduction operator. It's currently a little bit over-saturated, at least in some hue ranges (got some issue report about over-saturated blues, it's always about the blues it seems ). I'm hoping to be able to keep the saturation control hue-independent like it is today, but I'm not sure, maybe blue range needs (yet) some extra handling.

The tone reproduction has so far been tuned mostly against normal range colors, faces and such and in that range the principle has been that rather have a tiny bit too much saturation than too little, as the underlying luminance curve has a tendency to make things look grayer than they are. However I'm probably going to do the other way around for high saturation colors, rather present them with a tiny bit too low saturation.

[torger]

Found the cause of the over-saturation, mainly seen in dark blue colors. In the tone reproduction operator there's a chroma push of shadows that is there to counter-act the grayness of the luminance curve. This works well on faces and other low saturation material, but is no good on higher saturation colors. Blue is most affected as blues are dark and thus get into the shadowy range.

I will fix this to the next release (the fix is easy, simply rolling off the effect as saturation increases).

[ErikKaffehr]

Hi Anders,

Are you having the "blues" blues today?

Best regards
Erik

I'm now working on refining high saturation color handling of the neutral tone reproduction operator. It's currently a little bit over-saturated, at least in some hue ranges (got some issue report about over-saturated blues, it's always about the blues it seems ). I'm hoping to be able to keep the saturation control hue-independent like it is today, but I'm not sure, maybe blue range needs (yet) some extra handling.

The tone reproduction has so far been tuned mostly against normal range colors, faces and such and in that range the principle has been that rather have a tiny bit too much saturation than too little, as the underlying luminance curve has a tendency to make things look grayer than they are. However I'm probably going to do the other way around for high saturation colors, rather present them with a tiny bit too low saturation.

[torger]

I've just released v0.10.2.

Mostly tunings of the tone reproduction operator, including the important over-saturated shadows fix.

[Frederic_H]

Thanks, looks good, blues are still a tad more saturated than the rest but it probably won't bother many people.
I'll wait it 'stabilizes' a bit before attempting to design looks again.

[AlterEgo]

0.10.2 build for Windows ( mingw = dcamprof.exe + libgomp_64-1.dll + both manual & tutorial / = copies of Torger's web pages / in 3 formats : IE archive .mht, Mozilla archive .maff and regular .pdf ) : https://app.box.com/DCamProf

[howardm]

Version 0.10.2 for Mac OSX now available

[mgrayson]

+2 to AlterEgo and howardm!

[torger]

The design principle I still go by with the tone reproduction is to assume that the base colorimetric profile is accurate, and then the purpose of the tone reproduction is to maintain the same color appearance, not increase or decrease saturation. That is I'm not really designing "by taste", but by color appearance matching. Then there's some taste and judgements involved anyway of course as it's not possible to make perfect and then one must make tradeoffs.

With the ACR tone curve doing A/B swaps with a curve-less (colorimetric) rendering pushed 0.7 stops to equalize brightness is a base technique I'm using. The adjustments I've done recently has been possible to demonstrate as mismatch when doing these swaps on new test images I've received.

Blue is a little bit special as discussed in several past posts which has made it necessary to work with more tradeoffs. However with this last release I think I have quite good A/B matching with all the test material I have got. The error fixed in this last release was demonstrated in a small deep blue traffic sign in a Pentax 645z shot that was sent to me.

So if any of you has a test shot that demonstrates a clear mismatch I'm interested to have it if I may. The recent fixes has all been initiated by user contributions.

There are two major aspects to know about when it comes to camera profiles and hue accuracy. Nearly all bundled/commercial profiles renders colors warm (more yellow) than accurate, and nearly all renders blue much lighter and not rarely more cyan than accurate. In the tone reproduction operator I don't design towards these targets, the aim is accurate matching with the colorimetric profile as long as possible. With blue I had eventually to go lighter for technical reasons, but it's still rendering darker than most bundled profiles that go even lighter.

Then the idea that a custom look is designed on top of this, my "Neutral+" profiles add more yellow etc. I haven't added any specific blue adjustments to it though, I haven't really decided on that yet.

From the same person I got the blue road sign over-saturated issue, I also got a forest greens lacking yellow issue, I did not do anything about the latter as I judged that to be one of those bundled yellow-rendering profiles things.

Hue is not likely to be bad, except possibly in extreme ranges and such. Saturation matching is more tricky, and then we have this rolloff (desaturation) into clipping which is sort of all about taste. Saturation handling in the extreme range is also a bit about taste more than about matching.

[torger]

Thanks, looks good, blues are still a tad more saturated than the rest but it probably won't bother many people.

Despite forced lightening due to technical reasons as discussed in past posts, DCamProf's default blues are typically rendered considerably darker (and as such more realistic) than the typical bundled profile. However a darker (deeper) blue may look more saturated, especially when we've used to see lighter blues from bundled profiles. I've tried to under-saturate it a bit to compensate, but then it goes into grayness instead. For the "blue" test shots I have I'm pleased with the level of saturation produced by v0.10.2. However if you have some shot where over-saturation is seen I'd love to have a look to see if further tuning can be made.

There's a nagging thought in the back of my head that tells me that maybe we must render blues light light light (and even cyan-polluted) like most do in order to get pleasing tonality etc, so maybe even more "hacks" need to be made. So far I don't really think so (blues are already considerably lightened), but it's too early to say that I've "landed" the issue.

[Frederic_H]

Actually I was comparing the rendering against 0.9.x profiles. Saturation looked better balanced across the whole hue range.
You are right it may be darker blues mistaken for more saturated ones though, will keep on testing and send files if needed.

[hk1020]

From a personal message

I've now released v0.10.0 so you can play with the adjustments yourself.

Based on a cc24 imaging resource shot (rather than nex5.json) I got good results wiith this command:

dcamprof make-profile -a target-adjust.json cc24.ti3 my-profile.json

Where the target-adjust.json contains the following:

{ "PatchAdjustments": [ {  "Name": "C01", "ScaleRGB": [ 0.8, 1.0, 1.0 ] } ] }

If you use nex5.json you may need more aggressive scaling, like increasing green to 1.2 from 1.0.

I haven't tried 0.10.2 yet but proceeding as above with 0.10.1 I get much better results than before.  But certain blues still tend toward purple.  My aim is to get usable profiles for C1 as their profile for NEX6 are terribly reddish, so much so that I find them unusable.  But if I compare them now with dcamprof they have a remarkably good separation of various blues, which dcamprof doesn't quite match yet.  I will play a bit more with above patch adjustments. I feel some blues are really wrong now but it is in situations where I can't say what is right.

I did notice darker blues as well but found them quite good and closer to reality. I'll see if I can find more examples to post. And considering saturation the new profiles are generally more saturated it seems. I don't think that is an improvement. Again I might find an example to post later. Just thought to let you know what I found so far. Thanks for all the work.

[torger]

From a personal message
I haven't tried 0.10.2 yet but proceeding as above with 0.10.1 I get much better results than before.  But certain blues still tend toward purple.  My aim is to get usable profiles for C1 as their profile for NEX6 are terribly reddish, so much so that I find them unusable.  But if I compare them now with dcamprof they have a remarkably good separation of various blues, which dcamprof doesn't quite match yet.  I will play a bit more with above patch adjustments. I feel some blues are really wrong now but it is in situations where I can't say what is right.

I did notice darker blues as well but found them quite good and closer to reality. I'll see if I can find more examples to post. And considering saturation the new profiles are generally more saturated it seems. I don't think that is an improvement. Again I might find an example to post later. Just thought to let you know what I found so far. Thanks for all the work.

0.10.1 has an over-saturation issue in shadows (and deep blue reaches down in that range), so do test 0.10.2 before evaluating the level of saturation.

There's a tonality problem with rendering blues accurately dark which is that the eye is less sensitive there and thus you don't get as good tonal separation as if you transform the color into lighter more cyan tones. DCamProf does lighten blues today related to technical issues, but still keep it at a fairly realistic level. My intention is to not pull it further from realistic than technically required, and then if you want to improve tonality at expense of accuracy/realism you do that with possibly pre-adjusting blue (through target adjustment) and the rest with a look operator.

Your camera is of special interest though as it seems it's hard to measure accurately and/or make a suitable matrix for that keeps blues stable, that is there may be issues directly in the colorimetric aspect of the software rather than in tone reproduction. It could also be that the CC24 does not work well with the camera due to its special blue response, that is there would have to be more blue patches in order to make more non-linear LUT corrections.

Anyway, let us know what you find.

[torger]

When calibrating using a CC24, the deep blue is much decided by the C01 deep blue patch. This patch is very dark, and as such it's very sensitive to glare. If the measurement contains a large amount of glare then the software will think that the camera renders blues light and desaturated (as they become by glare) and will thus compensate by darkening and saturating.

One theory is that the blue issues some are experiencing could be related to this, that is that the tone reproduction operator may translate the blues from colorimetric profile accurately, but that the blues are too dark and saturated in the colorimetric profile itself due to measurement error.

If this is the case systematically I may need to revisit the glare compensation feature. When I did glare compensation (linearization) a few months back my conclusion was that it did not really work unless the glare was small in the first place, and even then it was not stable. You can play with glare compensation in testchart-ff command, but in normal workflows there's no glare compensation made. I'm thinking if you put the requirement that glare must be low one might be able to make some strongly simplified model which is simply modifying saturation and lightness of the reference values depending on their original lightness. Before I put an effort into that I must know that there actually is a measurement issue concerning blues.

If you do suspect an issue, I strongly suggest to also render a pure colorimetric profile (that is without a tone curve) and test that too to see if the problem is that the color is wrong in the colorimetric profile, or if the problem is due to the tone reproduction (that is the color transformations that take place when the curve is applied). As a curve generally brightens the image a lot too, to be able to compare before / after curve (switching between your two profiles) pushing the exposure with about 0.7 stops of the colorimetric profile is often helpful.

[torger]

Here's a screenshot from http://colorizer.org/ (I recommend that warmly if you need RGB sliders to play with) showing the issue of tonality and rendering accurate deep blues.

Here I'm comparing the transition from blue to cyan with blue to black. If you lighten blue and push it towards cyan you get more of the cyan range, while if you render deep blue more accurately you get more towards the blue-to-black range. It's much easier for the eye to separate tones in the lighter blue-to-cyan range.

In this screenshot you can also see why it's better to push deep blue towards cyan (that is add in more green) than it is to push it towards purple (add in more red). While cyan sort of looks like a "brighter/overexposed blue" and makes up a nice transition, if you taint blue with red it becomes purple which clearly is a vastly different color.

You can also see in the left range of the R/G sliders (purple and cyan scale) that blue with little red in it looks exactly the same as blue with very little green in it. There's simply many deep blue colors that look virtually the same.

[mi-fu]

Version 0.10.1 for Mac OSX now available.

hi howardm,
I downloaded your complied DCamProf to my OSX (10.10.4), but it is not executable. Do I miss something?

[howardm]

I doubt you did something wrong.  Have you downloaded other versions that I've created?

What appears to have happened, and this is not overly unusual when doing file transfers is that the Unix 'execute' bit
gets turned off (for security purposes).

So, you often have to turn it back on.

If you do a   'ls  -lt  ./dcamprof-v0.10.2'  command, does the left side show something like rw-r--r--   

If so, then do a  'chmod  755  ./dcamprof-v0.10.2'  and then you should be able to run it.

I will say that the various versions have been downloaded a total of 34 times and this is the first 'problem' report so
I dont think it's an actual problem, just a user education/knowledge issue.

I suppose if I 'zip'd' the file before I uploaded then the bit would remain set but that would force people to unzip.  Potato-Potatoe