DCamProf - a new camera profiling tool

[torger]

Nice to get some feedback, I'm very glad when people find it useful.

The current version works well except in extreme ranges, that is it has some issues with clipping (like sunsets) and super-saturated colors, especially deep blue. That will be fixed in the coming version, which however takes forever to complete I'm guessing 2 - 4 more weeks now.

Wohoo, reply #1000!

[Alexey.Danilchenko]

Nice to get some feedback, I'm very glad when people find it useful.

I must join then and say that I am finding it very useful - thanks for writing and maintaining it!

So far I used it to build a very good matrix profile from captured SSF (simulating CC SG target) and that was really an easy thing to do.

[ErikKaffehr]

+1, no += N!

Erik

Torger a big THANK YOU for DCamProf.  I was very unhappy with/disagreed with C1's interpretation of colour for the new 5DSR.  My photo partner has tried other software to generate a custom ICC profile for our camera and C1 with limited success.  Unfortunately, the previous profile he attempted with other software disallowed the use of the color editor in C1. 

DCamProf though has given us what looks to be an excellent profile to work from with full functionality in C1.  We only created the profile yesterday and much of it is over my head but I've been reading through much of your text to better understand it all. 

Thank you for creating such a comprehensive resource in understanding and applying custom profiling and I look forward to working with my custom profile.

[ErikKaffehr]

Sorry,

I of course mean *= N! and make that N big!

Best regards
Erik

+1, no += N!

Erik

[torger]

I've now released v0.10.0.

Going from 0.9 series to 0.10 may seem likes it's going from 90% complete to 10% complete, but hopefully I'll do a jump to 1.0 sooonish. I just had to make such big changes under the hood for the extreme value handling so I had to at least bump minor version.

There's a long list of changes on the home page. There's a bit of behavior change.

In terms of look it should be the same except clip bug fixes, plus on major change in blue handling. Cameras with "over-sensitive" blue channels (many sony sensors) will now render blue considerably lighter than before, and all cameras will render it a little lighter. The reason is a default negative factor matrix limiter, which can be turned off if you really want the dark blues traded for strong LUT bends at the gamut edge.

There's so much changes I expect a few minor updates before it fully stabilizes.

[Frederic_H]

May be something with my system (OS X) but make fails on argyllio.c
I'll have a look tonight.

argyllio.c:86:9: error: unknown type name 'ssize_t'; did you mean 'size_t'?
        ssize_t linefeed = -1;
        ^~~~~~~
        size_t
/usr/include/sys/_types/_size_t.h:30:32: note: 'size_t' declared here
typedef __darwin_size_t        size_t;
                               ^
argyllio.c:87:14: error: use of undeclared identifier 'ssize_t'; did you mean 'sizeof'?
        for (ssize_t i = 0; i < (ssize_t)data_size; i++) {
             ^~~~~~~
             sizeof
argyllio.c:87:22: error: use of undeclared identifier 'i'
        for (ssize_t i = 0; i < (ssize_t)data_size; i++) {
                     ^
argyllio.c:87:29: error: use of undeclared identifier 'i'
        for (ssize_t i = 0; i < (ssize_t)data_size; i++) {
                            ^
argyllio.c:87:34: error: use of undeclared identifier 'ssize_t'
        for (ssize_t i = 0; i < (ssize_t)data_size; i++) {
                                 ^
argyllio.c:87:53: error: use of undeclared identifier 'i'
        for (ssize_t i = 0; i < (ssize_t)data_size; i++) {
                                                    ^
argyllio.c:88:22: error: use of undeclared identifier 'i'
            if (data == '\n' || data == '\r') {
                     ^
argyllio.c:88:41: error: use of undeclared identifier 'i'
            if (data == '\n' || data == '\r') {
                                        ^
argyllio.c:89:48: error: use of undeclared identifier 'i'
                if (linefeed == -1) linefeed = i;
                                               ^
argyllio.c:91:52: error: use of undeclared identifier 'i'
                if (linefeed != -1 && isblank(data)) {
                                                   ^
argyllio.c:92:40: error: use of undeclared identifier 'i'
                    while (linefeed != i) data[linefeed++] = ' ';

[torger]

Thanks for testing, fixed that. Re-download it again.

[Frederic_H]

Fixed, thanks. Will test over the weekend.

[torger]

For those making their own custom looks using look operators I've become aware of that if you change increase saturation or modify hues etc close to the neutral axis there's a risk that you disturb noise reduction algorithms. I've only observed this as a real problem in Lightroom, but I guess it can happen elsewhere too.

A way to fix this is to roll off your effects towards the neutral axis. A common subjective adjustment is to make "neutrals more neutral" and if you have that type of adjustment you have already fixed this as a side effect. Reducing chroma of close-to-neutral colors help noise reduction as a side effect.

[howardm]

Version 0.10.0 for Mac OSX now available.

[AlterEgo]

0.10.0 build for Windows ( mingw = dcamprof.exe + libgomp_64-1.dll + both manual & tutorial / = copies of Torger's web pages / in 3 formats : IE archive .mht, Mozilla archive .maff and regular .pdf ) : https://app.box.com/DCamProf

[torger]

The tutorial has quite much updates, a new section on active matrix design:
http://www.ludd.ltu.se/~torger/photography/camera-profiling.html#step6_matrix
which you did not do before. It's rarely something you need to care about though unless you are specially interested and like to make your own tradeoffs.

However I've come across one camera that required some manual matrix design to get good results, the Sony NEX 6. The strongest blue subtraction factor ever and quite noisy too, a normally/automatically designed profile caused the deep blues to be incorrectly pulled against purple. By using the new target adjustment feature (pushing blue against cyan to give it some margin) and the matrix limiter it's possible to control the color also for that though.

[Frederic_H]

Built and tested a simple neutral profile on several pics, and blues look more saturated than other hues. Is it an expected side effect of the recent changes ?

[torger]

Built and tested a simple neutral profile on several pics, and blues look more saturated than other hues. Is it an expected side effect of the recent changes ?

Thanks for taking time to post images, it's really worth a thousand words as they say

As usual I'm not at my workstation with the calibrated screen so I must look at it there when I get there later. On my less-than-perfect screen here it also looks like the sky is considerably less magenta. The difference looks pretty big, so something must have happened.

I don't know what though. I shall look at some plain sky test shots and see if I can figure it out.

I've got one more complaint on "too saturated blues", but not in skies but on more saturated blues. That's on a 645z where the blues got a lot lighter, increasing lightness does increase saturation as a side effect but that should have been compensated for.

Considering hue, do you know which one of your shots that is most true to the original scene? I know P1 renders stuff quite warm (yellow), but anyway the blue in skies can turn towards magenta, cyan and indeed yellow depending on atmospheric conditions and sun position.

I think the change in hue may be due to a change in rolloff algorithm, but it changed as early as 0.9.3. In earlier versions it was more pure RGB-HSV-hue-oriented and HSV-hue has specifically an issue in the blue range where blue turns magenta. Since some time that is stabilized and hue should be kept more constant. It might be that effect we see regarding hue, but it doesn't match the claimed version so it's probably something else then.

I need to investigate a bit anyway.

[Frederic_H]

I'd say P1 and dcamprof 0.10 are closer to the 'true' sky hue, dcamprof 0.9.10 (or 11) is too magenta. I'm used to correcting skies as LCCs seem to mess a lot with hues in this range (take them to magenta territories), so the hue shift is not what strikes me most here.
The brighter part of the sky, the first half starting from the horizon here, feels almost electric with dcamprof 0.10. According to my tests on different pics it looks like only a limited range exhibits this over-saturated blue phenomenon, maybe it's only a matter of smoothing that new specific blue handling.

[torger]

I'd say P1 and dcamprof 0.10 are closer to the 'true' sky hue, dcamprof 0.9.10 (or 11) is too magenta. I'm used to correcting skies as LCCs seem to mess a lot with hues in this range (take them to magenta territories), so the hue shift is not what strikes me most here.
The brighter part of the sky, the first half starting from the horizon here, feels almost electric with dcamprof 0.10. According to my tests on different pics it looks like only a limited range exhibits this over-saturated blue phenomenon, maybe it's only a matter of smoothing that new specific blue handling.

Looking at my workstation screen now, and I can see now. There's been more than one trick to handle the blue range so this may be a side effect from that. I'm going to investigate.

Will investigate.

[torger]

The change in sky blue saturation sits in the updated rolloff handling in 0.10.0. The change was not intended. To quickly restore the old behavior, make this change in look.c (any large number will do so the alternate rolloff is never triggered):

--- a/look.c
+++ b/look.c
@@ -887,7 +887,7 @@ look_tone_rep_op(v3 *out,
             // (test on skintone highlights), so we make a fade between
             const double acr_chroma_lo_sat = acr_jch.v[1];
             const double acr_chroma_hi_sat = jch.v[1] * (acr_jch.v[1] / orig_jch.v[1]);
-            const double lolim = 10;
+            const double lolim = 1000000000;
             const double hilim = 60;
             if (orig_jch.v[1] < lolim) {
                 acr_chroma = acr_chroma_lo_sat;

The old has issues with gradient smoothness in sunsets though, so I need to make some better fix.

[Frederic_H]

It looks better now (new version on the right).

[Tim Lookingbill]

I see a hue shift in your sky blues, not saturation. I do see the P1 sky blue as having too much green.

DcamProf nails the UV bias the sun provides in cloudless daylight sky blue. Greenish skies are inaccurate, unless you have a lot of air pollution where that was taken.

[torger]

After a little gradient testing I have now released v0.10.1 which contains the hotfix of the rolloff (important fix!), in essence the same result as in the one-liner patch a few posts back.