DCamProf - a new camera profiling tool

[markanini]

how do you know that they are indeed accurate though ?

I don't, by default smootness is prioritized over acuracy and lightness is uncorrected. Nevertheless The DCamProf generated profile is closer to my memory of the scene.

[torger]

In this particular context it was two very different hues. Blue vs purple. You don't need very accurate color memory to remember if a color was purple or if it was blue. Unless it's a very strange light condition I expect a camera profile to at least be in the same ballpark concerning hue.

For true accuracy we need a controlled light condition and work reproduction-style, of course.

But if you just add some sense of proportionality you can also make judgments of general purpose profiles in varied conditions.

[markanini]

While waiting for better light (and weather) for shooting a charts I made a profile using EXIF matrix data from my Nexus 5 DNGs. Another gallery: http://imgur.com/a/ydILx/all

[one/and/a/half]

Hi. Does anyone have a A7R (original, I) profile made with DCamProf, preferably dual illuminant.

I was contemplating on a better place to ask this question, but failed.

[markanini]

Are these colorchecker shots useful? www.dpreview.com/reviews/image-comparison I notice some uneven lighting on the tungsten version.

[torger]

Are these colorchecker shots useful? www.dpreview.com/reviews/image-comparison I notice some uneven lighting on the tungsten version.

I have never tried them, but you can probably make ok profiles from them despite the uneven light, just make sure that you don't try to match lightness too much as that axis is not to be trusted with such uneven light. Imaging Resource's test shots are much better so if the camera you're interested in has been tested there it's a better choice.

[markanini]

I have never tried them, but you can probably make ok profiles from them despite the uneven light, just make sure that you don't try to match lightness too much as that axis is not to be trusted with such uneven light. Imaging Resource's test shots are much better so if the camera you're interested in has been tested there it's a better choice.

Thanks, I went with the Imaging Resource shot for profiling.
I found another example where DCamProf produces a subjectively less pleasing rendering or very saturated reds, if you have time to study the issue: http://filebin.net/0s1iarkd77
EDIT: One more: http://filebin.net/2zwxlq2l93

[torger]

Thanks, I went with the Imaging Resource shot for profiling.
I found another example where DCamProf produces a subjectively less pleasing rendering or very saturated reds, if you have time to study the issue: http://filebin.net/0s1iarkd77
EDIT: One more: http://filebin.net/2zwxlq2l93

Thanks very much for the examples. I've had a quick look and notice that it's artificial lights, likely narrow band, very high saturation. These colors are the most difficult as they excite the camera filters in unpredictable ways, the CC24 patches are not close to it. In general commercial profiles often have better robustness when it comes to this type of "extreme colors". This robustness seems to come at a price though, for example unrealistically low saturation in the normal range (to make a smoother transition into extreme colors).

I have done quite a bit of work to handle extreme colors better in DCamProf but I don't think I'm fully there yet, so I usually recommend to have another profile laying around to try when you have strange lighting conditions like this.

At some point I may have a look and try to do further work on these types of light sources, but it's surely not a "quick fix", and I'm not sure it's even possible to fix without certain sacrifice in the normal range (I suspect the only robust solution is a pretty strong gamut and wide compression). So I can't provide a fix for it at this time unfortunately.

With narrow band emissive colors like this one cannot really strive for correctness, as it will hurt performance of normal colors. So what one want to achieve is smooth gradients and avoid ugly clipping.

With all that said looking at your pictures DCamProf seems to do as intended. When a color gets very very bright and saturated there's a tradeoff to either clip or to desaturate. By desaturating you get better tonality (more visible shades) but with a less realistic desaturated color. I worked quite a bit with this to get a balance based on test images of red roses Bart sent to me (which I artificially converted to different colors to test highlight transitions for various colors).

In your particular image I too prefer the flatter rendering of the portrait profile provided, but in other cases, such as the roses or other detailed natural subjects, I prefer to have visible detail left rather than just flat color. The problem is that it's not possible to make one profile that makes the best tradeoff for all image material.

I'm very interested in hearing what you think about these particular images. I assume that you don't like that DCamProf desaturates the bright parts and prefer the more saturated rendering of the provided Portrait profile, but I'd like to hear so it's not some other issue I'm missing.

[torger]

Added here crops to show what I was talking about above.

The portrait profile gives a smoother flatter rendering of the bright reds, while DCamProf makes tonality differences more visible in those areas. For this particular subject I prefer the portrait profile result, but as said when testing for various subjects I found it better overall to keep detail at the cost of saturation/uniformity.

If you prefer a different behavior this can be controlled using the neutral tone operator configuration, the "Curve" section. (See data-examples/ntro_conf.json). It's been a while since I was deep into the code so I'm myself a bit rusty on it but I think you should be able to get a similar result as "portrait" by changing KeepFactor to 1.0. From the documentation:

        // The curve is always applied in the luminance channel, but the output luminanince can be derived from a
        // RGB-HSV curve, meaning that the resulting lightness will then be the same as a pure RGB-HSV curve.
        // For low saturation colors there's often a perceptual advantage of using the RGB-HSV curve, otherwise the shadow
        // dip of an S-curve may look too dark and a bit dull. For high saturation colors the RGB-HSV curve causes tone
        // compression (especially in the red range), ie poor tone separation and a less realistic look.
        // Therefore we here control a transition from the RGB-HSV curve to a pure luminance curve based on chroma of
        // input color.

[markanini]

I'm very interested in hearing what you think about these particular images. I assume that you don't like that DCamProf desaturates the bright parts and prefer the more saturated rendering of the provided Portrait profile, but I'd like to hear so it's not some other issue I'm missing.

Yes, the extreme reds appear desaturated and may shift earlier to orange and yellow. I've also seen the rose shots and I think you ended up with the best possible rendering for that particular subject. It's strange to realize that sometimes less color detail is preferable. I will experiment with KeepFactor.

EDIT:KeepFactor 1.0 did indeed make rendering closer to the portrait profile.

[eidsheim]

Hi guys,

I've been following this thread with interest and would like to thank you @Torger for making a profiling tool for C1 and @AlterEgo for providing a Windows compile. So far, I have had mixed results to be honest. The profile I get by following the ICC tutorial clips certain strong colours (as has been discussed here recently). Let me know if you would like to have a look at some examples.

As a passionate amateur I have no access to a studio, fine art printer or controlled light setup (most of my photos is taken outside, at night or in mixed lighting anyway) and have to rely on CC Passport and web resources. I bought CC SG but will be returning it as I understand the glare makes it useless outside.

I have downloaded RawDigger and would like to test how it performs with DCamProf as I like having a GUI and it has some export options. So far, I have not found any information regarding using the .txt file from RD in creating a profile. Could someone please help me out with setting this up and the commands for getting it to work? Optimal RD settings?

One last question for now: Any suggestions as to creating a robust profile for different lighting conditions with the above limitations in mind? Or do I need to create one for each standard temperature for example?

Thanks for any help!

Ole

[AlterEgo]

I have downloaded RawDigger and would like to test how it performs with DCamProf as I like having a GUI and it has some export options. So far, I have not found any information regarding using the .txt file from RD in creating a profile. Could someone please help me out with setting this up and the commands for getting it to work? Optimal RD settings?

well, you need to read the manual and rawdigger website first... for exampe for my DCamProf usage (actually for the matlab script that approximates SSF/CMF to be used in DCamProf) when exporting the raw data ("selection grid" -> "save samples to file") I do not use WB or data scaling/gamma, I do use flat fielding naturally ... if you are using C1 then you can combine rawdigger and makeinputicc GUI frontend for argyll to stay fully GUI (not as flexible as DCamProf of course)... in that case rawdigger must use WB and data scaling/gamma (either around 1.8 for C1 or you can modify data yourself with actual tf curve data from C1 for your camera model, etc) when outputting CGATS for argyll.

[AlterEgo]

So far, I have not found any information regarding using the .txt file from RD in creating a profile.

you need to combine the raw data exported from RD (after flatfielding, WB and scaling/gamma - whatever is necessary for your specific workflow...) with the target description (like cieXYZ/D50 numbers for patches) ... patch for patch matching

[markanini]

I want to report about a few shots I have where KeepFactor 1.0 made some extreme purples look too dark, whereas the default KeepFactor 0.0 looked more realistic to me. If you recall, for extreme reds we preferred KeepFactor 1.0. I hope everyone is having a happy easter!

[markanini]

Not to be conflated with the issue brought up in my previous post, I noticed extreme purples were rendered as "blobs":



Top: DCamProf Neutral+, default settings
Bottom: Camera Portrait

[torger]

Interesting artifact. If you can share raws I'd like to have a look and see what the reason is. It could be a unavoidable effect due to a trade-off, or maybe I need to make some tuning to the clip handling. I need to look at the raws to be able to figure out what it is.

[AlterEgo]

Not to be conflated with the issue brought up in my previous post, I noticed extreme purples were rendered as "blobs":

I think with such light you really just go with matrix profiles - no LUTs...

[markanini]

I think with such light you really just go with matrix profiles - no LUTs...

Good idea.

Interesting artifact. If you can share raws I'd like to have a look and see what the reason is. It could be a unavoidable effect due to a trade-off, or maybe I need to make some tuning to the clip handling. I need to look at the raws to be able to figure out what it is.

PM sent.

[torger]

Actually the normal matrix-only profiles will make it worse, the reason is that a matrix profile is optimized for "normal colors", meaning that ultra-saturated colors like this are pushed way out into negative and positive clipping.

You can however design a very low saturation matrix without any clipping, but the colors won't be that realistic. In fact if you open the file in RawTherapee, use the DCamProf profile and disable the LUTs (Base table and Look table checkboxes in RT) you have exactly that -- a low saturation no-clipping matrix.

When you then apply the base table the colors are stretched back into proper saturation, which causes a the blob issue with the magenta highlight. It will take quite some time for me to drill down to the exact reasons why the transition into clipping is not nicer, I'll have a closer look when I get some more time.

I had a similar issue with artificial blue light in a A7r-II image, it was a while ago so I don't remember the conclusion but it was something along the lines that in order to keep proper saturation and hue close to clipping it was impossible to make a really smooth transition into clipping. I'm not sure it's that which is happening here though...

For now I have to say that robustness of super-saturated clipped colors with high blue content is a known issue, and in those situations you may be better off using a different profile.

[AlterEgo]

You can however design a very low saturation matrix without any clipping, but the colors won't be that realistic.

for that subject in such light do we really need realistic colors or rather no artefacts from improper LUT in that areas ?