DCamProf - a new camera profiling tool

[torger]

Just released version 0.9.13, fixed a clipping bug (=ugly sunsets) and improved/changed gamut compression algorithm.

[torger]

Perhaps my error was using the linearized cc24.tif exported from C1 in the earlier workflow. I used Raw Therapee instead and have gotten further.

Alas - the resulting profile turns everything a bright green.

Here are dropbox links to the linear targets saved from C1 and Raw Therapee. They are clearly different. I am completely new to RT, so have no confidence in that sample. I clicked the "output TIFF for calibration" button.
 
https://www.dropbox.com/s/evvtmteueabeopn/C1cc24.tif?dl=0
https://www.dropbox.com/s/qnd4mvr7nleieob/RTcc24.tif?dl=0

As an aside, I had a few days access to a Leica S (007) and compared several dozen captures with the A7II and Canon 1DsII. The Leica and Canon colors were fairly close in LightRoom. The Sony quite yellow/green in comparison, but the use of some freely available profiles moved the Sony colors much closer to the other two. This, more than any amount of online discussion, has convinced me that it IS the profile that makes MOST of the difference between camera looks, optics and sensors being otherwise up to the task. (To be more precise, Leica S(007) with APO 120/2.5, Sony A7II with Minolta 100/2, Canon 1DsII with 24-105/4L , this was for color, not corner sharpness).

Yes I agree that profile is the main thing. There's too few photographers that have played around with profiles to discover this though, so the camera color myth lives on...

Anyway, if you are making a DCP you need to export linearly without white balance so you get a green-tinted dark file. RT can do this, there's a little checkbox when you export "Apply white balance", make sure it's unchecked. If you make a DCP with white-balanced shot you will get a green tint I think.

If you make an ICC profile the white balance does not matter, however if you make a profile for Capture One you need to export from Capture One, and since it sometimes does funky pre-processing it's not guaranteed that a profile made using Capture One exported file will work in any other raw converter.

An easy capture one workflow is described here:
http://www.ludd.ltu.se/~torger/photography/camera-profiling.html#the_easy_way_c1

An easy DNG workflow is described here:
http://www.ludd.ltu.se/~torger/photography/camera-profiling.html#the_easy_way

If you export a file from RT without white balance and make a DNG profile according to the above workflow it should work, and if you export from Capture One to make a Capture One ICC you should follow the Capture One workflow. The Capture One tiff file you provided looks fine.

[AlterEgo]

0.9.13 build for Windows ( mingw = dcamprof.exe + libgomp_64-1.dll + both manual & tutorial / = copies of Torger's web pages / in 3 formats : IE archive .mht, Mozilla archive .maff and regular .pdf ) : https://app.box.com/DCamProf

[mgrayson]

Yes, the C1 profile is quite nice! I'll keep working on the DCP. Thanks again!

Matt

Edit: Success! Wonderful!

[Iliah]

>it IS the profile that makes MOST of the difference between camera looks

There are some exceptions, if we discuss overall "looks" and go a little beyond "most". When the CFA is designed for low noise on a noisy sensor, it is often that, being too transparent, it produces not enough separation in certain ranges of colours, mostly green and yellow. You can easily see it reversing the colour transform. If the raw values from a set of RGB samples have the difference within the photon shot noise limits, those colours will not be resolved. Such CFAs are responsible for mashy greens and plastic skin, and also for wrong looking reds on firefighters' engines or problematic shifts between blue and purple.

[Iliah]

>so quite possible that source of that SSF (from RIT) didn't make a precise work in that part

Don't take those too seriously, it is more of a proof of concept than lab-grade results. Usual academic practice is that you need to show first you are on the right track before nice things happen, grants granted, etc.

[Alexey.Danilchenko]

reading the other forum about building a monochromator setup it is clear that it is not that easy to measure SSF directly

Erm - it is actually quite easy. My first SSF from SLR/n were quite good and resulted in a good quality profile. Where it gets difficult is with the cameras that tend to heat up (older ones like Kodak ProBacks) and a faster exposures are needed. This is what I am working on to improve right now (and also to make it really easy to process results with Hamamatsu micro spectrometer).

In general I found it a lot easier than to get good profiles from CC targets.

[Iliah]

> My first SSF from SLR/n were quite good and resulted in a good quality profile

In regards to http://www.cis.rit.edu/~dxl5849/projects/camspec/ - single-port sphere resulting in wrong / unmatched measurement geometry between the camera and spectrometer makes things problematic. Also, the SSFs are given for the particular raw extraction / conversion process (not necessarily without it's own quirks) and may be applicable for the same conversion procedure but not for a different one. There are many other unknowns and omissions in the description of the "ground truth" lab work there.

One of the most important things is that the response needs to be measured with the data numbers in raw being much higher than all noise levels. Each time SSF looks spiky this is the first thing to check. Specific damage to results is in the areas when channel curves overlap.

[AreBee]

Iliah,

There are some exceptions, if we discuss overall "looks" and go a little beyond "most". When the CFA is designed for low noise on a noisy sensor...

With regard to the low noise level of current state of the art sensors in consumer cameras, are the "exceptions" to which you refer...exceptionally rare?

[torger]

I'm also interested in what you (Iliah) know about the CFA. I'm quite new to this and have mostly looked at fairly modern cameras of 36x24mm and MF size, and they all seem to have such good color separation that I find it hard to believe that "plasticky" skin and other color issues would be a hardware-related thing today.

However I've seen results from really old cameras, and mobile phones and other tiny sensor cameras where color separation indeed is a hardware-related limitation. Shooting at high ISO you get noise issues too of course.

I haven't really made any thorough experiments to make a solid verification but my current assumption is that two reasonably modern cameras with reasonably large sensor in good light can be profiled to look almost exactly the same, but when you then change light from the profiled light the colors will modulate differently so the further away from the calibration illuminant you are the more the cameras will differ. I suspect that the difference will not be that big even then though as CFAs often seem to be quite similar between brands/models these days.

[howardm]

v0.9.13 for Mac OSX

I've created a folder on box.com to hold the current and past Mac binaries so I dont have to constantly provide URL's.

Pls give it a spin and let me know how well it works...........

https://app.box.com/s/tcnv5km0sh997ueqez7d3zqcqlh456nu

[kirkt]

Thank you - so far, so good.

kirk

v0.9.13 for Mac OSX

I've created a folder on box.com to hold the current and past Mac binaries so I dont have to constantly provide URL's.

Pls give it a spin and let me know how well it works...........

https://app.box.com/s/tcnv5km0sh997ueqez7d3zqcqlh456nu

[Iliah]

Iliah,

With regard to the low noise level of current state of the art sensors in consumer cameras, are the "exceptions" to which you refer...exceptionally rare?

It is totally wrong to speak of "brand colours" (Canon colours, SONY colours,...), I always maintained that. However noisy sensors are not that extinct, not to mention lots of people still shoot with the older cameras.

[AlterEgo]

It is totally wrong to speak of "brand colours" (Canon colours, SONY colours,...)

_but_ even with CFA SSF/CMF designs you can see somewhat constant distinction between how they look between Canons and Nikons, no ?

[torger]

I've got a good test image with some deep saturated artificial blue lights which show some rendering artifacts also with the latest v0.9.13. So you know it's a known bug. It will probably take a while to fix, the deep blue range has proven to be especially difficult as camera+profile often push artificial blues into undefined space, so you can't just normally gamut compress or something as you have no defined color to work on.

[Iliah]

> two reasonably modern cameras with reasonably large sensor in good light can be profiled to look almost exactly the same

Two reasonably low noise cameras, yes; if they are designed without CFA shortcuts. Point is, when somebody is speaking of "colour", colour per se is very flexible for all the cameras, notwithstanding the old ones. It is known with scanners, too. But the richness of gradation is a different matter. One of the good tests is shooting plums, like this: http://img01.quesabesde.com/media/img/noti/0092/nikon_j5_dsc_0084.nef

The colour separation is measured close to full well. Even with a monochromator it is often necessary to take 2 shots with different exposure, so that the data for the weak channels is not dug in noise. For a target chart, it may be 3 to 6 shots, depending on the chart design. CC DC takes 5.

[Iliah]

_but_ even with CFA SSF/CMF designs you can see somewhat constant distinction between how they look between Canons and Nikons, no ?

Mostly, cameramakers change the CFA design very often, sometimes even when using essentially the same sensor but with a different AFE (analogue front end), or to improve over the previous. So, that is not the part of the "brand look". However, the calculation formulas in use are different between companies, but that does not translate into the look of the final photo. One can see those differences in approacesc only if analyzing raw. End result depends on the colour transform from raw, close to 100%.

[AlterEgo]

so that the data for the weak channels is not dug in noise.

so how close to saturation in raw (blue channel) you move for a example a "blue" patch under a tungsten light ? within 2 stops ? 3 stops ?

[torger]

I think the source of the blue problem is that to get dark enough blue on screen you need to subtract raw blue, that is the luminance column in the forward matrix gets a negative blue value.

Adobe doesn't seem to put negative values there, but instead let blue render much lighter than realistic, and in return get no negative colors in extreme ranges, ie a more robust profile. Phase One often do this as well, although I do see extreme blue clip to black sometimes if I remember correctly.

I shall investigate this matrix design method, that is avoid negative blue value in the luminance/Y column, it could be a problem solver. The question is if this can be done without sacrificing blue accuracy in the normal range... the LUT can in theory compensate, but perhaps not without too aggressive bends.

[Iliah]

so how close to saturation in raw (blue channel) you move for a example a "blue" patch under a tungsten light ? within 2 stops ? 3 stops ?

3 stops down from saturation is midtones, that would be the lowest I go.