DCamProf - a new camera profiling tool

[torger]

for that subject in such light do we really need realistic colors or rather no artefacts from improper LUT in that areas ?

Indeed, there's little point in realistic colors as it's difficult in such light anyway. It should be possible to make it better than DCamProf does it currently though, not sure if it's possible to combine with the best color in good light though. I need to drill down deeper into the problem to figure that out.

[torger]

I have now released version 0.10.5, you find it as usual here:

http://www.ludd.ltu.se/~torger/dcamprof.html

direct download here: http://www.ludd.ltu.se/~torger/files/dcamprof-0.10.5.tar.bz2

I've solved Markanini's disco light problem (http://forum.luminous-landscape.com/index.php?topic=100015.msg899463#msg899463) by improving the extreme value compression algorithm. Those that have no extreme value problems with their profiles don't need to regenerate them.

I've also included a reference file for the CC24 manufactured November 2014 and later, the data provided by AlterEgo in this post: http://forum.luminous-landscape.com/index.php?topic=100015.msg872313#msg872313

I also made the documentation web page a little bit friendlier for mobile devices.

The "extreme value compression" works like this:

The basis for the LUT is a linear matrix, but some of those values will be outside a valid gamut (negative Y etc), for cameras with high sensitive blue like many Sony cameras a significant range can be there. The LUT can't work well with invalid inputs, so there's a pre-compression done directly on the (white-balanced) raw RGB values. Saturation is reduced by using RGB-HSV desaturation until it fits in a valid gamut. To not hurt gradients the transition range is quite wide (it can be controlled using the new -k parameter to the make-profile command, but generally there's little reason to modify it) and the larger out-of-gamut range the camera has for a particular hue the larger transition range. RGB-HSV desaturation is chosen as it results in lighter colors, so instead of a more saturated color you get a lighter color and this way tonality is kept.

This pre-compression of out-of-gamut matrix values is applied before the normal gamut compression (if activated). It can be disabled completely by "-k 0", but that's a bad idea(tm), unless you're making a reproduction profile and want to be 100% sure that the compression doesn't disturb colorimetric accuracy within the valid gamut.

[howardm]

Macintosh build of version v0.10.5 available at the below indicated URL

[AlterEgo]

0.10.5 build for Windows ( mingw = dcamprof.exe + libgomp_64-1.dll + both manual & tutorial / = copies of Torger's web pages / in 3 formats : IE archive .mht, Mozilla archive .maff and regular .pdf ) : https://app.box.com/DCamProf

[Hening Bettermann]

Thanks to all the three of you (Torger, howardm and AlterEgo)! - AlterEgo, how do you save a web page as .pdf??

[Bart_van_der_Wolf]

Thanks to all the three of you (Torger, howardm and AlterEgo)!

+1

Cheers,
Bart

[jrp]

Compiles on the Mac with the following warning:

gcc -o observers.o -c -I. -Wall -std=c99 -g -O2 -D_XOPEN_SOURCE=600 -D__EXTENSIONS__   -DDCAMPROF_VERSION=\"0.10.5\" observers.c
observers.c:107:21: warning: unused variable 'cmf_2006_2_390_830_5'
      [-Wunused-const-variable]
static const double cmf_2006_2_390_830_5[][4] = {
                    ^
1 warning generated.

No idea whether that is significant or not.

[howardm]

Nope.  He's got -Wall which is 'complain about everything' and in this case, he doesn't actually use that (constant) variable. (how's that for an oxymoron?)

Also note that in my compile/build, it's completely 'static' meaning there are no runtime dynamic library dependencies which can bite you (or someone else you give the binary to) in the butt.  Just an FYI in case you were not aware of that.

[howardm]

Please note that my previous build of v0.10.5 had a small error in it.
The problem would show up if you tried to use glare matching.

I have rebuilt it and re-uploaded it. 

If you downloaded it before 10:50PM EST 5/7, please throw
it away and re-download it.

Sorry for the inconvenience. 

[torger]

No need to worry about that warning. I haven't look into the observer stuff for quite a while. In the beginning of the project I had an idea to use the "best" observer functions, that is those that gave the best matching results in experiments. The 1931 observer is not really the best. However the CIE standards organization only change their standards if something comes along and provides significant improvement, otherwise they keep the old standard, and that's why the 1931 color matching function have survived for 85 years.

I was thinking that I could use some of the experimental matching functions though, but there's a major problem -- all the other color spaces are defined with the 1931 observer, so if I just change observer those are thrown out of balance. So in order to use a different observer one have to make some sort of transform from the new observer XYZ to a matching XYZ in the 1931, and I haven't really found any good way to do that.

The color matching functions does not say what color we see, just that spectrum A will have the same color as spectrum B if the resulting XYZ is the same, that is the colors match, hence the name "color matching functions". If we just exchange the color matching functions in the test target spectrum matching but keep the rest the same we will be in error, as the resulting XYZ coordinates will be transformed eventually to an RGB value on your screen and that transform is tuned for 1931 observer.

Anyway I don't think it's a big issue for this type of application to use the old 1931, from what I've read the matching errors seems to be largest for narrow band lights like some types of LED lights, where you may have an error of like 10 DE, but this is also the type of light conditions that cameras will naturally be in high error. For more sane colors the matching should be good with the old standard, and there the cameras have a better chance to perform well too.

[torger]

RGB-HSV desaturation is chosen as it results in lighter colors, so instead of a more saturated color you get a lighter color and this way tonality is kept.

A further comment about this. I often "complain" about the RGB HSV and HSL color spaces for not being psychovisually accurate, so why am I using it myself in this case? The RGB spaces have some advantages, they are very good at maintaining smooth gradients, and you don't need a valid XYZ coordinate to work with it. The camera registers colors in an RGB space, and when you transform that to an psychovisually advanced space like CIECAM02 you sort of mess up the camera's gradients. This is generally fine when you're some distance from clipping, but when you want to make a smooth transition into clipping it's hard to do that if not doing it in concert with the camera's RGB channels. This is represented in the DCamProf's neutral tone reproduction operator which blends in RGB space stuff close to clipping.

However in this compression case the reason is different, here we're working in a range where we don't have any valid XYZ coordinate and thus we can't transform it further into CIECAM02 or any other advanced color space which all start with a valid XYZ coordinate. So what we do is that we compress the camera's "extreme" RGB value into a new RGB value that can be transformed into a valid XYZ coordinate. Not being able to work in any advanced color space falling back to a raw RGB space is what we have to do. In this case RGB-HSV desaturation proved to show a visually pleasing effect as it lightens the color simultaneously which makes it look like it's closer to "film clipping".

Remember Bart's red tulips I used in refining high saturation highlight rendering? Those are unaffected by this change, as their coordinates where still in valid XYZ range, however the compression have a much similar effect in look as on those tulips where I also chose an approach to lighten the colors near clipping to keep tonality.

This new compression is primarily for the usual suspect -- Sony's high sensitive blue range. There's probably more cameras with this high sensitivity, but in my tests there's always been Sony sensors that have triggered issues. Not all Sony sensors though, I recently had a look on Phase One's IQ3 100MP which is using Sony's new 645 full-frame sensor, and it doesn't have the same "over-sensitive" blue range, possibly through a custom CFA, or perhaps more likely simply by dampening the blue range somewhat with the sensor cover glass. I don't know why Sony choose to have such high sensitive blues, it's probably some high ISO thing, but say Canon doesn't have poor high ISO either and their blue response is still not as challenging. Or they just like to optimize for indoor low temperature lights, then the high sensitive blue is an advantage as low temperature lights have very little blue content.

[fdisilvestro]

how do you save a web page as .pdf??

I don't know how AlterEgo did, but you can do it directly with Adobe Acrobat DC: File->Create->PDF From Web Page

[howardm]

In the Mac world, we'd simpy tell it to print and then Save As PDF.  Isn't there something similar in the Windows print dialog box (ie. a PDF pseudo-printer) ?

[Bart_van_der_Wolf]

In the Mac world, we'd simpy tell it to print and then Save As PDF.  Isn't there something similar in the Windows print dialog box (ie. a PDF pseudo-printer) ?

Yes, but because Windows is not as intimate as Apple / Adobe are, it requires a driver / program to do that. There are some free ones. and others are paid versions.

Cheers,
Bart

[howardm]

Everytime I have to delve into that world, I know my choice of OS was correct.

[AlterEgo]

Isn't there something similar in the Windows print dialog box (ie. a PDF pseudo-printer) ?

W10 has "Microsoft Print to PDF" 'printer', but I always use Adobe Acrobat.

[Tim Lookingbill]

In fact I can save this single LuLa discussion thread page in PDF in OS 10.6.8 and it will embed the original URL so if years later I want to hunt down the entire 60 page thread I won't have a problem finding it on the web.

I have a lot of forum discussion pages not just here but from other website forums in my Documents folder. Very convenient over hunting down bookmarks in my browser or doing a Google search online. And if by chance the site page is removed by administrators or just cleared from servers to make more room, I still have a copy. It's better than a screengrab.

[Hening Bettermann]

Thanks to all who responded to my question on how to save a web space as .pdf!

Since I, too, am on a Mac, the easiest way seems to be to open the print dialogue and then Save As pdf. It also records the URL. And the disk space is only 40% of the web archive. What annoys me about Apples web archive is that it pretends to save, but if you re-open the page after a while, the images are gone, so they are just cached. But Apple doesn't tell you that.

Well, I will not digress into off-topic any further. Thanks again!

[sebbe]

These are good changes. Thanks for this update!
Seb

[fdisilvestro]

In the Mac world, we'd simpy tell it to print and then Save As PDF.  Isn't there something similar in the Windows print dialog box (ie. a PDF pseudo-printer) ?

In Windows is the same (and no need for additional drivers, at least since windows 7), Acrobat has just more options to customize the output. You can even generate a pdf file with an entire multilevel website at once