-
As Argyll can't make DCPs and the commercial software seemed limited and have issues with smoothness I decided to make my own software, and I have now just released a first version (it's free and open-source):
http://www.ludd.ltu.se/~torger/dcamprof.html
It's command line only and quite technical, but if you can use Argyll you can use my DCamProf.
Hope you enjoy it.
Now I need some rest and start living again. I've been working nonstop on my spare time for almost two months with this...
-
thank you
-
There are a few unique features in the software, but perhaps the biggest thing is that you can make profiles directly from the camera's spectral sensitivity functions (=color filter responses), just like many manufacturers do.
You can make it the traditional way too from photos of test targets of course. The ability to experiment with SSFs is a great learning experience though if you want to get a feel of how camera color work. I have links to some sources of SSFs that have been made public through academic work, unfortunately it's hard to come by for very recent cameras.
There's the color spectral data too. I currently have builtin spectral data for cc24, munsell glossy set and spectra from nordic nature. I'd like to have a skin tone database too and more. If any of you know about some useful public spectral databases I'd like to know so I can consider them for inclusion. There are a few commercial databases out there, such as directly from CIE, but those cannot be redistributed in free software...
-
Great work.
-
Thanks. I was inspired by your work Trantor which showed me the power of using SSFs directly.
-
Wow Torger - a lot to think about and learn. Thank you for the effort!
kirk
-
Now I need some rest and start living again.
can you (after some rest) add an option to imitate matrix/trc model (like in icc containers) where you have a dcp basically with CM/FM for a color transform & WB operations for ACR/LR and HueSatMap (so it is before exposure adjustments done by a user in ACR/LR UI) that is just imitating (and nothing else) what in "icc" case TRCs (in a simplest case gamma != 1) 'd do per each RGB channel on WB/demosaicked data before matrix conversion into PCS (cieXYZ/D50)... something like "-g <gamma>" option for a simple scenario (more complex might pickup curve(s) data from some text/xml/etc file)
-
As Argyll can't make DCPs and the commercial software seemed limited and have issues with smoothness I decided to make my own software, and I have now just released a first version (it's free and open-source):
http://www.ludd.ltu.se/~torger/dcamprof.html
It's command line only and quite technical, but if you can use Argyll you can use my DCamProf.
Hope you enjoy it.
Now I need some rest and start living again. I've been working nonstop on my spare time for almost two months with this...
Hi Anders, do get some rest. I know how much it takes do put something like this together. Huge amounts of research, even before the programming effort can begin, and once you are encoding, you discover all sorts of new things that need research.
The effort and making the result available is much appreciated. Unfortunately for this version, I'm an ICC Camera profile user (Capture One), so DCPs are not my main interest. But if you get around to adding ICC profile generation, I'll surely try to compile a version for Windows if I can find some time myself.
Cheers,
Bart
-
But if you get around to adding ICC profile generation
wasn't there somewhere dcp2icc free conversion tool
-
I'd like to have a skin tone database too
so everybody here = get naked with your i1s (or whatever) in hands and donate :D !
-
There are skin databases out there: http://files.cie.co.at/699_Report%20for%20CIE%20R1-56.pdf haven't had times to research yet into if there's someone that can be used in open-source software.
I may look into making ICC profile generator for Capture One (and others) later. The original intention was to make a format-agnostic profile maker, and although DCamProf does store a "color matrix" and "forward matrix" (DCP names) it's generic concepts and it stores a generic format so ICC making should be possible. I need this to settle for a while first, get some early adopters to test etc. There's quite much new stuff to learn for me when it comes to ICC about how the LUTs work (it will probably be a camera RGB directly to Lab D50), and they're a bit tricky to work with as they're integer-based so you need to think it through to not break precision.
AlterEgo, I'm not sure if I follow your feature request. DCamProf can generate profiles with our without HueSatMap. I don't understand why you would want to have some sort of gamma feature? DCPs work in linear floating point space. When it comes to ICC I need to think about gamma of course as some raw converters (like Capture One) feeds the ICC profile with gamma-processed rather than linear image, so when generating ICCs there will have to be some options regarding those aspects.
-
AlterEgo, I'm not sure if I follow your feature request. DCamProf can generate profiles with our without HueSatMap. I don't understand why you would want to have some sort of gamma feature? DCPs work in linear floating point space. When it comes to ICC I need to think about gamma of course as some raw converters (like Capture One) feeds the ICC profile with gamma-processed rather than linear image, so when generating ICCs there will have to be some options regarding those aspects.
just experimenting between converters ! you can put a tone curve (probably a wrong wording for that purpose) in dcp profile that is applied after exposure adjustment operations, but if you want (for example) to do this before then HueSatMap trick is the only way, no ?
-
just experimenting between converters ! you can put a tone curve (probably a wrong wording for that purpose) in dcp profile that is applied after exposure adjustment operations, but if you want (for example) to do this before then HueSatMap trick is the only way, no ?
Ah, ok. Yes if you want to apply a curve before exposure adjustments you need to incorporate that in HueSatMap.
-
Ah, ok. Yes if you want to apply a curve before exposure adjustments you need to incorporate that in HueSatMap.
yes, so if you will find some strength after the deserved rest for the experimentation sake please
-
I have stepped through the basic workflow outlined in your web page "Basic workflow for making a profile using a test target".
I used a shot of the 24 patch section of my CCPassport, cropped it, checked the crop with the diagnostic image from scanin, etc. The output from scanin looks good, the whole process runs great, etc. This is from my own build with -fopenmp removed from the makefile. I'm on a Mac, running 10.10.3. I made the scanin input with the ColorChecker spectral data file that you provide with the downloaded archive.
Fair enough. I made the profile with the appropriate camera specific identifier I copied verbatim from the Adobe Standard profile, using dcp2json.
When I place the newly created DCP in the user level camera profiles folder, it down not show up in LR/ACR. When I use Iridient Developer to specify the newly created DCP, it gives me a little more info:
"Unable to load a valid 3 color DNG camera profile from the selected file."
I decompiled the newly made DCP using dcpTool to see if there was any difference between the json and the decompiled XMP version of the file. There does not appear to be. I compared it to the structure of the Adobe Standard profile for the Canon 5DIII and all seems correct.
I'm not sure what is preventing the profile from being used. Any thoughts?
Thanks,
kirk
-
Thanks for testing.
I've done most of my testing in RawTherapee, just generated a few profiles including a dual-illuminant one. It was a while since I tested in Lightroom though and lots of code has been written since. I shall make a new test in Lightroom and see so there's not some bug there preventing them from working. From the error message you're getting it does sound like a bug.
The only mistake one is supposed to be able to make is to name the camera incorrectly, so you can double-check that the UniqueCameraModel is *exactly* the same as in a profile that works with your camera.
-
Any thoughts?
may be you can just attach the dcp file or its dump from dcptool - makes it easier for somebody to notice any errors (if any)
-
Here is a log of the process, from Terminal output:
KT-Laptop-3:Debug kirkt$ dcraw -v -r 1 1 1 1 -o 0 -H 0 -T -6 -W -g 1 1 /Users/kirkt/Desktop/dcamprof/Debug/_MG_0044.CR2
Loading Canon EOS 5D Mark III image from /Users/kirkt/Desktop/dcamprof/Debug/_MG_0044.CR2 ...
Scaling with darkness 2047, saturation 15488, and
multipliers 1.000000 1.000000 1.000000 1.000000
AHD interpolation...
Building histograms...
Writing data to /Users/kirkt/Desktop/dcamprof/Debug/_MG_0044.tiff ...
KT-Laptop-3:Debug kirkt$ scanin -v -dipn target.tif ColorChecker.cht cc24_ref.cie
TIFFFetchNormalTag: Warning, ASCII value for tag "ImageDescription" contains null byte in value; value incorrectly truncated during reading due to implementation limitations.
TIFFFetchNormalTag: Warning, ASCII value for tag "Make" contains null byte in value; value incorrectly truncated during reading due to implementation limitations.
TIFFFetchNormalTag: Warning, ASCII value for tag "Model" contains null byte in value; value incorrectly truncated during reading due to implementation limitations.
TIFFFetchNormalTag: Warning, Incompatible type for "RichTIFFIPTC"; tag ignored.
Input file 'target.tif': w=2356, h=1562, d = 3, bpp = 16
Data input file 'cc24_ref.cie'
Data output file 'target.ti3'
Chart reference file 'ColorChecker.cht'
Creating diagnostic tiff file 'diag.tif'
About to allocate scanrd_ object
Verbosity = 2, flags = 0x42a01
About to read input tiff file and discover groups
adivval = 1.000000
About to calculate edge lines
379 useful edges out of 482
About to calculate rotation
Mean angle = -0.220790
Standard deviation = 1.164465
Robust mean angle = -0.189701 from 356 lines
About to calculate feature information
About to read reference feature information
Read of chart reference file succeeded
About to match features
Checking xx
Checking yy
Checking xy
Checking yx
Checking xix
Checking yiy
Checking xiy
Checking yix
Axis matches for each possible orientation:
0: xx = 0.320776, yy = 0.306004, xx.sc = 0.134631, yy.sc = 0.135603
90: xiy = 0.225432, yx = 0.351580, xiy.sc = 0.389820, yx.sc = 0.088742
180: xix = 0.301947, yiy = 0.347327, xix.sc = 0.134340, yiy.sc = 0.134642
270: xy = 0.224873, yix = 0.351576, xy.sc = 0.257560, yix.sc = 0.088742
r0 = 0.440145, r90 = 0.095076, r180 = 0.459195, r270 = 0.143794
There are 2 candidate rotations:
cc = 0.440145, irot = -0.189701, xoff = -49.567580, yoff = -74.423086, xscale = 7.427721, yscale = 7.374475
cc = 0.459195, irot = 179.810299, xoff = -2416.435540, yoff = -1602.952441, xscale = 7.443805, yscale = 7.427127
About to compute match transform for rotation -0.189701 deg.
About to setup value scanrdg boxes
About to read raster values
About to compute expected value correlation
About to compute match transform for rotation 179.810299 deg.
About to setup value scanrdg boxes
About to read raster values
About to compute expected value correlation
Expected value distance values are:
0, rot -0.189701: 2410.510929
1, rot 179.810299: 3842.704025
Chosen rotation -0.189701 deg. as best
About to compute final match transform
Improve match
About to setup value scanrdg boxes
About to read raster values
About to write diag file
Writing output values to file 'target.ti3'
KT-Laptop-3:Debug kirkt$ dcamprof make-profile target.ti3 profile.json
Reading target and generating values for the calibration illuminant D50...
Re-generating target reference XYZ values for illuminant D50...
Making camera profile...
Finding a camera raw RGB to CIE XYZ matrix for illuminant D50...
Whitest patch in target differs DE 1.18 from calibration illuminant,
close enough to calculate whitepoint preservation.
Inverting to get ColorMatrix:
{
"ColorMatrix1": [
[ 0.729155, -0.155578, -0.066452 ],
[ -0.482378, 1.364639, 0.172933 ],
[ -0.111882, 0.237018, 0.685179 ]
]
}
Matrix patch match average DE 1.38, DE LCh 0.61 0.63 0.97
mean DE 1.24, DE LCh 0.50 0.51 0.69
p90 DE 2.32, DE LCh 1.28 1.15 1.90
max DE 4.06, DE LCh 1.92 1.90 3.94
ColorMatrix optimal white balance for target: 0.478142,1,0.660402
Using previously calculated RGB to XYZ D50 matrix.
Applying white-balance to get ForwardMatrix:
{
"ForwardMatrix1": [
[ 0.738497, 0.156947, 0.072765 ],
[ 0.257033, 0.850257, -0.107290 ],
[ 0.031675, -0.268494, 1.049609 ]
]
}
Matrix patch match average DE 1.58, DE LCh 0.96 0.59 0.98
mean DE 1.48, DE LCh 0.91 0.60 0.69
p90 DE 2.58, DE LCh 1.83 1.05 1.91
max DE 4.23, DE LCh 2.40 1.78 3.95
ForwardMatrix optimal white balance for target: 0.478142,1,0.660402
Making 2.5D chromaticity-addressed lookup table for XYZ correction...
25.00% of the patches was put in a chromaticity group due to nearby neighbor.
Largest chromaticity group contains 6 patches. Patch count reduced from
24 to 18. Note that patch matching cannot reach 100% when chromaticity
groups are formed, as the LUT matches the average within a group.
Native LUT patch match average DE 0.37, DE LCh 0.14 0.15 0.23
mean DE 0.00, DE LCh 0.00 0.00 0.00
p90 DE 1.21, DE LCh 0.48 0.60 0.81
max DE 1.89, DE LCh 1.19 0.95 1.80
5 worst patches for Overall DE:
0 RGB 0.077 0.106 0.054 => XYZ 0.127 0.111 0.055 (0.130 0.114 0.054) DE 1.89 DE LCh 0.57 0.04 1.80 (dark brown)
18 RGB 0.484 1.000 0.652 => XYZ 0.965 1.000 0.779 (0.948 0.978 0.773) DE 1.40 DE LCh 0.48 0.95 0.90 (white)
23 RGB 0.019 0.041 0.027 => XYZ 0.034 0.035 0.029 (0.039 0.040 0.033) DE 1.21 DE LCh 1.19 0.23 0.05 (gray 20%)
20 RGB 0.194 0.408 0.267 => XYZ 0.380 0.394 0.320 (0.382 0.398 0.318) DE 1.02 DE LCh 0.19 0.89 0.46 (gray 70%)
22 RGB 0.048 0.103 0.068 => XYZ 0.094 0.098 0.081 (0.096 0.101 0.082) DE 1.02 DE LCh 0.42 0.60 0.71 (gray 40%)
5 worst patches for Lightness DE:
23 RGB 0.019 0.041 0.027 => XYZ 0.034 0.035 0.029 (0.039 0.040 0.033) DE 1.21 DE LCh 1.19 0.23 0.05 (gray 20%)
0 RGB 0.077 0.106 0.054 => XYZ 0.127 0.111 0.055 (0.130 0.114 0.054) DE 1.89 DE LCh 0.57 0.04 1.80 (dark brown)
18 RGB 0.484 1.000 0.652 => XYZ 0.965 1.000 0.779 (0.948 0.978 0.773) DE 1.40 DE LCh 0.48 0.95 0.90 (white)
22 RGB 0.048 0.103 0.068 => XYZ 0.094 0.098 0.081 (0.096 0.101 0.082) DE 1.02 DE LCh 0.42 0.60 0.71 (gray 40%)
19 RGB 0.319 0.671 0.440 => XYZ 0.622 0.645 0.521 (0.628 0.654 0.525) DE 0.64 DE LCh 0.28 0.58 0.03 (gray 80%)
5 worst patches for Chroma DE:
18 RGB 0.484 1.000 0.652 => XYZ 0.965 1.000 0.779 (0.948 0.978 0.773) DE 1.40 DE LCh 0.48 0.95 0.90 (white)
20 RGB 0.194 0.408 0.267 => XYZ 0.380 0.394 0.320 (0.382 0.398 0.318) DE 1.02 DE LCh 0.19 0.89 0.46 (gray 70%)
22 RGB 0.048 0.103 0.068 => XYZ 0.094 0.098 0.081 (0.096 0.101 0.082) DE 1.02 DE LCh 0.42 0.60 0.71 (gray 40%)
19 RGB 0.319 0.671 0.440 => XYZ 0.622 0.645 0.521 (0.628 0.654 0.525) DE 0.64 DE LCh 0.28 0.58 0.03 (gray 80%)
21 RGB 0.102 0.215 0.141 => XYZ 0.202 0.210 0.171 (0.202 0.210 0.168) DE 0.78 DE LCh 0.06 0.32 0.71 (gray 50%)
5 worst patches for Hue DE:
0 RGB 0.077 0.106 0.054 => XYZ 0.127 0.111 0.055 (0.130 0.114 0.054) DE 1.89 DE LCh 0.57 0.04 1.80 (dark brown)
18 RGB 0.484 1.000 0.652 => XYZ 0.965 1.000 0.779 (0.948 0.978 0.773) DE 1.40 DE LCh 0.48 0.95 0.90 (white)
1 RGB 0.253 0.363 0.201 => XYZ 0.424 0.382 0.204 (0.422 0.378 0.206) DE 0.84 DE LCh 0.22 0.03 0.81 (red)
21 RGB 0.102 0.215 0.141 => XYZ 0.202 0.210 0.171 (0.202 0.210 0.168) DE 0.78 DE LCh 0.06 0.32 0.71 (gray 50%)
22 RGB 0.048 0.103 0.068 => XYZ 0.094 0.098 0.081 (0.096 0.101 0.082) DE 1.02 DE LCh 0.42 0.60 0.71 (gray 40%)
5 best patches for Overall DE:
16 RGB 0.187 0.186 0.183 => XYZ 0.325 0.223 0.259 (0.325 0.223 0.259) DE 0.00 DE LCh 0.00 0.00 0.00 (purple-red)
3 RGB 0.071 0.154 0.065 => XYZ 0.123 0.141 0.056 (0.123 0.141 0.056) DE 0.00 DE LCh 0.00 0.00 0.00 (yellow-green)
2 RGB 0.089 0.249 0.219 => XYZ 0.189 0.211 0.284 (0.189 0.211 0.284) DE 0.00 DE LCh 0.00 0.00 0.00 (purple-blue)
14 RGB 0.152 0.093 0.045 => XYZ 0.224 0.134 0.041 (0.224 0.134 0.041) DE 0.00 DE LCh 0.00 0.00 0.00 (red)
9 RGB 0.048 0.078 0.084 => XYZ 0.090 0.073 0.117 (0.090 0.073 0.117) DE 0.00 DE LCh 0.00 0.00 0.00 (dark purple)
Writing output to "profile.json"...
Complete!
dcamprof make-dcp -c "Canon EOS 5D Mark III" profile.json testprofile.dcp
In the next post, I will post a Dropbox link with a ZIP archive of all of the files I used in this process.
kirk
-
I'm about to do some Lightroom tests myself... I just have some issues with my windows virtual box here so it can be an hour or two...
-
Here is a link to the archive of files I used in this example:
https://www.dropbox.com/s/xq5w8whoi8sq1av/dcamprofDebugArchive.zip?dl=0
it is uploading now (1400 hr eastern daylight time) so give it a few minutes if the download fails.
The archive includes the source raw, the dcraw conversion, a crop of the conversion, the scanin target chart in file, etc.
thanks in advance!
kirk
-
Here is a link to the archive of files I used in this example:
https://www.dropbox.com/s/xq5w8whoi8sq1av/dcamprofDebugArchive.zip?dl=0
it is uploading now (1400 hr eastern daylight time) so give it a few minutes if the download fails.
The archive includes the source raw, the dcraw conversion, a crop of the conversion, the scanin target chart in file, etc.
thanks in advance!
kirk
something is wrong with CM/FM inside DCP... if you replace a pair of CM/FM with the one from Adobe OEM - then ACR see the profile and can use it... so the way CM/FM were calculated put them against something that ACR code expects (it does some checks of course, you can't just feed any matrices)
-
Seems like I've messed up with something, I'll report back when I have a fix.
-
Seems like I've messed up with something, I'll report back when I have a fix.
check how you do FM - if you leave everything but use FM from Adobe OEM then ACR code can accept the profile... replace FM with yours and ACR does not accept it
-
check how you do FM - if you leave everything but use FM from Adobe OEM then ACR code can accept the profile... replace FM with yours and ACR does not accept it
Thanks for the help, I'll have a look.
-
Thanks for the help, I'll have a look.
and CM seems to operate properly in terms of how ACR works with WB - visual impression, so CM is calculated OK I guess...
-
Got it working now, I'll soon make a new release. Just need to really make sure what the error was.
Edit: there where two errors:
1) Lightroom cannot handle FM with too high precision in the rational numbers, they use 1/10000, I have 1/1000000000 but then it goes haywire
2) Lightroom cannot handle other observer than 1931_2, DCamProf uses the more realistic 2006_10. By some (stupid) reason Lightroom makes a check that FM x 1,1,1 = D50, and D50 varies slightly depending on observer, and thus if you have not used 1931_2 it breaks.
I'll add a workaround for this and publish in an hour or so, but I can't say I think Lightroom does it the right way. There's no reason to "sanity-check" the D50 whitepoint this way, and why they can handle high precision rational numbers is a bit strange too.
-
Got it working now, I'll soon make a new release. Just need to really make sure what the error was.
Edit: there where two errors:
1) Lightroom cannot handle FM with too high precision in the rational numbers, they use 1/10000, I have 1/1000000000 but then it goes haywire
2) Lightroom cannot handle other observer than 1931_2, DCamProf uses the more realistic 2006_10. By some (stupid) reason Lightroom makes a check that FM x 1,1,1 = D50, and D50 varies slightly depending on observer, and thus if you have not used 1931_2 it breaks.
I'll add a workaround for this and publish in an hour or so, but I can't say I think Lightroom does it the right way. There's no reason to "sanity-check" the D50 whitepoint this way, and why they can handle high precision rational numbers is a bit strange too.
the first issue is kind of funny... too much precision :-)
-
The observer thing is a tricky one... in a way I understand why they check it, as their XYZ to prophoto matrix is hardcoded and expect 1931_2 D50 whitepoint. The easy way out is just that I change the default from 2006_10 to 1931_2, but I think that is worse design, I'd like it to be possible to use more realistic/accurate observers.
Maybe I could just do some whitepoint remapping trick... I need to think.
-
10-degrees standard observer is not more accurate, it's a just different. Simply do not use it.
-
10-degrees standard observer is not more accurate, it's a just different. Simply do not use it.
I'm quite sure the 2006 versions, http://www.cvrl.org/cmfs.htm, are more accurate. When it comes to 2 vs 10 degree I agree, it's more a matter of taste, but AFAIK the 2006_2 is more accurate than 1931_2 and 2006_10 is more accurate than 1964_10. The reason those new versions are made is mainly to improve accuracy.
If it matters is another aspect though, probably not I guess.
I think I will do an auto-wp-remapping in the make-dcp step and maybe change default to 1931_2.
-
Maybe I could just do some whitepoint remapping trick... I need to think.
if you are so set on "10 degrees" then just supply a command line parameter for an end user to select and provide some explanantion in a manual and then may be it makes sense to default (if nothing is on the command line) to "2 degrees" - I 'd assume that all popular (for masses) software expects that and not "10 degrees" based on what I can read
-
if you are so set on "10 degrees" then just supply a command line parameter for an end user to select and provide some explanantion in a manual and then may be it makes sense to default (if nothing is on the command line) to "2 degrees" - I 'd assume that all popular (for masses) software expects that and not "10 degrees" based on what I can read
Command line parameter is already there, you can do "-o 1931_2", so if it weren't for the 1/10000000 thing it would have worked with the current version. Well, no more talk, I need to code :-)
-
I have now made a new release, and this time actually tested it on Lightroom :-)
http://www.ludd.ltu.se/~torger/files/dcamprof-0.5.1.tar.bz2
http://www.ludd.ltu.se/~torger/dcamprof.html
I added an auto-remapping of whitepoint in the make-dcp tool so regardless of observer used during profile creation it will be remapped to match what Lightroom and probably many other DCP-compatible converters are expecting.
I also changed the default observer to 1931_2 so the remapping is not really needed in the default case. Although I like the 2006_10 (or 2006_2) better I don't have the knowledge now to know if it's better to make profile with say 2006_2 and then remap the white, or if the supposedly slight improvements are nullified due to remapping and hard-coded prophoto matrices later in the pipeline which I can't control. So now by using 1931_2 it's done the way "everyone else" does it, which I suppose is a reasonable default anyway.
Kirk - now it should work for you too. Just compile the new version and run the make-profile and make-dcp steps again.
-
Prophoto is defined against 1931_2, and as DNG works in that space it was natural that the pipeline was hardwired to that. sRGB, AdobeRGB etc are all related to 1931_2. I missed that aspect. Learning something new every day.
This means that it will be tricky to make use of alternate observers in a proper way, seems to me to do it right one would have to make a conversion LUT between the different observers too, quite messy. So using the 1931_2 as default for now is certainly the right choice.
-
Done some reading on observers and did some further observer tests this morning, released a very minor update v0.5.2 which tunes the observer remapping a bit. This is however only relevant if experimenting with a different observer than the nowadays default 1931_2.
There's no great method to transform say 2006_10 XYZ to 1931_2 XYZ (found this paper while searching: http://www.rit.edu/cos/colorscience/re_WptNormalization.php), but as the difference is quite small I've tested with the Bradford transform and it seems to make a good result in practice, so good that any other inherent errors in profiling should be much larger.
I made some examples between 2006_10/2006_2 and 1931_2. The 2 and 10 degree observers render near zero visible difference, while it's quite large difference between 2006 and 1931. For the particular camera I tested this morning (Pentax K10D) I preferred the 1931_2 colors, they looked more on the spot to my eyes, so having to change the default to 1931_2 doesn't feel that bad :)
-
Thanks! I'll try recompiling again and report back.
Kirk
-
> By some (stupid) reason Lightroom makes a check that FM x 1,1,1 = D50, and D50 varies slightly depending on observer, and thus if you have not used 1931_2 it breaks.
That is not that stupid. FM is constructed in a way to map camera white balanced white point to XYZ D50 (that is according to DNG spec) and that implies this mapping exactly (white balanced camera space will have 1,1,1 as white point). FM is actually a very specific case of ICC profile matrix in this case.
-
Yes I agree after some analysis that it's not that stupid, although unnecessary. The pipeline does not break if there's a small white balance shift, and the reference code doesn't do this type of check.
At some point I'm probably going to again review if one can do something useful with the more modern observers, for now it seems like 1931_2 is the way to go to get best results.
-
It is necessary at least to calculate those matrixes. The profiles (and correction matrices) are build on the that assumption. Nobody prevents to use the matrices on imperfectly white balanced camera space (this is what creative WB is for - I for one never completely white balance tungsten lighting and prefer warmer colours).
What is limiting in DNG is that it has to map to D50 - unlike ICC which can have any white point.
Great tool - thanks for sharing it. I was writing something custom for my own purpouses but will attempt to make use of DCamProf now. Have not managed to compile it on Windows yet - it does not seem to accept gcc short syntax for initialising structs.
-
I just tried compiling with cygwin on windows, it worked but you have to remove long double stuff in blackbody code which probably is overkill anyway. I use c99 syntax, Microsoft compilers have been sloooow adopting that, but it's only a 16 year old standard :)
-
I was trying with Visual C++ (not the most recent) - does not work. Will have go with mingw.
Had a good look through the sources and your code does seem to fit very well to C++ (it is already kind of structured that way with data structures and supporting functions for them).
-
C++ with its flexibility suits pretty much everything, it's a great language. I am first and foremost a C systems programmer on Unix though so when I want fast results I use the tools I know and enjoy best. In my professional life it sometimes feels that more time is spent learning new languages and tools of the day than solving actual problems. When I do stuff for myself I rarely do it to become better at some language, then I just use the tools I like and know best and get on to problem solving.
That Microsoft fails to support the C99 standard, which is a known issue, is pretty much a scandal but it's not uncommon for the big software companies to ignore open standards and rather make up their own. Dot net and C sharp is their thing nowadays. Mac OS X doesn't support OpenMP, another well-established open standard, but it looks a bit better there though as they ride on open source and the clang compiler will be including it "soon".
Let us know if it works out compiling with mingw64. I'll remove the long double code to next patch release since it's not needed.
-
All I was saying is that your code is very well structured (so converting it to C++ for someone who wants to do so would be an easy task ;))
Mac OS clang support for OpenMP (lacking of it) is diappointing. I needed it for some of the things I did so the way I solved it was to build GCC version with it and create Xcode templates to use it from Xcode directly. It does not work with Mac OS native libraries but for me where UI was built with QT (from ground up) it worked quite nicely. I do with they start adding it though with clang to avoid all that.
-
Let us know if it works out compiling with mingw64. I'll remove the long double code to next patch release since it's not needed.
Torger, can you share the windows binary that you compiled, if you don't mind ? thank you
-
Torger, can you share the windows binary that you compiled, if you don't mind ? thank you
I don't think it works without the Cygwin environment installed, a mingw build can be made standalone statically linked I suppose. The advantage of having cygwin is that you get a command line terminal that actually is not painful to work with, unlike the cmd.exe in windows.
With cygwin you just install from www.cygwin.com, install all devel-packages then just open the command line and
1. get: $ wget http://www.ludd.ltu.se/~torger/files/dcamprof-0.5.3.tar.bz2
2. unpack: $ tar xjf dcamprof-0.5.3.tar.bz2
3. change dir: $ cd dcamprof-0.5.3
4. compile: $ make
5. run: $ ./dcamprof.exe
To shoot targets you'll need Argyll too.
I just released v0.5.3, with some minor additions. Added a txt2ti3 conversion command so you can read raw text spectral data which spectral databases are commonly distributed as. I also added a link to Lippmann2000 spectral database, which unfortunately only may be used for research purposes, but they do have lots of human spectral data.
-
I don't think it works without the Cygwin environment installed, a mingw build can be made standalone statically linked I suppose.
I last did anything like this (building the executable that way) with C/C++ code several years ago (myself - I am not a software developer by trade), so I was hoping to use a shortcut :D
-
I don't think it works without the Cygwin environment installed
It will - all it will need is cygwin.dll bundled with executable.
Here is a version compiled for 64 bit Windows (https://drive.google.com/file/d/0Bw2ZohnbXtyAQmFPVjBSXzFNODg/view?usp=sharing) with all needed cygwin dependencies to execute standalone.
-
Thanks Alexey for the info and the build!
-
I built v 0.5.3 on my mac (Mavericks) without openMP. It now generates DCPs that are visible and recognized by ACR and Iridient Developer. I will also build on my work machine (Yosemite).
Now to test.
kirk
-
Thanks Alexey for the info and the build!
No problem. It could potentially be compiled with mingw but some posix stuff needs customising (I can probably fix it with a few ifdefs - if it won't be too ugly, I'll send the changes to you if that's ok).
-
No problem. It could potentially be compiled with mingw but some posix stuff needs customising (I can probably fix it with a few ifdefs - if it won't be too ugly, I'll send the changes to you if that's ok).
Please do
-
No problem. It could potentially be compiled with mingw but some posix stuff needs customising (I can probably fix it with a few ifdefs - if it won't be too ugly, I'll send the changes to you if that's ok).
ok, years w/o programming taking a tall... I finally managed to find a stupid elementary school level error when using mingw... of course fopen shall be used with wb/rb flags on windows platform for binary i/o (profio.c file), damn... we were taught this like 25 years ago and I forgot :-(
so the things necessary to build DCamProf with mingw are :
1) replace mkdir(dir, 0777) with mkdir(dir) in dcamprof.c
2) add #include <pthread.h> in argyllio.c to get localtime_r and asctime_r
3) replace #include <alloca.h> with #include <malloc.h> in tpc.c
4) and damn it ! in profio.c use fopen with wb and rb flags to read/write .dcp (binary) files
that's it
Administrator@CLEVO /z/dcamprof-0.5.3
$ make
gcc -o dcamprof.o -c -I. -Wall -std=c99 -g -O2 -D_XOPEN_SOURCE=600 -D__EXTENSIONS__ -fopenmp -DDCAMPROF_VERSION=\"0.5.3\" dcamprof.c
gcc -o nmsimplex.o -c -I. -Wall -std=c99 -g -O2 -D_XOPEN_SOURCE=600 -D__EXTENSIONS__ -fopenmp -DDCAMPROF_VERSION=\"0.5.3\" nmsimplex.c
gcc -o profio.o -c -I. -Wall -std=c99 -g -O2 -D_XOPEN_SOURCE=600 -D__EXTENSIONS__ -fopenmp -DDCAMPROF_VERSION=\"0.5.3\" profio.c
gcc -o argyllio.o -c -I. -Wall -std=c99 -g -O2 -D_XOPEN_SOURCE=600 -D__EXTENSIONS__ -fopenmp -DDCAMPROF_VERSION=\"0.5.3\" argyllio.c
gcc -o tps.o -c -I. -Wall -std=c99 -g -O2 -D_XOPEN_SOURCE=600 -D__EXTENSIONS__ -fopenmp -DDCAMPROF_VERSION=\"0.5.3\" tps.c
gcc -o colmath.o -c -I. -Wall -std=c99 -g -O2 -D_XOPEN_SOURCE=600 -D__EXTENSIONS__ -fopenmp -DDCAMPROF_VERSION=\"0.5.3\" colmath.c
gcc -o matopt.o -c -I. -Wall -std=c99 -g -O2 -D_XOPEN_SOURCE=600 -D__EXTENSIONS__ -fopenmp -DDCAMPROF_VERSION=\"0.5.3\" matopt.c
matopt.c: In function 'matopt_find_matrix':
matopt.c:270:12: warning: variable 'de' set but not used [-Wunused-but-set-variable]
double de;
^
gcc -o lut.o -c -I. -Wall -std=c99 -g -O2 -D_XOPEN_SOURCE=600 -D__EXTENSIONS__ -fopenmp -DDCAMPROF_VERSION=\"0.5.3\" lut.c
gcc -o dngref.o -c -I. -Wall -std=c99 -g -O2 -D_XOPEN_SOURCE=600 -D__EXTENSIONS__ -fopenmp -DDCAMPROF_VERSION=\"0.5.3\" dngref.c
dngref.c:470:1: warning: 'dngref_XYtoXYZ' defined but not used [-Wunused-function]
dngref_XYtoXYZ(const double XY[2])
^
gcc -o spectrum.o -c -I. -Wall -std=c99 -g -O2 -D_XOPEN_SOURCE=600 -D__EXTENSIONS__ -fopenmp -DDCAMPROF_VERSION=\"0.5.3\" spectrum.c
gcc -o jsonio.o -c -I. -Wall -std=c99 -g -O2 -D_XOPEN_SOURCE=600 -D__EXTENSIONS__ -fopenmp -DDCAMPROF_VERSION=\"0.5.3\" jsonio.c
jsonio.c:77:1: warning: 'json_print' defined but not used [-Wunused-function]
json_print(cJSON *js)
^
gcc -o cJSON.o -c -I. -Wall -std=c99 -g -O2 -D_XOPEN_SOURCE=600 -D__EXTENSIONS__ -fopenmp -DDCAMPROF_VERSION=\"0.5.3\" cJSON.c
gcc -o target.o -c -I. -Wall -std=c99 -g -O2 -D_XOPEN_SOURCE=600 -D__EXTENSIONS__ -fopenmp -DDCAMPROF_VERSION=\"0.5.3\" target.c
gcc -o xyz2spec.o -c -I. -Wall -std=c99 -g -O2 -D_XOPEN_SOURCE=600 -D__EXTENSIONS__ -fopenmp -DDCAMPROF_VERSION=\"0.5.3\" xyz2spec.c
gcc -o observers.o -c -I. -Wall -std=c99 -g -O2 -D_XOPEN_SOURCE=600 -D__EXTENSIONS__ -fopenmp -DDCAMPROF_VERSION=\"0.5.3\" observers.c
gcc -o spectraldb.o -c -I. -Wall -std=c99 -g -O2 -D_XOPEN_SOURCE=600 -D__EXTENSIONS__ -fopenmp -DDCAMPROF_VERSION=\"0.5.3\" spectraldb.c
gcc -o spectraldb_munsell.o -c -I. -Wall -std=c99 -g -O2 -D_XOPEN_SOURCE=600 -D__EXTENSIONS__ -fopenmp -DDCAMPROF_VERSION=\"0.5.3\" spectraldb_munsell.c
gcc -o spectraldb_cc24.o -c -I. -Wall -std=c99 -g -O2 -D_XOPEN_SOURCE=600 -D__EXTENSIONS__ -fopenmp -DDCAMPROF_VERSION=\"0.5.3\" spectraldb_cc24.c
gcc -o spectraldb_kuopio_natural.o -c -I. -Wall -std=c99 -g -O2 -D_XOPEN_SOURCE=600 -D__EXTENSIONS__ -fopenmp -DDCAMPROF_VERSION=\"0.5.3\" spectraldb_kuopio_na
tural.c
gcc -o dcamprof -fopenmp dcamprof.o nmsimplex.o profio.o argyllio.o tps.o colmath.o matopt.o lut.o dngref.o spectrum.o jsonio.o cJSON.o target.o xyz2spec.o obse
rvers.o spectraldb.o spectraldb_munsell.o spectraldb_cc24.o spectraldb_kuopio_natural.o liblcms2.a -lm
Administrator@CLEVO /z/dcamprof-0.5.3
$
compiled .exe = https://app.box.com/s/mqv2dwhv0voswlc9sf58z8e1va21yb7l ( dcamprof.exe and libgomp_64-1.dll )
-
Hi,
One interesting aspect of DCamProf is that it can generate DCP profiles using different targets. I have earlier used one of Wolfgan Faust's IT-8 charts for checking rendition accuracy of my P45+ back.
With DCamProf it is possible to generate a DCP profile using different targets while Adobe DNG Profile Editor and ColorChecker Passport are limited to the ColorChecker.
I got a DCP profile foe my P45+ from Anders Torger that was based on test shot I made earlier. What I found is that the profile is pretty accurate, probably better than any of my profiles. BUT it needs a different WB setting in Lightroom. Setting WB on a ColorChecker gives a different WB with the DCamProf profile than with Adobe Standard Profile. Once WB is set correctly, the profile is very accurate.
There is an issue, that some of the light grey samples are to dark.
Excellent job from Anders. I hope he will continue to develop the tool.
Best regards
Erik
-
Thanks AlterEgo, will fix to next patch release.
ok, years w/o programming taking a tall...
-
Thanks for testing, I shall look at why it renders darker. In principle DCamProf does not care about absolute lightness as it's a 2.5D correction it makes, but it looks like there may be some relative lightness error too.
The WB thing is interesting. Few cameras have their preset "daylight" white balance at exactly D50, and DCamProf will correct for exactly D50 if instructed to do so, this means that it calculates which custom white balance is required to fulfill that. Conveying that information is not supported by DCP though so one have to set white balance using color pickers etc. The preset white balance will as it doesn't exactly match D50 not render neutral. For varying light conditions like in landscape this generally does not matter as it will be off anyway.
I may look into the possibility to calibrate not for D50 but for the camera's preset white balance so one will get neutral white with the preset. You would still call the light "D50" in the profile probably as there is no EXIF name that can match any light source, and the actual calibration illuminant used when lighting the target may be slightly different, this does not affect the calibration precision though, just the temp/tint information which indeed will be more wrong with this type of calibration so I'm still not sure it's a good idea. I haven't checked but I'm quite sure Lightroom ignores the DCP matrices for temp/tint numbers and always uses the builtin though. Otherwise the temp/tint numbers would change slightly when you switch profiles.
Hi,
One interesting aspect of DCamProf is that it can generate DCP profiles using different targets. I have earlier used one of Wolfgan Faust's IT-8 charts for checking rendition accuracy of my P45+ back.
With DCamProf it is possible to generate a DCP profile using different targets while Adobe DNG Profile Editor and ColorChecker Passport are limited to the ColorChecker.
I got a DCP profile foe my P45+ from Anders Torger that was based on test shot I made earlier. What I found is that the profile is pretty accurate, probably better than any of my profiles. BUT it needs a different WB setting in Lightroom. Setting WB on a ColorChecker gives a different WB with the DCamProf profile than with Adobe Standard Profile. Once WB is set correctly, the profile is very accurate.
There is an issue, that some of the light grey samples are to dark.
Excellent job from Anders. I hope he will continue to develop the tool.
Best regards
Erik
-
I got a DCP profile foe my P45+ from Anders Torger that was based on test shot I made earlier. What I found is that the profile is pretty accurate, probably better than any of my profiles. BUT it needs a different WB setting in Lightroom. Setting WB on a ColorChecker gives a different WB with the DCamProf profile than with Adobe Standard Profile. Once WB is set correctly, the profile is very accurate.
here is a purely matrix (single illuminant) profile (no LUTs) made from IT8 target (not individually measured), illuminated by halogen (no 80A filter on light or lens), w/o flatfielding (light was quite uneven), applied quickly in ACR (process 2010), with a raw containing a different target (not IT8, but colorchecker SG), w/o again measurements for that one and raw with ColorChecker SG was from imaging-resource where the light is naturally uneven on target and light was ~5xxxK something (not 2800-3000K halogen for IT8)... so a lot of handicaps here, still :
(http://s2.postimg.org/a54ubrj6x/A7_IT8_SG.jpg)
PS: I was told (by IBorg) that if errors are noticeable (say more than 2dE-whatever) it is more fair to use dE94, not dE2000... dE2000 makes your results better than they are (numbers-wise)
here is dE2000 which makes things better looking
(http://s1.postimg.org/wa2ktfagv/d_E2_K.jpg)
and visually - again, profile made off one target (IT8) applied to a totally different target (ColorCheckerSG) :
(http://s21.postimg.org/haj6b4m47/visual.jpg)
-
Otherwise the temp/tint numbers would change slightly when you switch profiles.
Adobe's profiles for a given camera model /at least for recent generations/ have always the same CMs...
-
Erik, I haven't used patchtool so I don't know how it works. How is the output from "my" profile made? Looking at the grayscale range at the bottom of the IT-8 target the patchtool image shows the brightest patch from "my" profile way too dark, even darker than the second brightest.
If I look with my eyes at the actual output from say RawTherapee there is no such error. I've attached an image of a linear rendering of the actual test target shot with the p45plus.dcp applied. As you can see the brightest patch down to the left is brighter than the second brightest unlike in the patch tool image.
-
Adobe's profiles for a given camera model /at least for recent generations/ have always the same CMs...
But what happens if you change to a custom profile, such as one generated by DCamProf? If I remember correctly the temp/tint will stay the same in Lightroom, which means that it derives those numbers from builtin matrices rather than from the profile. Most likely though if the custom profile is dual illuminant it uses those matrices to get the temperature which is used for matrix/LUT mixing but it doesn't show it in the GUI, so the temp/tint shown in the GUI is just for informational purposes, and I guess Adobe think those cannot be improved upon.
In the actual low level pipeline the white balance end up being multipliers for raw RGB channels, the temp/tint is just a transform using the CM(s)and the precision is of course limited as the camera doesn't measure the full spectrum. It becomes even more approximate if the illuminant used during profiling doesn't exactly match D50 or D65 or whatever the profile is tagged with. Most likely Adobe uses SSFs to derive the CMs, in that case the illuminants can be made exact and CMs will be better at predicting actual temperature.
-
Hi Anders,
I may have messed the conversion in Lightroom, to many sliders...
More testing today...
Best regards
Erik
Erik, I haven't used patchtool so I don't know how it works. How is the output from "my" profile made? Looking at the grayscale range at the bottom of the IT-8 target the patchtool image shows the brightest patch from "my" profile way too dark, even darker than the second brightest.
If I look with my eyes at the actual output from say RawTherapee there is no such error. I've attached an image of a linear rendering of the actual test target shot with the p45plus.dcp applied. As you can see the brightest patch down to the left is brighter than the second brightest unlike in the patch tool image.
-
I opted for spydercheckr with 24 patches. It's not that expensive or complicated and good enough for my purposes:
https://www.youtube.com/watch?v=rAG7pyA0V_8
-
PS: I was told (by IBorg) that if errors are noticeable (say more than 2dE-whatever) it is more fair to use dE94, not dE2000... dE2000 makes your results better than they are (numbers-wise)
Thanks for testing.
Concerning Delta E the idea of it is that color difference should be "perceptually uniform", that is DE 5 should *to the eye* look equally large difference between two yellows as between two reds. Today CIEDE2000 (dE2000) is the current best concerning perceptual uniformity, meaning that it sometimes produces larger numbers than older standards and sometimes smaller. That dE94 would be more "fair" because it produces larger numbers for larger errors is a strange thing to say, DE is not about fairness, it's about perceptual uniformity. If de94 exaggerates the error then de2000 is more correct.
dE2000 is like any color standard not perfect though, but I haven't read/heard anything about that dE94 would in some ranges produce more perceptually uniform numbers than dE2000.
This is the reason why DCamProf exclusively use dE2000 when optimizing. You can adjust the k-weights with the -w parameter though, so you can for example say that you're more interested in hue accuracy than in lightness.
-
I opted for spydercheckr with 24 patches. It's not that expensive or complicated and good enough for my purposes:
https://www.youtube.com/watch?v=rAG7pyA0V_8
Those are fine easy-to-use products. Although you could use it like that, DCamProf is not intended to be a replacement for those, it's a much more advanced tool aimed at users with special interest or need in camera profiling. It's not easy to use and it doesn't hurt to know a bit color science before digging in.
That said the basic workflow using a macbeth colorchecker 24 and just defaults is relatively easy to get through.
The main reason I made the tool is because there where so many unanswered questions about profiling and camera performance that the available software could not provide answers to. Without being able to process SSFs a lot of things remain unknown, how accurate is a matrix profile, how good is cc24-based profiles, what is the limits of camera color separation etc. Using DCamProf all those questions can get an answer. I haven't really had time to get that far though, but at some point I'm probably going to write an article on camera profiling and camera color that answers a few of those questions I have myself.
That and because the profiles the commercial tools I've tested seemed to have issues with smoothness and did not allow me to control tradeoff between smoothness and accuracy (haven't tested spydrchecker though). With DCamProf you can control the smoothness/accuracy tradeoff, and you can also do things like saying you're more interested in hue accuracy than lightness accuracy, and that way gain more smoothness.
-
But what happens if you change to a custom profile, such as one generated by DCamProf? If I remember correctly the temp/tint will stay the same in Lightroom, which means that it derives those numbers from builtin matrices rather than from the profile. Most likely though if the custom profile is dual illuminant it uses those matrices to get the temperature which is used for matrix/LUT mixing but it doesn't show it in the GUI, so the temp/tint shown in the GUI is just for informational purposes, and I guess Adobe think those cannot be improved upon.
In the actual low level pipeline the white balance end up being multipliers for raw RGB channels, the temp/tint is just a transform using the CM(s)and the precision is of course limited as the camera doesn't measure the full spectrum. It becomes even more approximate if the illuminant used during profiling doesn't exactly match D50 or D65 or whatever the profile is tagged with. Most likely Adobe uses SSFs to derive the CMs, in that case the illuminants can be made exact and CMs will be better at predicting actual temperature.
It is even worse - the only white balance that stays the same across various DCP profiles is "As Shot" one. This is where it goes from multipliers to colour temperature/tint (and that will be different for different profiles). For all other presets or direct changes of colour temperature/tint in Lightroom/ACR - the path is backwards, from temperature to the multipliers (or more precisely mapping from XYZ to camera unbalanced to get white levels for that colour temperature/tint using CM or its approximation from the two given ones). And it is the colour temperature/tint that is considered primary in Lightroom/ACR so this is the parameter that is preserved with raw development - not the chosen corresponding white level in raw (which would be more correct and work across profile changes).
The wb does not have to match D50 - it is chromatically adapted to that (by white balancing and applying the forward matrix). As far as the DCP is concerned the matter is in whitebalancing the scene. The idea with two matrixes for different illuminants was that the right colour temperature is determined from the WB of the scene (and considered to be a scene illuminant), then the approximated matrixes are calculated for the scene illuminant (proportionally being between illuminants of two matrixes in the profile) and that the forward matrix obtained that way will be chromatically adapted to D50 XYZ (with WB of the raw when applied will have 1,1,1 mapped to D50 with the FM obtained this way). I am yet to be convinced that this is any better than ICC profiles which are effectively a single FM without D50 XYZ limitation (if we consider only matrix with shapers type of ICC for the moment). RPP uses (in most cases) matrix shaper ICC profiles without any problems.
-
thanks bro
-
It is even worse
Thanks for the info, very interesting. Dual-illuminant profiles are really only a user convenience feature, so you don't need to switch between appropriate single illuminant profiles. I'm also doubtful that they actually provide any meaningful extra accuracy in the range between StdA and D65 which is the typical pair used. With DCamProf and SSF simulations you can answer that question though, you can render a dual-illuminant profile and then a single-illuminant for inbetweener and compare their performance. This is one of the experiments I intend to do in the coming weeks.
I'd like to add ICC support too, but it's quite massive work so it will probably be a while. The boring thing with ICC is that preprocessing is not standardized so Capture One ICCs, RT ICCs, Leaf ICCs, Phocus ICCs and I guess RPP ICCs are not really compatible with eachother, so I would have to add some "pre-processing specification" to support ICC. Argyll have some problem generating ICCs for say Capture One as Argyll expects linear input and when it gets curve-distorted inputs it makes less than ideal optimizations. For linearly preprocessed pipeline, like RT for example, I guess Argyll makes a really good job already, but it's optimization engine seems more targeted towards the scanner use case than the camera.
-
RPP uses (in most cases) matrix shaper ICC profiles without any problems.
From the limited experiments I've done so far it seems like matrix-only profiles for modern cameras are generally quite good performers for the colorcheckers. There's even a standardized measure for that, the Sensitivity Metamerism Index that DxOMark includes, the higher the number the better matrix-only match is possible. When it comes to high saturation colors outside the typical test target it seems though like a LUT can improve things significantly.
If the preprocessing step before ICC is applied is linear the shaper (ie linearization curves) step should not really add anything as far as I can see. Cameras today are linear, and I don't think shaper curves can do anything LUT-like, but maybe I'm wrong? For preprocessing like Capture One does shaping is required, but it would probably be better to use a fixed linearization shaper than trying to optimize something without knowing how the preprocessing is made. At least it seems so from the brief tests I've made using Argyll on Capture One.
-
I'd like to add ICC support too, but it's quite massive work so it will probably be a while.
Anders, that would be hugely appreciated, as is the current tour-de-force. I'm impressed with what you've already done so far, thanks.
The boring thing with ICC is that preprocessing is not standardized so Capture One ICCs, RT ICCs, Leaf ICCs, Phocus ICCs and I guess RPP ICCs are not really compatible with eachother, so I would have to add some "pre-processing specification" to support ICC.
Yes, but then DCamProf would immediately become the best solution for camera profiling, and fill a huge gap left by other 'solutions'. I've noticed some brightness shift issues in Capture One Pro with the Argyll based wrapper (MakeInputICC) that Iliah Borg made available, when I select a linear tone curve (the default ICC profile for my camera has gamma 1.9-ish TLCs, but not pure gamma). So being able to adjust for that in the profile creation stage would be fabulous.
Cheers,
Bart
-
Thanks for the info, very interesting. Dual-illuminant profiles are really only a user convenience feature, so you don't need to switch between appropriate single illuminant profiles. I'm also doubtful that they actually provide any meaningful extra accuracy in the range between StdA and D65 which is the typical pair used. With DCamProf and SSF simulations you can answer that question though, you can render a dual-illuminant profile and then a single-illuminant for inbetweener and compare their performance. This is one of the experiments I intend to do in the coming weeks.
Eventually perhaps - I am trying to get my own monochromator setup (not quite there yet) to obtain spectrum for my Kodak cameras/backs. Then I'll do the experimanting. For now single matrix profiles perform just as good.
I'd like to add ICC support too, but it's quite massive work so it will probably be a while.
You are already using lcms2 and it has capabilities to create profile from scratch and write it out to the file. The dcp2icc tool uses it like that.
The boring thing with ICC is that preprocessing is not standardized so Capture One ICCs, RT ICCs, Leaf ICCs, Phocus ICCs and I guess RPP ICCs are not really compatible with eachother, so I would have to add some "pre-processing specification" to support ICC. Argyll have some problem generating ICCs for say Capture One as Argyll expects linear input and when it gets curve-distorted inputs it makes less than ideal optimizations. For linearly preprocessed pipeline, like RT for example, I guess Argyll makes a really good job already, but it's optimization engine seems more targeted towards the scanner use case than the camera.
True - but then from prospective of getting the accuare colour none of that would not matter. This only start becoming a problem when profile adds some nonlinearity (as in LUT type of corrections which could be dependent on exposure corrections and so on).
From the limited experiments I've done so far it seems like matrix-only profiles for modern cameras are generally quite good performers for the colorcheckers. There's even a standardized measure for that, the Sensitivity Metamerism Index that DxOMark includes, the higher the number the better matrix-only match is possible. When it comes to high saturation colors outside the typical test target it seems though like a LUT can improve things significantly.
If the preprocessing step before ICC is applied is linear the shaper (ie linearization curves) step should not really add anything as far as I can see. Cameras today are linear, and I don't think shaper curves can do anything LUT-like, but maybe I'm wrong? For preprocessing like Capture One does shaping is required, but it would probably be better to use a fixed linearization shaper than trying to optimize something without knowing how the preprocessing is made. At least it seems so from the brief tests I've made using Argyll on Capture One.
My understanding is that LUT based profiles can introduce non linearity and as such result in colour shift (and transformation noise), matrix preserves linearity and is essentially better (though possibly less precise).
-
I've noticed some brightness shift issues in Capture One Pro with the Argyll based wrapper (MakeInputICC) that Iliah Borg made available, when I select a linear tone curve (the default ICC profile for my camera has gamma 1.9-ish TLCs, but not pure gamma). So being able to adjust for that in the profile creation stage would be fabulous.
MakeInputICC will calculate the gamma from the meaurements. So if you are using RawDigger to get the target measurement, it will be close to the one you select there when saving the samples grid (at least that was my experience).
-
My understanding is that LUT based profiles can introduce non linearity and as such result in colour shift (and transformation noise), matrix preserves linearity and is essentially better (though possibly less precise).
Yes for things like HDR merging a matrix-only profile is preferable thanks to the linearity, but if merging takes place on the raw level (like Lumariver HDR can do) it doesn't matter. Otherwise my personal opinion is that matrix profiles is not preferable, the linearity is a sort of mathematical beauty but I find that improved accuracy especially for high saturation colors is preferable. LUTs have a bit bad reputation though as software before has used them to correct without thinking about smoothness, and is often used to create "looks" which one may not agree with. I hope DCamProf can improve this reputation later on.
I have some ICC code since before, I'll see if I'll use lcms2 or not for parsing/writing, lcms2 supports the whole large ICC standard and DCamProf will use only a small subset, it may be less work sourcing in my old code than "dumbing down" lcms2. That part is anyway only say 20% of what needs to be done to get full support.
-
Thanks for testing.
Concerning Delta E the idea of it is that color difference should be "perceptually uniform", that is DE 5 should *to the eye* look equally large difference between two yellows as between two reds. Today CIEDE2000 (dE2000) is the current best concerning perceptual uniformity, meaning that it sometimes produces larger numbers than older standards and sometimes smaller. That dE94 would be more "fair" because it produces larger numbers for larger errors is a strange thing to say, DE is not about fairness, it's about perceptual uniformity. If de94 exaggerates the error then de2000 is more correct.
dE2000 is like any color standard not perfect though, but I haven't read/heard anything about that dE94 would in some ranges produce more perceptually uniform numbers than dE2000.
This is the reason why DCamProf exclusively use dE2000 when optimizing. You can adjust the k-weights with the -w parameter though, so you can for example say that you're more interested in hue accuracy than in lightness.
well, you shall be better off asking him directly, here is what he said not so long ago, about why dE94 is better when we evaluate the output using custom profiles vs simulations using .cie (with BabelColor Patchtool) :
> why dE94 is better ?
Because dE2000 by design is intended for evaluation of smaller differences. dE94 is a good middle ground.
-
Yes for things like HDR merging a matrix-only profile is preferable thanks to the linearity, but if merging takes place on the raw level (like Lumariver HDR can do) it doesn't matter. Otherwise my personal opinion is that matrix profiles is not preferable, the linearity is a sort of mathematical beauty but I find that improved accuracy especially for high saturation colors is preferable. LUTs have a bit bad reputation though as software before has used them to correct without thinking about smoothness, and is often used to create "looks" which one may not agree with. I hope DCamProf can improve this reputation later on.
My understanding is that the accuracy will be better with matrix simply because the linearity is preserved. LUT profiles will introduce colour shifts because they can introduce non linearity and hue shifts
-
MakeInputICC will calculate the gamma from the meaurements. So if you are using RawDigger to get the target measurement, it will be close to the one you select there when saving the samples grid (at least that was my experience).
Hi Alexey,
Yes, but (besides the mostly linear RawDigger to CGATS) the LUT gamma is not a pure gamma, when I look at the CLUT slope of the default EOS-1Ds Mark III profile that comes with C1 , and try to fit a function. So if that's what C1 expects, then the profile should be more flexible than having a simple Matrix and/or gamma based CLUTs. I therefore see the difficulty that that poses for Anders, given that different assumptions are being used by different applications.
Cheers,
Bart
-
My understanding is that the accuracy will be better with matrix simply because the linearity is preserved. LUT profiles will introduce colour shifts because they can introduce non linearity and hue shifts
LUT profiles also can be obtained from camera's SSFs, spectrum of illumination, etc... so while they can LUT is not a bad thing if done in a proper manner... guns do kill too, not that I am pro-NRA.
-
Hi Alexey,
Yes, but (besides the mostly linear RawDigger to CGATS) the LUT gamma is not a pure gamma, when I look at the CLUT slope of the default EOS-1Ds Mark III profile that comes with C1 , and try to fit a function. So if that's what C1 expects, then the profile should be more flexible than having a simple Matrix and/or gamma based CLUTs. I therefore see the difficulty that that poses for Anders, given that different assumptions are being used by different applications.
Cheers,
Bart
well, you can always use C1 itself to make the tiff and take it from there, instead of using RD (or lobby for RD v2.0, long overdue, to include some more complex processing when generating cgats in addition to simple gamma)
PS: there is also linear scientific curve .fcrv that can be used instead of linear curve (just rename it) - for example the content of linear scientific curve .fcrv will switch off blown highligts correction in C1... try it
-
LUT profiles also can be obtained from camera's SSFs, spectrum of illumination, etc... so while they can LUT is not a bad thing if done in a proper manner... guns do kill too, not that I am pro-NRA.
You are missing the point - it does not matter how you build them and with what precision. LUT is a grid of space to space mapping in the nodes and between the nodes it gets linearly interpolated. This interpolation can add nonlinearities especially if LUT nodes are chosen to correct some of the colours camera cannot distinguish. In these cases we are in with the hue shifts when exposure correction is applied.
-
You are missing the point - it does not matter how you build them and with what precision. LUT is a grid of space to space mapping in the nodes and between the nodes it gets linearly interpolated. This interpolation can add nonlinearities especially if LUT nodes are chosen to correct some of the colours camera cannot distinguish. In these cases we are in with the hue shifts when exposure correction is applied.
well, you can find out what "colours camera cannot distinguish" and then it is not going to be worse than with matrix in terms of what you do with camera RGB input there... why shifts if exposure correction is after color transform (not RPP) ?
-
well, you can find out what "colours camera cannot distinguish" and then it is not going to be worse than with matrix in terms of what you do with camera RGB input there
In what way it is not going to be worse? My understanding is that it will always be worse when your exposure is not spot on (at least it will increase the chances of those shifts).
-
DCamProf uses a 2.5D LUT, this means that regardless of lightness the same correction is applied. This means that there is no exposure-dependent hue-shift in the profile. In very rigid copy-style setups with known objects to shoot a 3D LUT can be better (this is disussed in the docs), but in all other cases a 2.5D LUT is wiser. Adobe uses 3D LUT "LookTable" in most their profiles but that is to create a subjective Adobe-look(tm) which is a different thing.
There are other sources of non-linearities and hue shifts though, S-contrast curve, desaturation of highlights to mimic film behavior etc.
DCamProf's LUT nodes are the patches, after patches with (almost) the same chromaticity has been grouped together, and then depending on weighting those nodes can be relaxed towards neutral to minimize stretch/compression/bend. You can plot the LUT using gnuplot and adjust weights as desired.
By setting grouping distance to zero (-d 0) you can get the LUT correct each patch spot on, but that can lead to crazy bends, and as same color can be represented by different spectra (and will produce different errors) it does not really make sense to get spot on.
Any fairly recent camera has overlapping filters and is relatively good at color separation. There are still minimas where they don't separate too well, but if that happens to be in one of your patches I don't really see how that would be a problem. The LUT will then make an average correction for that general group of colors.
It is a problem if the LUT makes extreme stretches, but if that happens then something is generally very wrong, bad lighting of test target, bad reading of test target, bad reference file etc.
-
In what way it is not going to be worse?
because you always can do around the same transform as matrix profile does around those areas, no ? matrix is LUT of sort too, just the one that does purely linear transform (multiplication).
-
because you always can do around the same transform as matrix profile does around those areas, no ? matrix is LUT of sort too, just the one that does purely linear transform (multiplication).
You can use matrix to generate the same nodes as LUT yes. But whether the results of applying such profile for colours that located between those nodes will match the matrix one remains debatable.
-
You can use matrix to generate the same nodes as LUT yes. But whether the results of applying such profile for colours that located between those nodes will match the matrix one remains debatable.
so Alex, what is going to happen if you (hopefully) get your monochromaror + sphere + ... working ? you will get SSF and then you will build matrix (plus possible gamma/s/ or shaper/s/) profile or you will go for LUT ?
-
so Alex, what is going to happen if you (hopefully) get your monochromaror + sphere + ... working ? you will get SSF and then you will build matrix (plus possible gamma/s/ or shaper/s/) profile or you will go for LUT ?
I will build my perfect matrix profile (not relying on CC24 or other targets) ;D
I was not even interested in LUTs. If you need specifics I am interested in finding the better fitting matrix for (a) different illuminants and (b) for better matching the digital back with specific IR filter (the one I have namely). LUT would be interesting for me if I had nonlinearities (perhaps like those in D800) to correct for but as it stands I don't need it.
DCamProf is a perfect tool for me to experiment with these.
-
I will build my perfect matrix profile
waiting to see the results then !
-
Just to show an example, this is how a LUT stretching can look like, the test target is an IT8 which has been cleaned up a bit to exclude most darker patches, the camera is a P45+.
Colored squares show the target positions, arrows show the LUT stretch from matrix position to end position (generally close to the target). The grid shows the LUT thin plate spline in action. There's some stretch in the lightness direction too (not seen in this view) but it's very small.
There are indeed quite large stretches, but no bad bends as there are no contradicting stretches.
It shows very typical behavior in that for low saturation colors there are small stretches (ie the matrix comes close to target), and for high saturation colors there are larger stretches. The u'v' diagram is not perfectly uniform though so it exaggerates that effect a bit.
This is natural, more saturation means more narrow band, which means that the difference between the camera's SSFs and the observer's CMFs are enlarged and becomes harder to correct with a linear equation.
A general comment on the IT8 target, the many repeats of the darker shades does not provide that much value, but the target does reach considerable higher saturation than a cc24, some patches even outside Pointer's gamut, so it looks like it's quite a good target.
-
A general comment on the IT8 target, the many repeats of the darker shades does not provide that much value
I'd love to see some further discussion, becuse just recently somebody was stating even that intentional bracketing is worthwhile - see http://forum.luminous-landscape.com/index.php?topic=22471.msg818606#msg818606
-
so it looks like it's quite a good target.
I had this one from WFaust for like 7 years and it was somewhere far away in the attic... but I extracted it back couple of days ago and liked (to my subjective taste) some results, I am certainly not an expert to render a scientific grade judgement - just my personal impression (so far)... for further experimenting I am assembling some additional gear - like it was suggested 80A filters for halogen light, waiting to get them... easel to mount targets on a proper surface (metal with magnets) - again as for example IBorg suggested
-
waiting to see the results then !
Sure - if I make the monochromator work...
-
I'd love to see some further discussion, becuse just recently somebody was stating even that intentional bracketing is worthwhile - see http://forum.luminous-landscape.com/index.php?topic=22471.msg818606#msg818606
I think Iliah is referring to bracketing as a method to keep down noise. I don't think that is worthwhile either though. If by principle there will be relatively large errors in the results, say "5%" it doesn't provide much value to reduce measurement error from 0.5% to 0.1%. There is such a thing called overkill.
Darker patches provide some value in that they are probably printed using a different colorant mix so you get a slightly different spectrum shape than a lighter color of the same hue and chroma, that way you can make some more average correction. For very dark patches I would guess they contribute more noise than meaningful correction though. And yes I guess in that case bracketing would help, but why not have a test target with brighter colors instead I think.
Anyway, if you try to make exact correction for each shade, you are in 3D LUT space and pretending that the camera will be used as a scanner, with fixed light, fixed exposure *and* fixed media. The IT8 target steps is nice for a slide film target for a slide film scanner, I actually have one and have made Argyll ICC profiles for a slide scanner using it. As long as the slides you're scanning are made on the same film as the slide target you're getting very accurate results.
With a camera used in general conditions it won't be like that. Let's pretend for a while that the light is fixed and the media is fixed, but not the exposure. Then the 3D LUT correction still breaks, since when a color is exposed brighter than it was for the LUT it will be corrected wrong.
That is to DCamProf with a 2.5D LUT the only thing the darker shades provide is slight variation, but then I think a neighboring chromaticity of similar lightness would be better use of target space.
-
Sure - if I make the monochromator work...
I'm thinking about getting a monochromator too. You can get one new for about $1200. Most use a spectroradiometer ($10000) to measure the output, and many an integrating sphere to smoothen the light ($2000 I suppose). So naturally one would like to skip the spectroradiometer and integrating sphere.
I'm thinking that one could use a halogen lamp, measure the spectra with a consumer spectrometer, then hope it will stay stable enough while filtering it through the monochromator and doing measurements, and you shoot the slit directly, without any diffuser. I think it could work, but I don't know, may be issues with reflections. If one uses a simple white card to reflect the light shooting becomes simpler but measuring the reference becomes harder, then one needs a spectroradiometer rather than a consumer spectrometer.
What's your plan?
Here below a high end setup:
http://www.cis.rit.edu/jwgu/research/camspec/img/camspec_database.png
-
Hi Anders,
I have checked back on this and I now believe it is Lightroom that applies some kind of shoulder on the highlights. So it is not a profile issue.
Best regards
Erik
Hi,
....
There is an issue, that some of the light grey samples are to dark.
Excellent job from Anders. I hope he will continue to develop the tool.
Best regards
Erik
-
With a camera used in general conditions it won't be like that. Let's pretend for a while that the light is fixed and the media is fixed, but not the exposure. Then the 3D LUT correction still breaks, since when a color is exposed brighter than it was for the LUT it will be corrected wrong.
what is the difference between more exposure time and more light (intensity wise = brighter patches) in this example ? you will not see the difference in the raw file (let us ignore sensor heating)
-
I'm thinking that one could use a halogen lamp, measure the spectra with a consumer spectrometer, then hope it will stay stable enough while filtering it through the monochromator and doing measurements, and you shoot the slit directly, without any diffuser. I think it could work, but I don't know, may be issues with reflections. If one uses a simple white card to reflect the light shooting becomes simpler but measuring the reference becomes harder, then one needs a spectroradiometer rather than a consumer spectrometer.
What's your plan?
Much more modest - I got the full spectrum monochromator (manual one with light source) but no power supply (and a weird connectors for PSU). Not sure if it will work and it will need some calibrating/checking. I also will have an integrating sphere (albeit a small one) though I am half way though building my own (not so high spec of course as spectralon coated ones). Will update with progress if I make it work.
I am not sure how accurate the measurement without the sphere will be since you need to measure the irradiation (intensity) to adjust the camera responses and without sphere those readings will be hit or miss.
These discussions should perhaps go to a different thread - this one is about DCamProf after all.
-
Hi Anders,
I have checked back on this and I now believe it is Lightroom that applies some kind of shoulder on the highlights. So it is not a profile issue.
Best regards
Erik
generally it shall be process 2010 in ACR/LR and also just in case infuse a linear tone curve (applied post exposure adjustments) in profile as
<ToneCurve Size="2">
<Element N="0" h="0.000000" v="0.000000"/>
<Element N="1" h="1.000000" v="1.000000"/>
</ToneCurve>
in dcptool speak
-
What's your plan?
have to rob nearby university... or enroll in MSc :D (repeat for each new camera)
-
generally it shall be process 2010 in ACR/LR and also just in case infuse a linear tone curve (applied post exposure adjustments) in profile as
<ToneCurve Size="2">
<Element N="0" h="0.000000" v="0.000000"/>
<Element N="1" h="1.000000" v="1.000000"/>
</ToneCurve>
in dcptool speak
I should probably add a function to add a curve to the DCP. If there is no curve, like DCamProf DCPs are, some raw converters seem to add a default curve which may not be what you want. Meanwhile one can add one manually with dcp2json / json2dcp (or dcptool I suppose)
-
what is the difference between more exposure time and more light (intensity wise = brighter patches) in this example ? you will not see the difference in the raw file (let us ignore sensor heating)
No difference. But I was describing an example when exposure changes for a fixed light condition.
The difference then is that the LUT was designed with the patches at a different lightness. With a 2.5D LUT as DCamProf uses it does not matter, as the index into the LUT is only chromaticity, not lightness. If there's a 3D LUT index is hue, saturation (chromaticity) plus lightness. To match the correct patch the exposure must be the same as when the LUT was designed.
When lightness is not a parameter into the LUT there is much less value in making steps, the only value is in adding slight variation in terms of chromaticity and spectral shapes.
-
I should probably add a function to add a curve to the DCP. If there is no curve, like DCamProf DCPs are, some raw converters seem to add a default curve which may not be what you want. Meanwhile one can add one manually with dcp2json / json2dcp (or dcptool I suppose)
Lightroom/ACR add default one which is mix between gamma and contrast curve. So if linear profile is needed, then curve should be explicitly stated.
-
Just a bug (maybe) in the "txt2ti3" command of dcamprof.
I assume that the input is a text file, whitespace delimited, that contains the spectral data from, for example, PatchReader. Nothing else. My PatchReader data is in 10nm increments, in rows, from 380nm to 730nm. I cut and paste the spectral data from the PatchReader output text file into a text document called "spectral_data.txt" and issue:
$ dcamprof txt2ti3 -f 380,730,10 -l rows spectral_data.txt spectral_data.ti3
The range of the spectral elements is correct (380-730) but the increments are still in 5nm (the default) not 10nm (my command line argument to the -f flag). Attached is a test file - spectraltest.txt - that is a copy of the whitespace delimited input file.
Still, it beats editing the PatchReader output by hand to conform to the reference CIE file you include with your distribution!
kirk
EDIT -Is this command supposed to reissue/recalculate/interpolate my input spectra into 5nm increments, regardless of what the specified input spacing is? If so, sorry, I misunderstood what the output was supposed to be!
EDIT2 - The scaling seems to be buggy though - I noticed in the reference CIE file that you include, the range of spectral values is 0-100 (not 0.0-1.0). Given this reference range, I output my values from PatchReader as 0-100. When I run txt2ti3, I do not specify a scaling value (I assume the default, -s 1.0 according to the help message). However, my spectral values get scaled by 100 in the output file. If I use "-s 0.01" then the scaling is not applied and the output values are identical to the input values. Is there a multiplication of 100 going on somewhere? Or is the default value for the -s flag 100 and not 1.0?
-
Currently the txt2ti3 always resamples to 5nm increments and 0-100 scale. I'll probably change to not resample, I just thought it would be easier to manually cut and paste if always in 5 nm, as a single file must have the same sampling.
The 0-100 scaling takes place because the file is supposed to look like an Argyll file and Argyll uses 0-100 as default scale for spectra, ie 100 means 1.0. When reading the .ti3 file DCamProf translates to 1.0 internally.
I suppose the problem is that you want to use the .ti3 file in some other software than DCamProf or Argyll and therefore want a 0-1 range? The "hack" with providing 0.01 as scale I guess works then, or do I need to add some feature?
-
Hi Anders,
No problem- I am using it with dcamprof. The problem was I exported from patch reader already scaled 0-100. I thought the txt2ti3 "-s" flag was a scaling value to use when computing the output. That is, because my values are already 0-100 I would want to use "-s 1.0" (don't scale). I guess the scale value is the scale rNge that exists in the in the input file?
Either way, please don't wory about it!
Kirk
-
Alexey, concerning the use of monochromator to make the best matrix profile I think you're in overkill space.
The thing is that to make a good matrix fit the matching task must not be too difficult. With modern cameras the CC24 is matched pretty good with a matrix, but that's because there are no extreme saturation colors. It can be seen that this good matching of the CC24 is made at cost of extreme colors.
If you instead try to make a matrix that will match also extremely saturated colors, with the help of a glossy target or indeed SSF and spectral data, you will get a much poorer match so the CC24-range of colors will suffer. You can try this with DCamProf now using the provided SSF example. Pure matrix profiles are best when optimized only for normal range colors, and for normal range colors a standard test target procedure will do just as good as SSF, possibly even better as I think it's harder to make an exact monochromator measurement.
I also know that SSF is not really something *all* manufacturers are using, some design the profiles the traditional way using targets and subjective tunings, so one cannot really say that SSF will make better profiles. It's probably easier to make dual-illuminant DCPs with better color temp predictions though as you can use the exact standardized illuminants though. Adobe is probably using SSFs.
That said I very much myself like to be able to measure SSF since once you have them you can make lot of interesting experiments, but I'm not really sure I would use an SSF-based profile for "production", haven't decided on that. It is possible to merge both SSF and traditional measurement into one .ti3 and make a profile from that too, but it's probably difficult as light sources need to match exactly.
But if I would only want a matrix profile, I would do it based on a test target.
-
Anders,
I have build a few of the matrix profiles from larger CC24 and passport and without further editing in daylight lighting they are hit or miss to say the least. Iliah makes cracking matrix profiles for RPP but as far as I know this involve a lot of manual colour correction past shooting profile based on sample images of known colours and reprofiling again.
The idea with spectral data for a sensor is to automate that manual colour correction step because all the expected responses for the needed colours of a scene can be generated and better fitting matrix produced. So the approach is far from useless to me.
The only decent matrix profiles I have managed to build from target and without post processing were made with QPCard 203 (a version of it not glued to the cardboard holder so was very easy to achieve needed flatness). Those still weren't perfect in some areas even on cameras with good colour separation (in that particular instance Kodak Pro Back with custom IR filter).
-
Ok, for that purpose having SSF will be very convenient of course.
-
I also know that SSF is not really something *all* manufacturers are using, some design the profiles the traditional way using targets and subjective tunings, so one cannot really say that SSF will make better profiles.
who are those ? are they "manufacturers" of 3rd party raw converters of note or "manufacturers" of cameras that do supply either their own OEM software (or putting OEM dcp profiles in DNG files - like Ricoh/Pentax for example) ? it is interesting to know the names and compare vs others who then allegedly are using SSFs then, no ?
-
who are those ? are they "manufacturers" of 3rd party raw converters of note or "manufacturers" of cameras that do supply either their own OEM software (or putting OEM dcp profiles in DNG files - like Ricoh/Pentax for example) ? it is interesting to know the names and compare vs others who then allegedly are using SSFs then, no ?
I get some things shared in confidence so in this case I did not want to share the names, you just have to trust me ;-)
-
you just have to trust me
it is not about trust that they do, it is about who they are, so that we can see how they fare vs others really... but it seems that you are not going to tell, unless we get some help from CIA :D !
-
it is not about trust that they do, it is about who they are, so that we can see how they fare vs others really... but it seems that you are not going to tell, unless we get some help from CIA :D !
Anders simply may be bound by NDA
-
I have some ICC code half-working now, probably release next week. I just need to figure out some white balance confusion I'm having...
It will be matrix-only for linear pipelines only in the first release, so no Capture One. But I plan to support that eventually too.
-
Hi Anders,
This image may be an interesting sample: http://echophoto.dnsalias.net/ekr/Articles/OLS_OnColor/SimpleCase/Data/20150107-CF046070_AdobeStandard.iiq
Color samples are here:
http://echophoto.dnsalias.net/ekr/Articles/OLS_OnColor/SimpleCase/Data/GreenBlade_spectrum.txt
http://echophoto.dnsalias.net/ekr/Articles/OLS_OnColor/SimpleCase/Data/Violet.txt
http://echophoto.dnsalias.net/ekr/Articles/OLS_OnColor/SimpleCase/Data/Green.png
http://echophoto.dnsalias.net/ekr/Articles/OLS_OnColor/SimpleCase/Data/Violet.png
Best regards
Erik
-
If you instead try to make a matrix that will match also extremely saturated colors, with the help of a glossy target or indeed SSF and spectral data, you will get a much poorer match so the CC24-range of colors will suffer.
As I understand it, to make a useful profile using sensor spectral measurements, you need to introduce a weighted model of real world colors, so that your matrix has least errors for "typical" use (where "typical" is set by your real world color weighting).
-
As I understand it, to make a useful profile using sensor spectral measurements, you need to introduce a weighted model of real world colors, so that your matrix has least errors for "typical" use (where "typical" is set by your real world color weighting).
Yes that's probably a good idea, and you can do that with DCamProf using the -w parameter in the make-profile step.
I haven't had time to experiment that much with matrix-only profiles but it seems like the best "weighting" is actually to exclude those highly saturated colors and only include those that you expect to see in normal situations. It seems like the extreme colors are struggling too much in a different direction that it's impossible to make a reasonable match for those without clearly degrading the normal colors. Results will vary depending on camera though.
-
The link to the IIQ doesn't work, and the greenblade spectrum text file lacks spectral data.
I'm using the provided patchtool text file examples to make the dcamprof make-target file parser a little bit more accepting on formats.
Hi Anders,
This image may be an interesting sample: http://echophoto.dnsalias.net/ekr/Articles/OLS_OnColor/SimpleCase/Data/20150107-CF046070_AdobeStandard.iiq
-
0.5.4 executable built for windows with mingw ( includes dcamprof.exe and libgomp_64-1.dll ) = https://app.box.com/s/l9w24lobqr2nr08439zxpzndcmvwkj70
-
Hi Anders,
Sorry for the mess!
I put all info here: http://echophoto.dnsalias.net/ekr/Articles/ColorProfiles/Samples
Hopefully it works! For some reason a lot of files get protected for read from time to time.
I added a sample from my Sony Alpha 99, too.
Some background:
I had some discussion with Tim Parkin. He finds that the P45+ has inherently bad reproduction of chlorophyll green, making it yellowish. I don't disagree but I wanted to find out if it is a profiling thing or not.
So I bought a small flower, cut away some samples to measure with my ColorMunkey and shot it with strobe light. The reason a bluish purple flower was chosen was that I have seen it was a colour that easily turned into blue or reddish blue in conversion.
Best regards
Erik
The link to the IIQ doesn't work, and the greenblade spectrum text file lacks spectral data.
I'm using the provided patchtool text file examples to make the dcamprof make-target file parser a little bit more accepting on formats.
-
Thanks, I will play with the files later.
The use of camera profiling and tools to do it is not so widespread so cameras and raw converters plus profiles is inseparable for most photographers. I think the camera is blamed many times when it is more about profile design, but if you do not have the tools or knowledge to make your own the color you get from the factory and bundled software is the final result.
Cameras do differ in separation ability though, but all cameras today, even older ones like the P45, has overlapping filters so they can separate colors rather well within their noise limits. So I think it is more up to profile design than many may think.
That chlorophyll green becomes yellowish can definitely be adjusted, but if there is metamerism no profile can fix it. The color separation test in DCamProf will show where color separation is weak, but SSF curves are required to do that test. I think I have curves for the H3D 39, same sensor, probably different IR filter though. I shall play around with those and see what I find. The builtin database kuopio-natural has leaves of many kinds, and there are more databases to find online.
I know Tim thinks the SMI measure (which dxo shows) correlates well with actual performance, but I am less sure about that when you have LUT profiles. More testing is required before I could form my own view though.
A camera with poor SMI will likely have better color separation ability in some color ranges than an ideal camera (a camera where SSF = observer CMF), and that could be greens.
-
That chlorophyll green becomes yellowish can definitely be adjusted, but if there is metamerism no profile can fix it.
what if you can move the color from the area of metameric failure with some filters though... for a particular scene
-
I had some discussion with Tim Parkin. He finds that the P45+ has inherently bad reproduction of chlorophyll green, making it yellowish. I don't disagree but I wanted to find out if it is a profiling thing or not.
Best regards
Erik
From the excellent looking images I've seen of your work off the P45+, Erik, I've noticed they do come off as having a slight green (more like an ashen blueish green) bias with a light hand on saturation. Remember my suggestion of applying a luminance/saturation increase with Hue/Sat tool in Photoshop on your gallery exhibit image of the sunlit green knoll with the background mountains you linked to in the LuLa Coffee Corner forum?
For me adaptation working too long on a landscape with a magenta-ish blue sky (cobalt blue has a magenta element) requires constant retweaking of WB to get the right looking green. I do know from years of examining sunlit green plants that the sunlit portion of a non-waxy, midtone green leaf/grass does have a yellowish bias, but ACR/LR's exact hue that makes it look right requires very gradual adjusts due to the cyan portion of the green that can make shadows appear too cool which invokes the adaptive effect of seeing green highlights as warm or a thalo green.
To filter this cyan requires moving toward magenta which slightly warms up green highlights but with a much different hue. I would never let my custom DNG profile fix this because it made the greens look a slightly dull yellowish orange green. And I didn't like moving temp slider toward blue because it desaturates everything.
After a long WB edit with ACR4.4 profile I'ld walk away and come back to see I'ld made everything too green because I kept trying to get the cyan in the green tint slider to freshen up the green. So I went back to As Shot WB (returned tweaked 0 back to +6 to +10 toward magenta), selected the custom DNG profile, walked away, took a break, came back and it looked perfect. Hue changes applying the custom DNG profile is so subtle in warming up the greens that I never considered how much WB affects the overall perception of color cast. I think one has to consider color constancy in the mix of things rather than attributing it to metamerism.
I mean I had trouble getting the right green hue shooting Live Oak trees that I did a google image search and found WB and greens all over the map...
https://www.google.com/search?newwindow=1&hl=en&site=imghp&tbm=isch&source=hp&biw=1556&bih=941&q=live+oak+in+texas&oq=live+oak+in+texas&gs_l=img.3..0i8i30l2j0i24.3707.9093.0.9964.17.16.0.1.1.0.114.1335.15j1.16.0.msedr...0...1ac.1.64.img..0.17.1334.aZKWyeTORI4
None of them are correct looking.
-
I just became aware that Argyll's printtarg can output chart files, that is it's probably really easy to make your own targets if you have a good inkjet printer and a spectrometer to measure it. I shall test and update the docs accordingly. I think an inkjet print will have at least as good spectral qualities as an IT8 target on photo paper.
Maybe I'll add a target generator later on as Argyll's targen is more tailored for printer and scanner targets.
-
I just became aware that Argyll's printtarg can output chart files, that is it's probably really easy to make your own targets if you have a good inkjet printer and a spectrometer to measure it. I shall test and update the docs accordingly. I think an inkjet print will have at least as good spectral qualities as an IT8 target on photo paper.
Maybe I'll add a target generator later on as Argyll's targen is more tailored for printer and scanner targets.
Would the use of the spectrometer and software determine whether certain colors in a custom target are beyond a display's ability to reproduce like say Pantone 313C (coated)? A Fuji Frontier drylab inkjet can reproduce it but currently there are no reference displays. No telling what other colors there are for creating a custom target that would fit nicely and improve current photographic workflow processes.
Has someone come up with a more robust color target with a specific selection of colors that can fit within both Raw capture and display gamut workflows that would result in the least "gamut crunching" color errors?
-
Has someone come up with a more robust color target with a specific selection of colors that can fit within both Raw capture and display gamut workflows that would result in the least "gamut crunching" color errors?
Hi Tim,
Gamut crunching only happens on profile conversions. It does not affect the creation of a camera profile IMHO, unless one seeks to make a dumbed-down target that only covers the Adobe RGB colorspace (at best). But such a profile that's based on such a limited target would not be able to reliably map colors outside the limited aRGB gamut.
It's much better to use a target with relevant colors that the camera is able to 'see', and covers it full gamut as far as relevant existing colors go, and sometimes only use part of its full gamut if the subject poses a limited challenge.
Cheers,
Bart
-
I just became aware that Argyll's printtarg can output chart files, that is it's probably really easy to make your own targets if you have a good inkjet printer and a spectrometer to measure it. I shall test and update the docs accordingly. I think an inkjet print will have at least as good spectral qualities as an IT8 target on photo paper.
Maybe I'll add a target generator later on as Argyll's targen is more tailored for printer and scanner targets.
There is little benefit of printing your own camera targets on common 4 colours inkjets. The number of paints used for each patch will be limited to that of the original 4.
-
Hi Tim,
Gamut crunching only happens on profile conversions. It does not affect the creation of a camera profile IMHO, unless one seeks to make a dumbed-down target that only covers the Adobe RGB colorspace (at best). But such a profile that's based on such a limited target would not be able to reliably map colors outside the limited aRGB gamut.
It's much better to use a target with relevant colors that the camera is able to 'see', and covers it full gamut as far as relevant existing colors go, and sometimes only use part of its full gamut if the subject poses a limited challenge.
Cheers,
Bart
How has it been determined what colors the camera is able to 'see'?
I'ld have to assume this optimized for digital sensor color target would have to comprise a certain gamut shape and size that would be tuned to the math that can map colors to the display. Not sure if the shape of the gamut is more important than the size. The 3D model of most inkjets show some colors go outside the AdobeRGB gamut where most others are well within.
Can distinction between subtle differences of colors be improved by using a custom target that's more tuned to the real color gamut shape that describes how a digital camera 'sees' color?
-
There is little benefit of printing your own camera targets on common 4 colours inkjets. The number of paints used for each patch will be limited to that of the original 4.
I think many like me has the higher end pigment inkjets, which have several more inks. I shall do a more thorough spectral comparison later.
-
I think many like me has the higher end pigment inkjets, which have several more inks. I shall do a more thorough spectral comparison later.
A quote from stephen stuff (https://stephenstuff.wordpress.com/2011/09/30/targets-for-camera-profiling/) to describe some of the problems.
Spectral problems
Camera profiling targets are commonly photographed in sunlight, which includes ultraviolet wavelengths (UV). Photographic papers can contain “fluorescent whitener additives” (or “optical brightening agents”) which makes these papers appear more blue. These papers and printed camera profiling targets can show colour shifts towards blue when there is ultraviolet in the light source.
Metamerism is another problem for photographic and printed targets. The processes are optimised to produce colour, usually from just three of four colourants, that look natural to the human eye but actually might be composed of quite different spectra. This is a problem if the camera spectral sensitivities are different to the human eye. ColorChecker targets are made using multiple different pigments, giving reflective spectra that are more representative of the real world.
In fact if the solution would be as easy as to print your own target with however good colour coverage you need then anyone would not have problem obtaining good profiles. Alas this is not what I see happening yet.
-
DCamProf's color separation diagram gives an idea what colors a camera can "see", most modern cameras can differentiate colors in most of the human gamut, well beyond limits of say AdobeRGB.
Profiles generally distort extreme colors though and may move them outside valid space so they get clipped to a nearby color. Extremely saturated colors is a narrow special ccase so few have problems with that.
If you really want to cover an extreme gamut with your profile, designing using SSF is the answer, which you can do with DCamProf.
Spectrometer data will indeed give the profiler full information of the color. Consumer spectrometers are a little limited in range, often cover only 400-700nm which can be a problem in some special cases.
-
A quote from stephen stuff (https://stephenstuff.wordpress.com/2011/09/30/targets-for-camera-profiling/) to describe some of the problems.
Already well-known and discussed in DCamProf's docs. I need more testing but initial results suggest that OBA-free paper and pigment inkjets do a good job rivaling what you find in cc24 or munsell book of colors. There are probably some problem colors though, need to look at the spectra of all patches to give a full report. The bad reputation comes from old print systems and basic consumer papers.
I'm currently travelling so I can't test that much.
-
How has it been determined what colors the camera is able to 'see'?
Isn't that the purpose of DCamProf, to allow encoding colors that the camera can see? Whether and how that translates into humanly visible colors is another exercise, and whether and how our output modalities reproduce them is yet another. If we cannot adequately encode the input, there is no hope for getting good output.
I'ld have to assume this optimized for digital sensor color target would have to comprise a certain gamut shape and size that would be tuned to the math that can map colors to the display. Not sure if the shape of the gamut is more important than the size. The 3D model of most inkjets show some colors go outside the AdobeRGB gamut where most others are well within.
Yes, that's why an inkjet printed target may not be the best basis. Remember the exercise that Bruce Lindbloom did to find the parameters for his BetaRGB colorspace (http://www.brucelindbloom.com/index.html?BetaRGB.html). It comprises many important colors that may need to be encoded, but as few more extreme ones as possible, to improve the quantization step precision.
Can distinction between subtle differences of colors be improved by using a custom target that's more tuned to the real color gamut shape of a digital camera 'sees' color?
Not all colorspace coordinates represent humanly visible colors, and they are thus by definition not 'colors'. What matters is that a camera can capture and distinguish between as many 'colors' as possible. This is not going to be perfectly possible due to the Luther-Ives condition (more here (http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4299059/)), so some form of perceptual mapping will ultimately be needed.
The goal of creating a camera profile is to get a solid basis for the conversions that are to follow. Accurate for important colors, smooth transition to intermediate colors, able to reduce metameric and color constancy issues. But that is more an input or scene referred profiling than the ACR centric output referred profiling that gives us hue twists and other trouble.
I prefer clean scene referred profiling, making use of an adequate but not overly large working space, and a perceptually based output processing, e.g. using a CIECAM like color appearance model for output. I am not sure if an inkjet print will be challenging enough for a camera sensor, although it does allow to produce some saturated Cyans, Yellows, and Magentas. Apparently those dyes or pigments are available in nature, so we should also be able to encode those.
Cheers,
Bart
-
Afaik only Argyll and now DCamProf allows custom targets, which is a cost saver if you already have good printing gear, plus that commercial targets are focused on ease of use, minimize risk of reflection and other lighting errors rather than maximizing profile performance. Custom targets are definitely worth investigating deeply.
Target design is only a small part of the problem though. Profiling is an unsolvable problem (infinite spectral variation etc), it's always a compromise and therefore just as much an art as a craft. Many (most?) people prefer "looks" rather than neutrally accurate too, so it all boils down to taste. The best design method/target for you may not be the best for me.
-
Already well-known and discussed in DCamProf's docs. I need more testing but initial results suggest that OBA-free paper and pigment inkjets do a good job rivaling what you find in cc24 or munsell book of colors. There are probably some problem colors though, need to look at the spectra of all patches to give a full report. The bad reputation comes from old print systems and basic consumer papers.
I'm currently travelling so I can't test that much.
I just measured a regular IT8 target from Wolf Faust (my is quite old, but spent most of the time in dark non humid no extreme temperatures storage) with i1pro2, here are the spectral files = https://app.box.com/s/o4gwrk0d5slw7y1g0nu4bhs1fevg775d that is a bottom level of what can be printed if you have access to proper printer/ink/paper, no ?
-
DCamProf only cares about human gamut currently, it's not good for IR cameras for example. You can probably do SSF design with IR but color must be related to an observer, and currently only human observers are covered. How a color we can't see should look is not so easy to define ;-)
Also note that DNG profiles unlike ICC by design clips the gamut to prophoto. DCamProf's native format is unbounded though so the upcoming ICC support will not be clipped.
The prophoto clipping can be an issue in scientific applications, but hardly for ordinary photography.
-
Afaik only Argyll and now DCamProf allows custom targets
ProfileMaker for example too - just supply .tiff and matching .cie
-
Thanks AlterEgo that it8 data will be great for reference. I do expect that a pigment inkjet target will be at least as relevant as that. However the criticism above also targets photographically printed targets like Faust's It8, and suggests special printing like used for cc24 is required for adequate results.
Worth investigating.
-
I just became aware that Argyll's printtarg can output chart files, that is it's probably really easy to make your own targets if you have a good inkjet printer and a spectrometer to measure it.
Note that input targets created using a printing process have their limitations compared to targets created using a wider range of pigments.
The chart ideally should represent the spectra of what you are trying to capture, and something created with 4 or so inks has limited coverage of possible metamers.
Those capturing something like original artwork have had good success in creating custom test charts composed of the types of pigments that they intend capturing.
-
Would the use of the spectrometer and software determine whether certain colors in a custom target are beyond a display's ability to reproduce like say Pantone 313C (coated)?
Perfectly possible. You need to be aware of what you are assuming with regard to white point adaptation though. (i.e. because reflective colors are measured standard to D50 illuminant, and most displays are D65).
Has someone come up with a more robust color target with a specific selection of colors that can fit within both Raw capture and display gamut workflows that would result in the least "gamut crunching" color errors?
Not following you there. The gamut of an input device test chart has nothing to do with how out of gamut colors get handled further down the chain. Limiting the input test chart merely makes the camera/scanner characterization less accurate in the part of the gamut you haven't tested. Colors beyond that will still be captured by the input device. All colors are within a cameras gamut if its exposure is properly set (i.e. a camera doesn't have a gamut in the same sense an output device has.)
-
How has it been determined what colors the camera is able to 'see'?
I'ld have to assume this optimized for digital sensor color target would have to comprise a certain gamut shape and size that would be tuned to the math that can map colors to the display.
A camera doesn't have a gamut. If one of the channels hits saturation, you reduce exposure. If everything is black, you increase exposure.
What a camera has instead is a spectral mismatch to a human observer. There are colors that are metamers to a camera that are not a match to a human observer, and vise versa. Unless it's spectral sensitivities are exactly the equivalent of the standard observer, a matrix or other color profile can only approximately convert a camera RGB into the human XYZ values, and can do nothing about metameric mismatches.
A consequence of this is that any particular profile will allow a camera to produce imaginary colors, as well as there being colors that the camera can never produce (i.e. the human spectral locus and profiled camera spectral locus will not match).
-
A camera doesn't have a gamut in the sense of a printer, as it's an input profile, but it does have limitations of what colors it can differentiate (and capture). This can be visualized with the DCamProf color separation diagram, here an example of a Canon 5Dmk2:
(http://www.ludd.ltu.se/~torger/img/ssf_csep.png)
The darker color the worse color separation (related to the standard observer), and black it can't separate at all, ie all local colors are registered as one and the same.
The camera's SSF rarely stretches out as wide as the CMF so you get limits on maximum red and maximum violet, and in the case of older cameras you don't have red/blue overlap so you have some metamerism, modern cameras have very little metamerism though although the SNR will vary of course. The SSF shapes differ greatly from the CMF so you get some minimas here and there seen as dark spots in the diagram.
The diagram doesn't involve a profile, just the SSFs. A modern camera can generally separate colors near or at the locus, but to get a match you need strongly non-linear correction (LUT) and the strong non-linearity can hurt performance of normal colors so it's generally not a good idea to try to make a profile with maximum gamut.
A profile optimized for a smaller gamut will generally cause chaos for extreme colors, even moving them into invalid positions so they are clipped away, visualized in this diagram:
(http://www.ludd.ltu.se/~torger/img/testprof2.png)
where you see short error vectors for normal colors and looong ones for extreme colors, some stretching outside the gamut. I'm sure many that has SSFs for their cameras will be tempted to make the "ultimate" profile which makes the camera perform in almost the whole human gamut but that's probably a mistake as it will most likely hurt performance of the normal colors that are 100% of the colors in 99% of the scenes you shoot.
-
Note that input targets created using a printing process have their limitations compared to targets created using a wider range of pigments.
The chart ideally should represent the spectra of what you are trying to capture, and something created with 4 or so inks has limited coverage of possible metamers.
Those capturing something like original artwork have had good success in creating custom test charts composed of the types of pigments that they intend capturing.
I'm very aware of this, however it seems to me than noone has really tested this to depth the past 10 years or so. Pigment inkjets today doesn't have 4 inks, they like six colors and 3 neutrals, and for each new ink generation improvements have been made in metamerism etc. I shall make a thorough spectral analysis when I get some time, I'm sure there are problem colors but I'm also sure today's pigment inkjets perform much better now than consumer printers did when the criticism towards printed test targets was formed, so I think there are reasons to re-consider this option.
Also special-printed test targets like a CC24 or munsell's book of colors has limitations. I have compared with spectral databases from skin and nature and real spectra of course show much larger variation than you can capture in a few patches. With DCamProf you can profile against those spectral databases directly using camera SSF, but due to the natural limitations of camera profiling it's not that sure it will be that much better. Likewise I don't think it's that sure that a special-printed target will be that much better than a pigment inkjet printed target, and the home-made target has the advantage that it can if you desire be made much more saturated (cover a larger chromaticity gamut) than typical commercial matte test targets do.
Apart from traditional real-world testing with your eyes, with DCamProf it's possible to make comparisons virtually, you can measure your home-made target with a spectrometer, and make a target from a spectral database, and use the spectra from a CC24 or other commercial spectra for one more, and then make profiles from SSFs for all those and compare how well they match various spectra. So there's good opportunity to make some serious testing. I'm still in for making some more features in the software though so I don't have that much time to do all those tests myself.
-
Here's attached a few patches from canon pigment lucia ink on oba-free baryta paper, CC24 spectra, some real reflectance spectra from nordic nature (leaves, flowers etc), and finally some patches from IT8 Faust spectra provided by AlterEgo.
For the Baryta it seems like we have a colorant gap around 470nm and 570nm, and it's less smooth than CC24.
I think these plots indeed does show that the CC24 does match natural spectral shapes better than printed spectra, but that the printed spectra is certainly not as bad as it could be. The printed spectra also show higher saturation colors than the CC24 and nature so it will therefore make steeper shapes. A better/easier comparison would be to try matching the CC24 colors with the printer and see how those shapes look, I may do that later when I have some time.
On a first look the IT8 spectra looks pretty good, smooth. Certainly not obvious that it would be worse than the spectral shapes provided by CC24. There's the rise of all spectra towards 720+nm, but I don't think that matters that much as the sensitivity of the observer is low there.
-
Just released 0.6.0 which now also can make ICC profiles in addition to DNG profiles (added make-icc, icc2json and json2icc commands). It's still limited for ICCs though as it can only make matrix-only ICCs (no LUT), and only for raw converters that feed linear data to the profile. That is no Capture One support yet, but RawTherapee and probably many other ICC-using raw converters out there should work.
-
Just released 0.6.0 which now also can make ICC profiles in addition to DNG profiles (added make-icc, icc2json and json2icc commands). It's still limited for ICCs though as it can only make matrix-only ICCs (no LUT), and only for raw converters that feed linear data to the profile. That is no Capture One support yet, but RawTherapee and probably many other ICC-using raw converters out there should work.
Hi Anders,
Thanks. You did intend to take it a bit slower, catch a breath, and recuperate from the exhausting climb sofar, but I'm glad you didn't ... ;)
Eagerly awaiting Capture One usable ICCs, but I'm patient (I know the next step is quite a bit more complex).
Cheers,
Bart
-
Eagerly awaiting Capture One usable ICCs
matrix profiles are perfectly usable in C1 unless you want to use color editor, which requires LUT to be present there... tiff itself can be output from C1 instead of making it with dcraw (or similar)
-
Hi everyone.
Interesting thread.
This may be of use to some.
Iain
-
matrix profiles are perfectly usable in C1 unless you want to use color editor, which requires LUT to be present there... tiff itself can be output from C1 instead of making it with dcraw (or similar)
Capture One makes non-linear tiffs which DCamProf doesn't handle yet. DCamProf's matrix ICC doesn't have any shaper curves so input must be linear. I'll fix that eventually :-)
-
Just released 0.6.0
to build with mingw I had to
1) procure arpa/inet.h - borrowed from matlab (as I have it already installed)
and
2) in profio.c
#include <pthread.h> - that fixes localtime_r ()
and borrowed the following code snippet from internet
static inline uint16_t
htons (x)
uint16_t x;
{
#if BYTE_ORDER == BIG_ENDIAN
return x;
#elif BYTE_ORDER == LITTLE_ENDIAN
return __bswap_16 (x);
#else
# error "What kind of system is this?"
#endif
}
#define ntohs htons
static inline uint32_t
htonl (x)
uint32_t x;
{
#if BYTE_ORDER == BIG_ENDIAN
return x;
#elif BYTE_ORDER == LITTLE_ENDIAN
return __bswap_32 (x);
#else
# error "What kind of system is this?"
#endif
}
#define ntohl htonl
-
DCamProf's matrix ICC doesn't have any shaper curves so input must be linear. I'll fix that eventually :-)
but you can quickly put pure gamma trc tags, 1.8 is working for C1... instead of calculating some curve
-
0.6.0 binaries for Windows build with mingw ( dcamprof.exe and libgomp_64-1.dll ) = https://app.box.com/s/6k6zw0suw3bkxrtqwmo6cl1pzrg2zp19
-
Thanks for windows info, shall update to make easier to compile again to next release.
I guess the ntohl/ntohs etc should work if you include <winsock2.h> instead of <arpa/inet.h>? I'll make an ifdef for that.
-
but you can quickly put pure gamma trc tags, 1.8 is working for C1... instead of calculating some curve
I think I need to look at the TIFFTAG_TRANSFERFUNCTION tag which since C1 6 (or 7?) is included in the exported TIFF. It's not a pure 1.8 curve.
-
Thanks for windows info, shall update to make easier to compile again to next release.
I guess the ntohl/ntohs etc should work if you include <winsock2.h>?
removing code snippet and replacing with <winsock2.h> results in linker errors
undefined reference to `__imp_ntohl'
undefined reference to `__imp_ntohs'
so you need to add some extra libraries then
-
I think I need to look at the TIFFTAG_TRANSFERFUNCTION tag which since C1 6 (or 7?) is included in the exported TIFF. It's not a pure 1.8 curve.
yes, that tag was mentioned somewhere here by some P1 employee, but gamma = 1.8 still works OK meanwhile - may be not for purists though...
-
removing code snippet results in linker errors
undefined reference to `__imp_ntohl'
undefined reference to `__imp_ntohs'
so you need to add some extra libraries then
Uhh... I'll see, maybe I'll remove the swap dependencies to next version it's easy for me to do.
-
I think I need to look at the TIFFTAG_TRANSFERFUNCTION tag which since C1 6 (or 7?) is included in the exported TIFF. It's not a pure 1.8 curve.
FWIW, attached is how the Green TRC of the default Capture One profile for the EOS 1DS Mark III profile looks. Definitely not a simple gamma, and the C parameter of a curve fit is closer to 1/1.942.... So maybe something like that can be found back in the TIFF's metadata tags if saved with Camera profile.
Cheers,
Bart
-
here is for example rawdigger/makeinputicc argyll frontend (so w/o tiff generated by C1 - albeit you can perfectly convert C1 tiff to DNG and feed to the same workflow, just don't WB samples when exporting to CGATS file and set black level = 0 in RD) with g1.8 vs (where is which profile ?) C1 v8.2 with Sony A7 "Standard" profile... I am not advocating g = ~1.8, I am just saying that meanwhile you can live with it.... PS: profile was built using printed IT8 target illuminated by halogen light not filtered up to D50 (I did not have filters handy) by the way...
(http://s21.postimg.org/5m04s2c05/C1_IT8_RDA.jpg)
-
removing code snippet and replacing with <winsock2.h> results in linker errors
undefined reference to `__imp_ntohl'
undefined reference to `__imp_ntohs'
so you need to add some extra libraries then
Winsock is a library, simple include does not make it. I guess a separate makefile for mingw is needed.
-
Winsock is a library, simple include does not make it. I guess a separate makefile for mingw is needed.
including the code to define the functions for mingw build works OK I guess....
-
Here's attached a few patches from canon pigment lucia ink on oba-free baryta paper, CC24 spectra, some real reflectance spectra from nordic nature (leaves, flowers etc), and finally some patches from IT8 Faust spectra provided by AlterEgo.
also QP202 spectral (with i1pro2 and with colormunki) = https://app.box.com/s/un6liz50nrkpn59aem0gnrg7395wbear
-
FWIW, attached is how the Green TRC of the default Capture One profile for the EOS 1DS Mark III profile looks. Definitely not a simple gamma, and the C parameter of a curve fit is closer to 1/1.942.... So maybe something like that can be found back in the TIFF's metadata tags if saved with Camera profile.
I don't think the TRC curves of the LUT profiles say that much, if I've understood things correctly they are not even used at all in terms of camera profiling. As far as I know if you have an A2B0 LUT the *TRC and *XYZ tags are ignored, as there is both input curves and matrix in the LUT A2B0 tag itself.
And in that case the input curve may only be there to make a mapping trick to get better integer precision (ICCv2 is an old format integer math unfortunately) and the LUT itself may cancel out that curve and apply something new, which you can't see without putting data through the profile. In other words, it's not possible to use an ICC viewer and that way figure out which pre-processing C1 does. I think the TIFFTAG_TRANSFERFUNCTION in the exported profiling TIF is the thing to look for, looking into that now.
Those TRC curves also show an inverted curve, like you say 1/1.942 when a matrix profile actually needs ~1.8 to linearize, so they really doesn't make any sense to me.
The ICCv2 standard show two types of input profiles 1) XYZ+TRC tags (matrix plus shaper curves), 2) A2B0 only. The mix of both 1&2 is some non-standard stuff by Capture One as far as I understand, which may be because they are used both as camera profiles and can also be attached to a TIFF file just like sRGB.icm or AdobeRGB.icm.
If any ICC guru knows how it all fits together, please let me know...
-
Hi Tim,
I tried to your advice, but this is a season to taste thing.
BTW, those mountain images were all with the Sonys. No P45+ image made it to the wall. I don't know why. I have just had it for something like 22 months, so that matters. Also, I didn't use it on the workshop, for different reasons.
I am going to look into your recommendations, they are much appreciated.
Best regards
Erik
From the excellent looking images I've seen of your work off the P45+, Erik, I've noticed they do come off as having a slight green (more like an ashen blueish green) bias with a light hand on saturation. Remember my suggestion of applying a luminance/saturation increase with Hue/Sat tool in Photoshop on your gallery exhibit image of the sunlit green knoll with the background mountains you linked to in the LuLa Coffee Corner forum?
For me adaptation working too long on a landscape with a magenta-ish blue sky (cobalt blue has a magenta element) requires constant retweaking of WB to get the right looking green. I do know from years of examining sunlit green plants that the sunlit portion of a non-waxy, midtone green leaf/grass does have a yellowish bias, but ACR/LR's exact hue that makes it look right requires very gradual adjusts due to the cyan portion of the green that can make shadows appear too cool which invokes the adaptive effect of seeing green highlights as warm or a thalo green.
To filter this cyan requires moving toward magenta which slightly warms up green highlights but with a much different hue. I would never let my custom DNG profile fix this because it made the greens look a slightly dull yellowish orange green. And I didn't like moving temp slider toward blue because it desaturates everything.
After a long WB edit with ACR4.4 profile I'ld walk away and come back to see I'ld made everything too green because I kept trying to get the cyan in the green tint slider to freshen up the green. So I went back to As Shot WB (returned tweaked 0 back to +6 to +10 toward magenta), selected the custom DNG profile, walked away, took a break, came back and it looked perfect. Hue changes applying the custom DNG profile is so subtle in warming up the greens that I never considered how much WB affects the overall perception of color cast. I think one has to consider color constancy in the mix of things rather than attributing it to metamerism.
I mean I had trouble getting the right green hue shooting Live Oak trees that I did a google image search and found WB and greens all over the map...
https://www.google.com/search?newwindow=1&hl=en&site=imghp&tbm=isch&source=hp&biw=1556&bih=941&q=live+oak+in+texas&oq=live+oak+in+texas&gs_l=img.3..0i8i30l2j0i24.3707.9093.0.9964.17.16.0.1.1.0.114.1335.15j1.16.0.msedr...0...1ac.1.64.img..0.17.1334.aZKWyeTORI4
None of them are correct looking.
-
And in that case the input curve may only be there to make a mapping trick to get better integer precision (ICCv2 is an old format integer math unfortunately)
Little to do with integer encoding (it's rarely an issue), but to reduce the impact of the limited resolution of cLUT tables. Practical cLUT tables sizes are limited by memory/disk space, net bandwidth & CPU speed.
The ICCv2 standard show two types of input profiles 1) XYZ+TRC tags (matrix plus shaper curves), 2) A2B0 only. The mix of both 1&2 is some non-standard stuff by Capture One as far as I understand, which may be because they are used both as camera profiles and can also be attached to a TIFF file just like sRGB.icm or AdobeRGB.icm.
It's a tag format, so you can add almost any tags you want, including both matrix profile and cLUT tags. To be legal though, the PCS needs to be XYZ if the matrix profile tags are included. I have seen illegal profiles that label PCS as L*a*b* for the cLUT and then includes matrix profile elements, but not every CMM will cope with such a Frankenstein.
-
Little to do with integer encoding (it's rarely an issue), but to reduce the impact of the limited resolution of cLUT tables. Practical cLUT tables sizes are limited by memory/disk space, net bandwidth & CPU speed.It's a tag format, so you can add almost any tags you want, including both matrix profile and cLUT tags. To be legal though, the PCS needs to be XYZ if the matrix profile tags are included. I have seen illegal profiles that label PCS as L*a*b* for the cLUT and then includes matrix profile elements, but not every CMM will cope with such a Frankenstein.
Thanks for the info. I'll have to figure out the cLUT mapping/resolution issue when I implement cLUT support. I think I have an idea of how to do it...
Anyway, that makes Phase One profiles illegal (what a surprise...), as they contain matrix profile tags plus lab PCS for the A2B0. LittleCMS seems to swallow it though as it can display TIFFs with these profiles embedded, and it looks right to my eyes. I don't know which tags it looks at in this displaying case, it would be interesting to know. I guess the A2B0 tag needs to be used somehow otherwise it would not display right.
I've noted that some of the newer profiles don't have TRC and XYZ tags though, but only the A2B0 tag.
(I've got my matrix profiles sort of working in Capture One now by parsing out the TIFFTAG_TRANSFERFUNCTION from their profiling tiffs and applying that as TRC shaper curve. Brightness/contrast looks right, but the colors are too saturated... I have so far no idea why.)
-
Attached a plot of the transfer functions as reported in the profiling tiff by Capture One. A pure gamma 1.8 curve is used as reference (yellow), and then the two curves "linear scientific" (almost gamma 1.8 ), and then "film standard".
This is for the P45+.
Note that linear scientific does not reach 1.0, which I assume is intended to clip away noise close to clipping.
-
Anders,
Thanks for the new version. It does not seem to parse v4 ICC profiles. Had a look at the code and it looks like you parsing the ICC file yourself - any reason why? The DCamProf already requires lcms2 and it has a full loader and acces to all profile internals already there (and handles v2 and v4).
-
Thanks for the new version. It does not seem to parse v4 ICC profiles. Had a look at the code and it looks like you parsing the ICC file yourself - any reason why? The DCamProf already requires lcms2 and it has a full loader and acces to all profile internals already there (and handles v2 and v4).
The reason why is because the ICC standard is very big and if I parse anything DCamProf needs to handle anything. It's much better to block things early so the software internals doesn't have to see things it doesn't understand. To actually generate ICC profiles I need to know exactly how they work, which tags and headers that are included etc. I have deliberately narrowed down DCamProf's ICC handling to only camera profiles. Getting into the huge LCMS2 ICC API, which is huge because it's supporting all aspects of ICC standard, is probably not much easier than making the parser for ICCv2 profiles, and when making the parser I learnt a lot about the format which I needed to know anyway. Only the LCMS2 header file is longer than the complete ICC parsing code in DCamProf. DCamProf's internal ICC representation is very scaled down, only contains the elements required for cameras, it's not a full generic ICC.
I don't love ICC that much that I want to spend lots of time to implement JSON conversion for printer profiles or handle and generate ICCv4 if no popular raw converter requires it :)
That different raw converters handle ICCs differently, and even throws around invalid ICCs (like Capture One) makes ICC a pain to work with. I just like to have as little pain as possible :)
(there's still some bugs in the icc2json and icc lut stuff which will be fixed to the next patch release.)
-
The reason why is because the ICC standard is very big and if I parse anything DCamProf needs to handle anything. It's much better to block things early so the software internals doesn't have to see things it doesn't understand. To actually generate ICC profiles I need to know exactly how they work, which tags and headers that are included etc. I have deliberately narrowed down DCamProf's ICC handling to only camera profiles. Getting into the huge LCMS2 ICC API, which is huge because it's supporting all aspects of ICC standard, is probably not much easier than making the parser for ICCv2 profiles, and when making the parser I learnt a lot about the format which I needed to know anyway. Only the LCMS2 header file is longer than the complete ICC parsing code in DCamProf. DCamProf's internal ICC representation is very scaled down, only contains the elements required for cameras, it's not a full generic ICC.
I don't love ICC that much that I want to spend lots of time to implement JSON conversion for printer profiles or handle and generate ICCv4 if no popular raw converter requires it :)
That different raw converters handle ICCs differently, and even throws around invalid ICCs (like Capture One) makes ICC a pain to work with. I just like to have as little pain as possible :)
The LCMS2 profile hanling API - that is the one that accesses tags and checks the version is very simple. So is creational one (have a look at dcp2icc sources it creates ICC profile there via LCMS calls and writes LUT to it). I was not suggesting to use it in other capacities or other profiles - so I am not really sure I understang the reasons to code the parsing and reading in full. The only API you need to access profile data is to open profile from a file and to get the tag data. The writing is a bit more since it involves calling various set methods for the header but still not the full CMS API.
-
The code in DCamProf ICC parsing has an error btw - the logic where it determines version of the profile is incorrect and valid V2 profile with version encoded as 0x22000000 is producing error stating that it is V4 profile. These lines
Which software has produced that profile? The version check should not be incorrect according to the standard document, the first byte is major, the second byte is minor + patch, ie 0x22000000 means ICCv22.0.0 not ICCv2.2.0. I'm grateful for help finding bugs, but think it's better to deal with typo bugs in private messages as most Lula readers are not programmers :)
-
Getting into the huge LCMS2 ICC API, which is huge because it's supporting all aspects of ICC standard, is probably not much easier than making the parser for ICCv2 profiles, and when making the parser I learnt a lot about the format which I needed to know anyway.
I really don't recommend cooking up your own (the Firefox guys did that, and made a pigs ear of it). It has taken some of us rather a long time to figure out some of the subtleties, best practices, and interoperability gotchas - far better to leverage Marti's experience and support.
-
I really don't recommend cooking up your own (the Firefox guys did that, and made a pigs ear of it). It has taken some of us rather a long time to figure out some of the subtleties, best practices, and interoperability gotchas - far better to leverage Marti's experience and support.
Making ICCs work for raw converters is full of that regardless, it's a whole different space. The most popular ICC-supporting raw converter, ie Capture One, uses illegal ICC profiles and does some pre-processing outside the ICC's intention. I can't just go for basic use cases and say Capture One is broken, sorry guys. Oh well, I could, but then I could just skip ICC support all-together. I've implemented support for various ICC profiles in RawTherapee and there are at least four specific incompatible types I've had to handle. I'm not going to support them all with DCamProf though.
I'm not into re-inventing the wheel, but it's ~500 lines of fairly trivial code with fixed scenarios, the hardest part is converting to from the various fixed point formats. I may actually switch back to LCMS2 later on (it is a good idea using existing code) and replace those 500 lines, but now this is the fastest way forward. The code is there. I first looked at using LCMS2 but I got very frustrated that there were no easy to find example code for the use cases I needed, made a quickslam hack with own code just to get going and learnt/reapeated a lot of important aspects of the format meanwhile.
LCMS2 will almost certainly be used when the test-profile command gets ICC support, ie for running colors through the profile.
I plan to support Capture One, excluding Leaf-style ICCs, and then any raw converter with basic linear pre-processing, possibly add some trivial thing on request. I won't do Nikon-style ICCs, I won't do v4 ICCs.
-
Attached a plot of the transfer functions as reported in the profiling tiff by Capture One. A pure gamma 1.8 curve is used as reference (yellow), and then the two curves "linear scientific" (almost gamma 1.8 ), and then "film standard".
interesting to compare that with curves data in .fcrv files from C1 itself
x/y coordinates [0.0...1.0] pairs are coded as 4 bytes x 2 containing an UINT (4 bytes, 32bit, little endian) which shall be divided by 4294967295 ( 0xFFFFFFFF ), for example 4 bytes = "0x86" "0xF2" "0x1A" "0x0A" = 169538182 and / 4294967295 = 0.03947368 = 0.039474
the content (partial) of .fcrv files
(http://1.bp.blogspot.com/-OrghB5FurJI/VRsm9EJuoxI/AAAAAAAAHtA/LZNa-W3HNV0/s1600/1.jpg)
(http://3.bp.blogspot.com/-AF97oeshlqI/VRswMIR0GTI/AAAAAAAAHtQ/SBIXi4sdXG4/s1600/2.jpg)
(http://3.bp.blogspot.com/-HofjW71Jmzc/VRs4CsghyQI/AAAAAAAAHtg/H9gPIr4L9W0/s1600/3.jpg)
-
Without checking I would guess that they are the same, seems logical. I want to reverse-engineer as little as possible of Capture One though, they don't deserve this attention for making it hard to make ICC profiles for them :).
interesting to compare that with curves data in .fcrv files from C1 itself
-
Without checking I would guess that they are the same, seems logical.
but they are not... linear and linear scientific are linear curves as coded in .fcrv files [0.0,0.0] and [1.0,1.0] - at least in recent cameras... so whatever your see is C1 code doing something with each curve when writing the tag
-
but they are not... linear and linear scientific are linear curves as coded in .fcrv files [0.0,0.0] and [1.0,1.0]... so whatever your see is C1 code doing something with each curve when writing the tag
Have you checked the P45+ curves? It was those I plotted. Maybe they are different per camera? I haven't checked but I know there are different sets of curves available depending on camera chosen.
-
Have you checked the P45+ curves? It was those I plotted. Maybe they are different per camera? I haven't checked but I know there are different sets of curves available depending on camera chosen.
true - may be that camera indeed has different curve - I was checking the very recent ones, with .fcrv = 128 bytes... it seems that older camera are not that linear :D
PhaseOneP45+-Linear Scientific.fcrv = 128 bytes and the content indicates 2 points for the curve, so it is linear there
PhaseOneP45+-Linear Response.fcrv = 9 points for the curve - not linear
for comparison
PhaseOneP65+-Linear Response.fcrv
PhaseOneP65+-Linear Scientific.fcrv
are both 128 bytes with 2 points for curves and the only one byte (content) difference, which apparently is a tag that tells C1 that if Linear Scientific then it shall not invent data to paint clipped highlights in a nice way
-
Thanks for the info Alterego, those things help in figuring out what's happening in the C1 ICC conversion pipeline.
-
Which software has produced that profile? The version check should not be incorrect according to the standard document, the first byte is major, the second byte is minor + patch, ie 0x22000000 means ICCv22.0.0 not ICCv2.2.0. I'm grateful for help finding bugs, but think it's better to deal with typo bugs in private messages as most Lula readers are not programmers :)
My fault - you right, and your code is correct. It is not my profile but some of the users on russian forums. I'll try to get more information about how it was made.
-
Anyway I agree that it's a good idea using LCMS2 for parsing, but I had my reasons not to and now I'm not going to change it this moment. I'm already on the next challenge :). Cleaning up code comes later (=usually meaning never ;) )
-
Anyway I agree that it's a good idea using LCMS2 for parsing, but I had my reasons not to and now I'm not going to change it this moment. I'm already on the next challenge :). Cleaning up code comes later (=usually meaning never ;) )
I can have a go at doing this but closer to the end of May ...
-
I think I have it working on C1 now, can hopefully release in the next few days.
The saturation issue was simply because the P45+ raw data looks different (more saturated) out from C1 than from Adobe DNG / DCraw / Anyone Else. I don't know why. Maybe C1 makes a pre-matrixing to reduce color space to get a smaller space for the ICC to work on, which probably is a good idea as the LUT has limited resolution.
It would be interesting to know what it is though, because without being able to revert it it won't be possible to make profiles for C1 directly from SSFs.
I haven't checked how it is with other cameras though, the P45+ could be an exception...
-
I think I have it working on C1 now, can hopefully release in the next few days.
The saturation issue was simply because the P45+ raw data looks different (more saturated) out from C1 than from Adobe DNG / DCraw / Anyone Else. I don't know why. Maybe C1 makes a pre-matrixing to reduce color space to get a smaller space for the ICC to work on, which probably is a good idea as the LUT has limited resolution.
It would be interesting to know what it is though, because without being able to revert it it won't be possible to make profiles for C1 directly from SSFs.
I haven't checked how it is with other cameras though, the P45+ could be an exception...
may be C1 sees its own back and simple does something extra hardcoded there... I mean more than they do for regular dSLR/dSLM/P&S
-
A few C1 findings:
- tifftag_transferfunction seems to be it concerning linearizing the profiling tiff data
- C1 linear is very similar to but not exactly the same as gamma 1.8
- some more preprocessing than the film curve is applied before ICC, sometimes extra clip range, and possibly some matrixing (P45+), more investigation can be done, not required for DCamProf though, unless someone wants to make C1 profiles from SSFs.
- C1's own profiles applies a slight S-curve also in the ICC Lab LUT (probed several profiles, always the same), ie linear curve won't be linear for C1 profiles, but generated profiles that are truly linear and apply the tifftag_transferfunction should be linear as far as I can see.
- The only way to see which curve C1's native profile applies so to run data through the LUT and see what you get. The curves embedded in the profile say nothing about the end result.
I have some issues with ultrasaturared blues in my P45+ test profile, may be a clipping issue.
-
is it possible for the sake of saving some (quite some) time during the (multiple !) experiments to augment "make-dcp" command with options to write 1) profile name and 2) baseline exposure offset instead of repeating dumping, editing those fields, compiling back ?
-
is it possible for the sake of saving some (quite some) time during the (multiple !) experiments to augment "make-dcp" command with options to write 1) profile name and 2) baseline exposure offset instead of repeating dumping, editing those fields, compiling back ?
I'll probably look into streamlining the workflow a bit later on and add more options when the program has matured a bit. Meanwhile you could make a script for that using your favourite script language :)
-
I'll probably look into streamlining the workflow a bit later on and add more options when the program has matured a bit. Meanwhile you could make a script for that using your favourite script language :)
that is certainly doable in many ways, yes - so willl do this for now... another question is - may be instructions for DCamProf can be updated that it is possible to use rawdigger (albeit profile edition is not free) to extract raw RGB values to CGATS and then use txt2ti3 to produce .ti3 from that and .cie instead of scanin route ( and you can always position the grid on target better manually )... plus scanin does not put spectral data in .ti3 by itself ... additional advantage of involving rawdigger ( www.rawdigger.com ) is to do flatfielding and txt2ti3 is to provide spectral data from .cie (if they are there of course) when making .ti3 in one step.
-
Thanks for the tip, I'll update the docs when/if I test the workflow myself.
scanin will include spectral data in the .ti3 if the .cie file contains it.
-
scanin will include spectral data in the .ti3 if the .cie file contains it.
does it ? it seems that it didn't... I just tried with .cie file that has XYZ, LAB, CH and spectral data and .ti3 only has
SAMPLE_ID XYZ_X XYZ_Y XYZ_Z RGB_R RGB_G RGB_B STDEV_R STDEV_G STDEV_B
but no spectral data in it
next I removed XYZ LAB CH data from .cie and left only spectral and scanin says
"D:\argyll\bin\scanin.EXE: Error - Input file 'qpnm.cie' doesn't contain field XYZ_X or LAB_L"
am I missing something then ?
-
does it ?
It probably does not understand the patchtool CGATS format, it the spectral data comes in Argyll's own format it will include it. See the provided cc24_ref.cie as an example.
-
It probably does not understand the patchtool CGATS format, it the spectral data comes in Argyll's own format it will include it. See the provided cc24_ref.cie as an example.
but txt2ti3 does understand the same format in the same file w/ no issues... so that's something that Graeme G. can comment - how come that scanin does not understand what txt2ti3 does for the purpose of creating the same ti3 file...
anyways will check cc24_ref.cie then, thank you for the note - it did not happen to me to think about that difference
-
I've just released 0.6.1, which can make ICC profiles with LUT, and also supports Capture One.
http://www.ludd.ltu.se/~torger/dcamprof.html
I also made a small adjustment to the native LUT such that whitepoint is always preserved.
I haven't tested it a whole lot, I hope some Capture One users will help out with some ICC testing :)
-
I've just released 0.6.1, which can make ICC profiles with LUT, and also supports Capture One.
http://www.ludd.ltu.se/~torger/dcamprof.html
I also made a small adjustment to the native LUT such that whitepoint is always preserved.
I haven't tested it a whole lot, I hope some Capture One users will help out with some ICC testing :)
ouch.... now to build libtiff - is there a place where people just share prebuilt static libraries :( ?
-
ouch.... now to build libtiff - is there a place where people just share prebuilt static libraries :( ?
RawTherapee uses libtiff and a bunch of other builds, pre-built stuff for windows is found at, not sure how libtiff is configured though.
http://www.rawtherapee.com/shared/builds/windows/dependencies_for_creating_builds/
Otherwise the compile.txt for RawTherapee can help when it comes building yourself:
https://code.google.com/p/rawtherapee/source/browse/COMPILE.txt
The Cygwin environment has libtiff there from the beginning so there it's just "make" to build it, but of course a pure Mingw build is nicer.
Libtiff is currently only needed for reading the TIFFTAG_TRANSFERFUNCTION that Capture One workflow needs, but in the future I may add other TIFF functionalities.
-
RawTherapee uses libtiff and a bunch of other builds, pre-built stuff for windows is found at, not sure how libtiff is configured though.
http://www.rawtherapee.com/shared/builds/windows/dependencies_for_creating_builds/
Otherwise the compile.txt for RawTherapee can help when it comes building yourself:
https://code.google.com/p/rawtherapee/source/browse/COMPILE.txt
The Cygwin environment has libtiff there from the beginning so there it's just "make" to build it, but of course a pure Mingw build is nicer.
Libtiff is currently only needed for reading the TIFFTAG_TRANSFERFUNCTION that Capture One workflow needs, but in the future I may add other TIFF functionalities.
thank you... I prefer to have just as few pieces as possible so I want to use mingw of course.
-
Some comments about ICC LUT; this really is a weak spot of ICC. The LUT maps from camera RAW RGB triplets to LabD50 triplets. A problem is that a very large space of the LUT will be meaningless values, a camera never registers 1,0,1 for example, you nearly always have some signal on all three channels.
Capture One seems to have worked around this a little by pre-matrixing to reduce the "raw" color space and thus make more use of the LUT space.
Another issue is that it always have to be 3D even if you have 2.5D data, which means that you have to have quite low resolution or else the ICC file will be huge, a 33x33x33 cube could have been a 190x190 2.5D square instead. With a suitable input curve the low resolution seems to work alright in practice though.
An ICC profile is typically applied after exposure correction (DCP HSM is applied before), and that's fortunate otherwise resolution could suffer if you push a file a lot.
If you bring up a DCamProf ICC in a viewer and look at the Lab gamut you will see a large "box", the reason is that the native profile have information that extends all the way up to +/-128 ICC clip point. Those extreme colors will never be used in a real scene though (that wasted LUT space I mentioned) so don't worry that it looks a little rough and boxy. Capture One's native profiles aren't that boxy, probably because they do some sort of gamut mapping in the profile. DCamProf won't do any gamut mapping and I don't plan to add that either as I don't think it's the job of the profile, if you have out of gamut colors just reduce saturation etc in the raw converter with your color tools until it fits.
-
0.6.1 build for Windows (mingw = dcamprof.exe + libgomp_64-1.dll) : https://app.box.com/s/h2lwvnzi96c9jkdzv6m7gs9drrsrdfdb
-
DCamProf-generated icc profile(s) do not work for C1 (v8.2, PC/Win8.1x64)... I used the same json that was technically OK for dcp profile in ACR 9 (I mean that ACR recognized, showed and I was able to select it in ACR UI) and C1 does not recognize or show it.. for simplicity I used just a pure matrix/g1 profile... also a regular tools like http://www.color.org/wxProfileDump.zip or http://www.color.org/ICCProfileInspector2_4.zip can't open it ???
CORRECTION - it does recognize them, I simply put in a wrong directory - however the tools do not open profiles still (probably some important tags are still missing ?)
CORRECTION 2 - not so simple... for example when I put a custom made (with argyll) icc profile for my camera in either "C:\Users\<user>\AppData\Local\CaptureOne\Color Profiles" or even directly in "C:\Windows\System32\spool\drivers\color" then C1 sees it, but not the profile I generate with dcamprof - it only can be seen when it goes to where OEM profiles are ("C:\Program Files\Phase One\Capture One 8\Color Profiles\DSLR")... which is certainly wrong, user supplied profiles for C1 shall go to "C:\Users\<user>\AppData\Local\CaptureOne\Color Profiles" location.
-
but txt2ti3 does understand the same format in the same file w/ no issues... so that's something that Graeme G. can comment - how come that scanin does not understand what txt2ti3 does for the purpose of creating the same ti3 file...
txt2ti3's whole purpose is to understand a foreign CGATS format and translate them into ArgyllCMS format. For every tool to have all that code in it to deal with extra multiple matching input files (early LOGO/Gretag/X-Rite tools kept device and measured values separate) and do the translation, as well as recognize and make sense of the variations in ArgyllCMS format would make the code totally unwieldy.
-
The LUT maps from camera RAW RGB triplets to LabD50 triplets. A problem is that a very large space of the LUT will be meaningless values, a camera never registers 1,0,1 for example, you nearly always have some signal on all three channels.
Yep - that's another reason for the per channel Luts - you can restrict the box size. If you are in XYZ PCS you can also use the matrix to better fit your box around the gamut.
Another issue is that it always have to be 3D even if you have 2.5D data, which means that you have to have quite low resolution or else the ICC file will be huge, a 33x33x33 cube could have been a 190x190 2.5D square instead.
I think ICCV4 allows different resolutions per channel.
-
Thanks for testing.
I have only tested C1 by installing the profile, ie it gets to c:\windows spool color directory like printer profiles, I thought it was like that you should do. It works for me then in C1 7. The profile should not miss any tags in terms of standard, but maybe C1 needs more tags if put in the directory you describe. I shall investigate. Meanwhile just install by right click => install.
I have only tested with C1 7, and Mac OS X default ICC tool (it's really nice), and iccexamin and Argyll's iccdump. Sounds strange that they wouldn't work in color.org tools, I shall investigate that too.
The DCamProf LUT profiles don't have the XYZ or TRC tags like most C1 native profiles have, whose function is unknown to me. If the XYZ and TRC are involved in pre-matrixing/pre-processing somehow it maybe won't work in their native profiles directory.
-
Tested a profile with the color.org tools, it worked. I get one warning in ProfileDump validator though - unknown CMM, but it should not warn on that, the standard clearly states 0x0 should be used if no specific CMM is preferred, and that is also what many other profiles (including C1) use.
The profile was generated by the Linux build though, maybe there was some issue with the windows code, there are some specifics there. Investigating...
Edit: Tested your windows compiled version and indeed it makes invalid profiles, investigating why...
Edit2: the htonl/nthol macro code I copied was bad, it assumes the compiler defines BYTE_ORDER LITTLE_ENDIAN etc, and if it doesn't the code is written such that big endian becomes default ie all words in the icc gets in the wrong order, it doesn't look like that but C preprocessing has some traps... will fix and release a patch soon.
Edit3: updatated code and uploaded, hasn't been out for many hours so didn't care to change patch level so it's still at http://www.ludd.ltu.se/~torger/files/dcamprof-0.6.1.tar.bz2
I don't know if mingw defines __BYTE_ORDER__ like gcc do, but if it doesn't you should get an error message not silent accept big endian like before... if you do get an error with Mingw please let me know what macros it uses for byte order identification.
-
Tested a profile with the color.org tools, it worked. I get one warning in ProfileDump validator though - unknown CMM, but it should not warn on that, the standard clearly states 0x0 should be used if no specific CMM is preferred, and that is also what many other profiles (including C1) use.
The profile was generated by the Linux build though, maybe there was some issue with the windows code, there are some specifics there. Investigating...
Edit: Tested your windows compiled version and indeed it makes invalid profiles, investigating why...
Edit2: the htonl/nthol macro code I copied was bad, it assumes the compiler defines BYTE_ORDER LITTLE_ENDIAN etc, and if it doesn't the code is written such that big endian becomes default ie all words in the icc gets in the wrong order, it doesn't look like that but C preprocessing has some traps... will fix and release a patch soon.
Edit3: updatated code and uploaded, hasn't been out for many hours so didn't care to change patch level so it's still at http://www.ludd.ltu.se/~torger/files/dcamprof-0.6.1.tar.bz2
I don't know if mingw defines __BYTE_ORDER__ like gcc do, but if it doesn't you should get an error message not silent accept big endian like before... if you do get an error with Mingw please let me know what macros it uses for byte order identification.
ok, I will redo my build then - thank you.
-
0.6.1 recompiled for Windows (mingw) = https://app.box.com/s/mcdk72i5hgrvbgpnrpb8pp8xs1n55bfh
-
lobby for RD v2.0, long overdue, to include some more complex processing when generating cgats in addition to simple gamma
A simple script or even a spreadsheet can take the linear CGATS or CSV and apply a curve of your liking. It can be done within a profiling engine as well. IMHO RawDigger proper is not the place where it should be done as the variety of needs is quite overwhelming.
-
Edit2: the htonl/nthol macro code I copied was bad, it assumes the compiler defines BYTE_ORDER LITTLE_ENDIAN etc, and if it doesn't the code is written such that big endian becomes default ie all words in the icc gets in the wrong order, it doesn't look like that but C preprocessing has some traps... will fix and release a patch soon.
Another reason to be using lcms ?
[ I personally prefer writing endian independent code, rather than relying on compile time endian detection and byte swapping. ]
-
A simple script or even a spreadsheet can take the linear CGATS or CSV and apply a curve of your liking. It can be done within a profiling engine as well. IMHO RawDigger proper is not the place where it should be done as the variety of needs is quite overwhelming.
certainly... but it is a commercial product and so customers as expected are wishing all kind of things, overwhelming or not :) ...
-
certainly... but it is a commercial product and so customers as expected are wishing all kind of things, overwhelming or not :) ...
I can't even start on what functions I want to be added to exposure meters. Nobody manufacturers listen ;)
That's not to say we are not willing to add features, but fulfilling some requests lead to more dissatisfied users, as they want the same but in yellow and with a grey strip ;)
-
Another reason to be using lcms ?
[ I personally prefer writing endian independent code, rather than relying on compile time endian detection and byte swapping. ]
I've written 13000 lines of codes in about two months in addition to having a full-time job, and now it's documented and freshly released to the public. If everyone's going to make points for each bug that is found this will be a very tiresome thread.
-
I've written 13000 lines of codes in about two months in addition to having a full-time job, and now it's documented and freshly released to the public. If everyone's going to make points for each bug that is found this will be a very tiresome thread.
As promised I will have a go at converting it to LCMS but in a few weeks time (I'll send you the code when it will work).
-
I've written 13000 lines of codes in about two months in addition to having a full-time job, and now it's documented and freshly released to the public. If everyone's going to make points for each bug that is found this will be a very tiresome thread.
I don't write code, don't know a lick about programming, but it will be a tiresome thread for me if I don't get to see results from all this hard work in the form of an A/B comparison photo showing the code works.
I look forward to seeing this. In a sense the anticipation of what's been accomplished in this discussion is keeping me from seeing this thread as tiresome.
-
I don't write code, don't know a lick about programming, but it will be a tiresome thread for me if I don't get to see results from all this hard work in the form of an A/B comparison photo showing the code works.
The results of DCamProf can hardly be judged in such a primitive way as A/B photo comparison. The code is also not producing any photos so not really surte what you expect as "showing the code works"
-
I've written 13000 lines of codes in about two months in addition to having a full-time job, and now it's documented and freshly released to the public. If everyone's going to make points for each bug that is found this will be a very tiresome thread.
Thanks for the huge effort and research that goes with such a project.
There will always be things to change/improve/hindsight/etc. Don't take it as criticism, but as a pat on the back because it's apparently perceived as worthwhile enough to care about and build on, and a add a bit more functionality e.g. for Capture One (although not originally intended). But I also understand that there are limits to the resources for such a labor of love, thanks.
Cheers,
Bart
-
As promised I will have a go at converting it to LCMS but in a few weeks time (I'll send you the code when it will work).
Thanks but you don't need to, your valuable time is probably better spent on some other project than rewriting code that now works (hopefully ;) ). It seems like some think that I've written custom code for a major part of the software which LCMS already does, but that's not the case. If I strip out the code LCMS could do the ICC I/O stuff is ~500 lines out of those ~13000. LCMS is already used for LUT color conversions, CIEDE2000 and other complex code. Even if reading/writing is converted to LCMS code I still need/want the internal simplified profio_icc format as it will complexify the code dealing with a generic do-it-all ICC in the internals so the difference will be very small. Just as little as if Argyll would switch to using LCMS for reading and writing profiles DcamProf won't automatically gain ICCv4 support or any other feature.
Another point is that although I'm very grateful for your will to contribute to my personal project, I rather keep the code close to my own chest for now it's like my own child ;), patches with bug fixes yes, rewriting the code, no.
-
Thanks for the huge effort and research that goes with such a project.
There will always be things to change/improve/hindsight/etc. Don't take it as criticism, but as a pat on the back because it's apparently perceived as worthwhile enough to care about and build on, and a add a bit more functionality e.g. for Capture One (although not originally intended). But I also understand that there are limits to the resources for such a labor of love, thanks.
Capture One support was originally intended, just not this soon :-).
-
Fair enough.
Even if reading/writing is converted to LCMS code I still need/want the internal simplified profio_icc format as it will complexify the code dealing with a generic do-it-all ICC in the internals so the difference will be very small. Just as little as if Argyll would switch to using LCMS for reading and writing profiles DcamProf won't automatically gain ICCv4
I don't think we are still on the same page here. LCMS2 usage is not about complexity dealing with all ICC stuff it is about not writing the code to parse ICC. Whether you use it to access more complex areas is up to you. Opening a profile, checking the version and getting a matrix out of it for example is done in 3 calls. Creating profile from scratch is also simple. I kept pointing to dcp2icc tool - here is a snippet from there that creates a complete LUT profile (the lut creation are done with callbacks to dcp2iccc routines so these are all LCMS2 calls):
int CreateICC(char *fname, Sampler *smp, char *name, char *model)
{
cmsHPROFILE h;
LPLUT clut;
h=cmsOpenProfileFromFile(fname,"w");
if (!h) {
printf("Error creating '%s'\n", fname);
return -1;
}
cmsSetDeviceClass(h, icSigInputClass);
cmsSetColorSpace(h, icSigRgbData);
cmsSetPCS(h, icSigXYZData);
cmsSetRenderingIntent(h, INTENT_PERCEPTUAL);
cmsAddTag(h, icSigProfileDescriptionTag, name);
cmsAddTag(h, icSigCopyrightTag, "(c) Nobody");
cmsAddTag(h, icSigDeviceModelDescTag, model);
clut=cmsAllocLUT();
cmsAlloc3DGrid(clut, 33, 3, 3);
if (!cmsSample3DGrid(clut, dcpTest1, smp, 0)) {
printf("Error creating CLUT\n");
}
if (!cmsSample3DGrid(clut, dcpTest2, smp, 0)) {
printf("Error creating CLUT\n");
}
if (!cmsSample3DGrid(clut, dcpSampler, smp, 0)) {
printf("Error creating CLUT\n");
}
if (!cmsAddTag(h,icSigAToB0Tag, clut)) {
printf("Error adding CLUT\n");
}
cmsFreeLUT(clut);
cmsCloseProfile(h);
}
-
I don't write code, don't know a lick about programming, but it will be a tiresome thread for me if I don't get to see results from all this hard work in the form of an A/B comparison photo showing the code works.
I look forward to seeing this. In a sense the anticipation of what's been accomplished in this discussion is keeping me from seeing this thread as tiresome.
There many things you can do A/B tests for, but each test will be quite narrow and quite much work to perform. You don't see much A/B images for Argyll either (although I actually have one on my Argyll print workflow page).
I will myself be busy with maintaining the code and fixing improving for some while and will unfortunately not spend much time for these types of tests, except for those I do myself during development but those are not really presentable directly. At some point I'll probably write an article more in-depth about profiling and then there can be these type of A/B examples, but that's some time away now.
I'll maybe drop in some picture at some point if I come up with something interesting and presentable. So far I mostly do things "oh yes this seems to work" sanity check types of things, plus plot a lot of things.
Some ideas to test:
* A/B test between Ligthroom/C1 standard and DCamProf ICC/DCP on some typical scenes
* A/B test between files generated with different type of targets, CC24 vs IT8
* A/B test SSF-profile vs target profile, see if for example skin tone spectral data helps or not
it's difficult to show everything in a picture though as it cannot contain all colors, so plots are useful too.
There are still to me many undiscovered aspects of profiling, and the previously existing tools did not have the features I needed to allow me to research them, DCamProf has that and hopefully I'll actually do the tests now when I can, but writing the code turned out to be interesting too :)
-
The results of DCamProf can hardly be judged in such a primitive way as A/B photo comparison. The code is also not producing any photos so not really surte what you expect as "showing the code works"
Works in regard to color management and how it profiles a camera's response recording color which requires we see this in a photo compared to other camera profiling packages that use different coding and color descriptors to define how a camera records color.
Does that make it more clear for you?
-
There many things you can do A/B tests for, but each test will be quite narrow and quite much work to perform. You don't see much A/B images for Argyll either (although I actually have one on my Argyll print workflow page).
I will myself be busy with maintaining the code and fixing improving for some while and will unfortunately not spend much time for these types of tests, except for those I do myself during development but those are not really presentable directly. At some point I'll probably write an article more in-depth about profiling and then there can be these type of A/B examples, but that's some time away now.
I'll maybe drop in some picture at some point if I come up with something interesting and presentable. So far I mostly do things "oh yes this seems to work" sanity check types of things, plus plot a lot of things.
Some ideas to test:
* A/B test between Ligthroom/C1 standard and DCamProf ICC/DCP on some typical scenes
* A/B test between files generated with different type of targets, CC24 vs IT8
* A/B test SSF-profile vs target profile, see if for example skin tone spectral data helps or not
it's difficult to show everything in a picture though as it cannot contain all colors, so plots are useful too.
There are still to me many undiscovered aspects of profiling, and the previously existing tools did not have the features I needed to allow me to research them, DCamProf has that and hopefully I'll actually do the tests now when I can, but writing the code turned out to be interesting too :)
I'm getting the sense there will be no photo comparisons. Just say it so I can move on.
-
I'm getting the sense there will be no photo comparisons. Just say it so I can move on.
You can do them yourself too you know ;). Erik showed some patchtool dumps a while back. I don't know what other users will do, I will myself prioritize the todo list on software features rather than making demonstrations though. I'd love to do both but I think the early adopters prefer I focus on filling in the gaps.
Next up is ICC test-profile and plotting capability to wrap up the ICC support. I'm also thinking of adjusting the whitepoint preservation handling, seems like there's often problems with targets that have a scale of neutrals, which give contradicting corrections (possibly due to precision issues in the target reference files) and this can lead to some local stretching badness of the LUT when whitepoint is preserved. May need to do some special case handling for that.
-
If everyone's going to make points for each bug that is found this will be a very tiresome thread.
I'm not sure what you mean by "make points for each bug" -
my comment was intended just as food for thought - something to contemplate as you consider your longer term development plans.
-
I'm not sure what you mean by "make points for each bug" -
my comment was intended just as food for thought - something to contemplate as you consider your longer term development plans.
Fine, I probably overreacted, no problem. There's been a bit too much focus on a thing that I don't think matter. DCP and DNG ref code is custom too, but I better shut up about that... Oops :-)
-
Fine, I probably overreacted, no problem. There's been a bit too much focus on a thing that I don't think matter. DCP and DNG ref code is custom too, but I better shut up about that... Oops :-)
DNG SDK is nice to have but it does not suite you for other reasons - it is C++ (and you are not using that) and hard to compile (too many dependencies that one does not need).
-
Just released v0.6.2, now test-profile supports ICC and you can make ICC plots, that wraps up the ICC support which is now at the same level as DCP. In the usual place: http://www.ludd.luth.se/~torger/dcamprof.html
-
0.6.2 build for Windows (mingw = dcamprof.exe + libgomp_64-1.dll) : https://app.box.com/s/t851von5samj5tq8hzyw5qtontd8z3zo
-
Thanks to both Anders and 'AlterEgo' for the updated code and binaries.
Cheers,
Bart
-
Noticed I broke some of the old plot code in 0.6.2 when ICC plots were added. Will fix that and make some whitepoint handling improvements to next patch release.
-
Just released v0.6.3, fixed the plot bug, added two more error vector plots, better whitepoint preservation handling of the LUT.
Now I don't have any particular urgent things on the todo list left, so hopefully I can keep it in just-fixing-bugs mode for a while.
-
0.6.3 build for Windows (mingw = dcamprof.exe + libgomp_64-1.dll) : https://app.box.com/s/gtli840m59r5yd1yi9bzjgaub1bo4gdf
-
Slow day at work so I've done some testing of matrix optimization, found a bug in the overall printed stats which is fixed in 0.6.4 now out. I also adjusted the matrix optimization very slightly and updated the docs on weighting a bit.
In practice weighting does not seem to have that big impact on low patch targets like the CC24, weighting results can for those appear a bit random, eg if you reduce importance on say lightness it may sometimes actually improve lightness accuracy rather than the other way around. For those the "weighting" is sort of already done from the choice of patches, by excluding colors you get better precision on the colors you include.
Weighting behaves more predictably on larger patch count targets with higher saturation patches like a glossy IT-8 target.
One interesting result I found is that it's often a good idea to reduce importance of chroma (saturation), this will naturally push the matrix towards lower saturation as it gives more room to fine-tune hue.
-
0.6.4 build for Windows (mingw = dcamprof.exe + libgomp_64-1.dll) : https://app.box.com/s/bervrhj671j70l7e2fuca9q9ug9sw9zp
-
There is some problem with the LUT causing some purple colors to be pulled towards deep blue. I'm investigating. Sounds like kind of the reverse of "blue turns purple" lab problem, but there should be no such stuff... but well, it's something weird going on. Matrix profiles seems unaffected.
I've also noted that DCamProf produces saturated profiles compared to typical bundled profiles. I'm not sure why. If saturation is correct is hard to verify by eye, but it seems like DCamProf could be the more correct, while bundled profiles are often desaturated to not become too saturated when a film-curve is applied. I haven't thought about this before.
Maybe I'll have to add a feature that makes it possible to adapt the profile to a certain film-curve. You can't get perfect accuracy with that as it will depend on where on the S-curve the color is how saturated it will be, but I guess one would adapt assuming the color is in the midtone range.
Has anyone any input on this, that is how profiles should relate to the film curve? Today DCamProf makes "corrrect" profiles for linear curve, which means saturation will increase to higher levels than real-world when an S-curve is applied.
-
There is some problem with the LUT causing some purple colors to be pulled towards deep blue. I'm investigating. Sounds like kind of the reverse of "blue turns purple" lab problem, but there should be no such stuff... but well, it's something weird going on. Matrix profiles seems unaffected.
There was no LUT problem, what happened was that I got some extreme stretching in the LUT, and then bad things can happen if the white balance is set to a bit different from what the profile was designed for. When white balance is changed, the input colors to the LUT shift positions and if you have some extreme local stretches it can suddenly get into the position of a neighboring color and get stretched away.
I'm experimenting with high saturation custom targets when I got this problem. I'm still a bit surprised that the LUT had to stretch as much to hit the colors, I'm getting a quite different result when using SSF on the same target, so I still have some things to investigate.
Anyway, moving the white balance back and forth in the raw converter while looking how the color change is a good "sanity check" of a LUT profile. Preferably use a shot with many colors, a color checker of some sort. If one or more colors suddenly changes a lot faster than others you probably have too strong stretch in the LUT. Of course you can also plot the LUT to see these problems.
-
There is some problem with the LUT causing some purple colors to be pulled towards deep blue. I'm investigating. Sounds like kind of the reverse of "blue turns purple" lab problem, but there should be no such stuff... but well, it's something weird going on. Matrix profiles seems unaffected.
I've also noted that DCamProf produces saturated profiles compared to typical bundled profiles. I'm not sure why. If saturation is correct is hard to verify by eye, but it seems like DCamProf could be the more correct, while bundled profiles are often desaturated to not become too saturated when a film-curve is applied. I haven't thought about this before.
Maybe I'll have to add a feature that makes it possible to adapt the profile to a certain film-curve. You can't get perfect accuracy with that as it will depend on where on the S-curve the color is how saturated it will be, but I guess one would adapt assuming the color is in the midtone range.
Has anyone any input on this, that is how profiles should relate to the film curve? Today DCamProf makes "corrrect" profiles for linear curve, which means saturation will increase to higher levels than real-world when an S-curve is applied.
Since you're describing visually what DCamProf does to saturation and hue shifting in purples toward blue, have you considered constructs within the display profile as influences?
Haven't seen this mentioned here.
Also your point about uneven saturation levels across a wide range of colors as a sanity check also happens with my custom DNG profiles on some scenes. One slight tweak to WB and things shift unexpectedly saturation and luminance wise on certain colors which is why I've questioned ICC display profile influences.
I've often thought there needs to be more accurate descriptors of saturation appearance connected to the math under the hood because your description indicates a pattern of behavior with certain colors that suggest complexities that haven't been defined thoroughly mathematically. Maybe there needs to be a mathematical formula that takes into account human adaptation on saturation appearance separate from WB influences.
-
Screens with profiles, or factory builtin profiles (to emulate sRGB for example), can indeed sometimes have ugly nonlinearities that can make images appear to have problems when there are none, I've seen that. However I move between several computers, some without screen profiles, and I also know the profile quite well of my own screen (shaper profile designed with Argyll) so there was no such problem here.
The problem was as described in the last post, due to that there was a poor fit the LUT had to stretch a lot, and then the results become very white-balance dependent. If the white balance is not exactly as the profile expects the LUT can drag the wrong colors which happened in this case.
The next problem is why there was a poor fit and I'm still investigating that. I think there was some measurement problem, either in my XYZ reference measurements or in the test target photo, or both. I'm starting to suspect that high saturation patches as I have on my test target may not be a good idea when you shoot a target as it seems like the impact of small measurement errors becomes large, while lower saturation targets are much more robust. This is still just a theory though which I need to verify.
-
If this is too far off topic, my apologies…
Seems like an opportunity for someone here: create a good DNG profiling application, put a Mac interface on it (please!), allow for the user to load custom measurements from his/her Color Checker or CC Passport target, make it robust enough to handle larger-size DNG's (Pentax 645z). I would pay good $$ for this, and I suspect many others would, too.
Seems like there would be no competition for such a product - X-rite's Passport software is not only showing its age, but apparently will not receive any further development (users over on Pentax user forum claim to have been told this directly by X-rite staff).
John
JWL Images
Emeryville, CA
-
If this is too far off topic, my apologies…
Seems like an opportunity for someone here: create a good DNG profiling application, put a Mac interface on it (please!), allow for the user to load custom measurements from his/her Color Checker or CC Passport target, make it robust enough to handle larger-size DNG's (Pentax 645z). I would pay good $$ for this, and I suspect many others would, too.
Seems like there would be no competition for such a product - X-rite's Passport software is not only showing its age, but apparently will not receive any further development (users over on Pentax user forum claim to have been told this directly by X-rite staff).
John
JWL Images
Emeryville, CA
and why kind of UI you are missing exactly - because everything else is already there and works (on Mac too) ?
-
If this is too far off topic, my apologies…
Seems like an opportunity for someone here: create a good DNG profiling application, put a Mac interface on it (please!), allow for the user to load custom measurements from his/her Color Checker or CC Passport target, make it robust enough to handle larger-size DNG's (Pentax 645z). I would pay good $$ for this, and I suspect many others would, too.
Seems like there would be no competition for such a product - X-rite's Passport software is not only showing its age, but apparently will not receive any further development (users over on Pentax user forum claim to have been told this directly by X-rite staff).
It's probably not a bad idea, in a distant future (1 year plus) I might make a commercial GUI version of DCamProf (keeping command line open source), with the added feature to make custom hand-tuning to create "looks" (for that I think a GUI is really needed). I personally don't like having the look in the profile but the majority of users do, so to make a commercial I think that is a must-have feature. Still it would not be a huge seller, there's a reason X-rite staff is not developing their product further, there's more money to make on other products. I think most are using Adobe DNG Editor and are satisfied with that.
I also think one problem with commercializing a product is that making camera profiles beyond the very basics is relatively hard. Making a camera profile with a look requires a good eye for color and taste, and quite some technical understanding of camera limitations too. Looking at X-rite's products they are very focused on ease-of-use, so one reason they're not developing their profiling product further may be that it would be a "too complicated" product, so maybe it's not only about money-making potential...
-
here's more money to make on other products.
I read just recently somewhere - poster alleged that he was told by a big B&M photo shop that they sell to the tune of 10Ks xrite passports annually (???), so indeed if that is the case
-
Making a camera profile with a look requires a good eye for color and taste
so the business is to sell canned looks actually - a lot of companies do (even in non profile form)
-
there's a reason X-rite staff is not developing their product further
they killed profilemaker !
-
It's probably not a bad idea, in a distant future (1 year plus) I might make a commercial GUI version of DCamProf (keeping command line open source), with the added feature to make custom hand-tuning to create "looks" (for that I think a GUI is really needed).
-- Yes, I follow your reasoning here, but I too am not fond of this/would have no use for "looks" like that - kind of like all the Lightroom "presets" people are trying to sell. Yes, it's a one-button procedure, but the results generally suck, IMHO.
Yes, X-rite killed ProfileMaker, their software generally leaves a lot to be desired - I wonder if this isn't just their longtime corporate culture reasserting itself. Seems like they were always thought of as an outfit that made good hardware/not-so-good software. Once Gretag-MacBeth got gobbled up, maybe it was only a matter of time before that company's software skills (developer/engineers?) were gone.
Still I would dearly love to see a decent DNG-profile generating app, that could accept custom measurements & deal with larger DNG files. Something to help me render colors predictably, hopefully relatively "accurately" - all that business of simulating a Kodachrome look, or whatever, can easily be handled in Photoshop.
Just a thought, anyway…
Thanks for your efforts.
John
-
Still I would dearly love to see a decent DNG-profile generating app, that could accept custom measurements & deal with larger DNG files.
why "would" - what prevents you from using dcamprof right now this moment ?
-
If you want neutral/"accurate" profiles DCamProf should indeed do the job. I may need to make some contrast/curve/desaturation support though to work better with film curves to give you full satisfaction, but you can start testing and playing around today. DCamProf is today very "colorimetric", accurate results with linear gamma, which may lead to a bit too saturated look when a typical film curve is applied. I shall look into that later on.
Currently I'm experimenting with inkjet printed targets; I've made some quick comparisons with real natural spectra and real skintone spectra and a homemade printed target seem to do as good or slightly better than a CC24, a Colorchecker Digital SG is slightly better on skintones (it has many patches for that) but not on nature, the skintone matching might be compensated if I actally increase density in the skintone range. The spectral shapes looks a bit better from the X-rite colorcheckers if you plot them, but in practice it doesn't seem to make that big a difference.
Grayscale patches are no good from the inkjet, black patch is good and white (=unprinted) is good, but grays don't have completely flat spectra so no good for curve calibration, my current assumption is that lens does not alter the linearity of the camera that much that curve calibration would be necessary.
With the printer on semigloss baryta paper (make sure to use OBA-free) you can make supersaturated targets, but these seem difficult to match, high saturation colors are difficult. A matte target is much more user-friendly. Matte targets can't make colors in the dark violet range though.
-
what prevents you from using dcamprof right now this moment ?
-- sad to admit it, one phrase: "command line" (same thing that has kept me away from Argyll). With all the apps I need to stay current with, my brain is too full already. Just too hard to keep adding more & more things to learn & master, I guess.
John
-
-- sad to admit it, one phrase: "command line" (same thing that has kept me away from Argyll). With all the apps I need to stay current with, my brain is too full already. Just too hard to keep adding more & more things to learn & master, I guess.
John
you can prepare .ti3 files with rawdigger profile edition - it is basically GUI to handle the target shot for your purposes... then it is something that you can just write once in a command file - 3 lines along the following lines, for example only, you can hardcode there all directories too :
txt2ti3 -v -i <cgats file from rawdigger> <target .cie file, with spectral data for example> <output goes to .ti3 filename>
dcamprof make-profile -i <illuminant> <use the file from the line above .ti3 filename> <output goes to profile.json>
dcamprof make-dcp -c "your camera model" -t <curve of choice> <from the line above - profile.json> <output.dcp>
not much in a very simple case !
PS: I also modified (not in published .exe build of course !) the code for myself (.exe that I am using privately) to be able to put the baseline exposure offset for my camera and camera profile name through command line to save the time.
-
I have now released v0.7.0, includes test chart generator and a flatfield correction command. Patch names are now kept so it's easier to track them in the reports and plots.
I've changed the default such that targets are auto-rebalanced so the whitest patch becomes perfect reference whitepont, this is because most users expect the profile's optimal white balance be exactly the same as when you color-pick the whitest patch on the target (which in reality is a tiny bit off-white). You can disable that behavior with the -B flag.
I've also updated the documentation on white balance stuff, test target shooting and design. I've noted that semi-gloss targets are tricky to shoot, but if you do it right good results is had. Doing it right means shooting indoors with controlled (halogen) light, it's virtually impossible to shoot in daylight without some glare destroying the linearity of the photo. I haven't checked but I suspect that matte target do have some glare issues too, so for the best results shooting indoors seems to be the way to go.
In the usual place: http://www.ludd.ltu.se/~torger/dcamprof.html
-
I have a question about .sp (illumination) files - it seems that you 10nm steps are not enough ? I am getting an error message when I tested one... and why 5nm steps in your "spectrum.json" example ? argyll spotread with -H option outputs 3.x nm steps, something like = 380.00000 383.333333 386.666667 390.000000
-
0.7.0 build for Windows (mingw = dcamprof.exe + libgomp_64-1.dll) : https://app.box.com/s/0dk90n0h4k171f0jr6ftxsmuupzpzl4b
-
I have a question about .sp (illumination) files - it seems that you 10nm steps are not enough ? I am getting an error message when I tested one... and why 5nm steps in your "spectrum.json" example ? argyll spotread with -H option outputs 3.x nm steps, something like = 380.00000 383.333333 386.666667 390.000000
DCamProf currently always resamples to 5nm, this is due to laziness and that I do a lot of merging of targets when working with SSFs, and you can only have one sample interval in a single .ti3. I'll probably fix that in an update so resampling is only made when necessary.
I use .sp files now and then and it works for me, but so far I've always used with smaller bands than 10nm, maybe there's a bug, I'll check.
-
A few things I've learnt about camera profiling so far:
* Modern cameras make quite good matching matrix-only in D50, I can certainly see that some will satisfy with matrix-only profiles
* In Tungsten light cameras sweat more, matching is hard.
* Glossy targets are very difficult to shoot due to patch glare, dark room with directed halogen spots seems to be the way to go.
* You can't really do magic to improve matrix matching, if it's a bad match it will be bad even if you remove 50% of the highest delta E patches, in other words you don't need that many patches to get the matrix behavior.
* Targets with many closely spaced patches, like my generated one, is at risk of making conflicting adjustments. Plotting, LUT weighting/relaxation is your friend.
* If you have SSF curves, like I have for my 5Dmk2, you will see that profiles generated from targets and those from SSF look highly similar -- a good sanity check to see that it works.
* One should start with a D50/D65 profile as the camera is better at matching there, if you get a poor match it's most likely due to a bad measurement. Tungsten profiles require more tradeoffs and will likely behave badly unless you apply some LUT relaxation etc.
-
A few things I've learnt about camera profiling so far:
a question :
for example I am building a simple matrix profile for ACR/LR -single illuminant, CM+FM+curve, nothing else... I have a target, measured (spectral data) + a light (H3200K) & filter (80A), measured (spectral data) = .sp
now my exercise ends with (a simple case) :
dcamprof make-profile -i <.sp file> <.ti3 file> <profile.json>
dcamprof make-dcp -t linear -L -c "SONY ILCE-7" <profile.json> <profile.dcp>
does -i <.sp file> in this case bring any benefit for ForwardMatrix really in addition to having a "better" ColorMatrix which provides more logically sane temperature/tint values in ACR/LR UI (slider values) - but ColorMatrix does not do anything with color transform so to say (only ForwardMatrix does, after WB of course), it certainly helps somewhat to a user with a "proper/right/you name it" starting point for WB corrections in UI ? <.ti3 file> has raw RGB data from a target shot and spectral data from target measurments... it seems to me that ForwardMatrix is the same no matter what illuminant I use, right ?... but shall it be so ?
-
a question :
for example I am building a simple matrix profile for ACR/LR -single illuminant, CM+FM+curve, nothing else... I have a target, measured (spectral data) + a light (H3200K) & filter (80A), measured (spectral data) = .sp
now my exercise ends with (a simple case) :
dcamprof make-profile -i <.sp file> <.ti3 file> <profile.json>
dcamprof make-dcp -t linear -L -c "SONY ILCE-7" <profile.json> <profile.dcp>
does -i <.sp file> in this case bring any benefit for ForwardMatrix really in addition to having a "better" ColorMatrix which provides more logically sane temperature/tint values in ACR/LR UI (slider values) - but ColorMatrix does not do anything with color transform so to say (only ForwardMatrix does, after WB of course), it certainly helps somewhat to a user with a "proper/right/you name it" starting point for WB corrections in UI ? <.ti3 file> has raw RGB data from a target shot and spectral data from target measurments... it seems to me that ForwardMatrix is the same no matter what illuminant I use, right ?... but shall it be so ?
Calibration illuminant has no effect on the ForwardMatrix. It will only affect the ColorMatrix and thus it's ability to predict the correlated color temperature, which is a task specific to the DNG color pipeline (ICC will only use the forward matrix and the calibration illuminant is 100% informational, it has no effect whatsoever).
If you have a single illuminant profile the DNG pipeline does not need to know which temperature the CM was designed for, as it has only one to choose from. When making dual illuminant you need to specify EXIF lightsource so the CM is tagged with a designed temperature. You can't choose any temperature, as the temp is just a table lookup on the EXIF lightsource. Run DCamProf without args to see a list of them.
Currently it's a bit messy, your workflow would be to make the profile with the spectrum (which causes the XYZ reference values to be regenerated from spectra), but the resulting DCP will then have "Other" as calibration illuminant which will work for a single illuminant profile but not for dual, so then you need to use dcp2json/json2dcp to replace that with whatever EXIF illuminant with the closest temperature. It doesn't matter if the light is called "fluorescent" or whatever, DNG only translates it to a CCT value, no spectral shape is taken into consideration.
If the measurement data is the same, the forward matrix should not change if you specify a different calibration illuminant, yes that is correct. The reason for this is that the forward matrix specifies conversion from white-balanced camera RGB to XYZ D50, that is always D50 regardless of calibration illuminant, that is the same way as ICC profiles do. The reason one always goes to D50 is that it's more well-defined to convert further, eg to RGB for your screen and printer.
That is, colormatrix is only about the profile's ability to estimate the CCT the chosen white balance corresponds to. How much of a value that is in the Lightroom user interface I guess is a personal thing. I don't use Lightroom much, but I guess you typically start off with "As shot" white balance, and the profile won't affect the WB multipliers provided by the camera, if the color matrix is off an incorrect temperature translation will show though. If you would apply a processing preset (or whatever it's called in lightroom) which has a temperature and a tint in it the profile will affect the result as Lightroom will calculated the multipliers by using the colormatrix.
When you make a dual-illuminant profile the derived CCT will be used to mix your two forward matrices, so if you have StdA 2850K and D65 6500K, and the color matrices says the white balance is 4675K (exactly inbetween) the average of both forward matrices will be used. So in that case the color matrix has some effect on the actual color conversion, but only then.
Well there's also those ancient DCP profiles which did not have any forward matrix, in that case the color matrix has a direct effect on color rendition of course, you can make such profiles with DCamProf but I don't recommend it. Forward matrices was introduced by Adobe, maybe because they realized that the ICC way was actually better... binding to D50 gives more robust color conversions.
EDIT: corrected some statements on illuminant temperatures to not confuse future readers.
-
the resulting DCP will then have "Other" has calibration illuminant which I don't think Lightroom will like (as it can't look up a temp from that), so then you need to use dcp2json/json2dcp to replace that with whatever EXIF illuminant with the closest temperature.
in fact ACR does just fine with "255" value in illumination tag... on one other forum I posted some simple experiment
Adobe's CMs (2 from from Adobe Standard dual illuminant) vs DCamProf CM/"255", single illuminant, generated with .sp for H3200K+80A filter :
Adobe + styrofoam + no 80A = 3050К/+2
Adobe + whibal + no 80A = 3050К/+2
DCamProf + styrofoam + no 80A = 3100К/-2
DCamProf + whibal + no 80A = 3100К/-2
AND
Adobe + styrofoam + 80A = 4950К/+20
Adobe + whibal + 80A = 4900K/+19
DCamProf + styrofoam + 80A = 5200/+21
DCamProf + whibal + 80A = 5150K/+19
quite close, not completely insane at least
-
If the measurement data is the same, the forward matrix should not change if you specify a different calibration illuminant, yes that is correct. The reason for this is that the forward matrix specifies conversion from white-balanced camera RGB to XYZ D50, that is always D50 regardless of calibration illuminant, that is the same way as ICC profiles do.
yes, _but_ ... I have raw rgb data from the actual target shot (from raw) and the actual target measurements, sprectral, from i1pro2 (which brings its own illumination) combined in .ti3 file ... now that I have the actual illuminant in .sp then I 'd assume that the data in .ti3 file shall be adjusted somewhat to account for that, no ? raw RGB data stays the same, because it is what it is - but now that we have the spectral data of the actual illumination during the shot that shall change how target measurements now are... unless I am missing something very simple... I mean why only CM change shall be enough to account for that ?
-
How much of a value that is in the Lightroom user interface I guess is a personal thing. I don't use Lightroom much, but I guess you typically start off with "As shot" white balance, and the profile won't affect the WB multipliers provided by the camera, if the color matrix is off an incorrect temperature translation will show though.
offtopic, but - I use ACR, my "as shot" WB is always UniWB, so I start with "auto" WB or I subjectively type something between 4500-6500 with zero tint based on whether I want it colder/warmer in general...
-
in fact ACR does just fine with "255" value in illumination tag... on one other forum I posted some simple experiment
Adobe's CMs (2 from from Adobe Standard dual illuminant) vs DCamProf CM/"255", single illuminant, generated with .sp for H3200K+80A filter :
Adobe + styrofoam + no 80A = 3050К/+2
Adobe + whibal + no 80A = 3050К/+2
DCamProf + styrofoam + no 80A = 3100К/-2
DCamProf + whibal + no 80A = 3100К/-2
AND
Adobe + styrofoam + 80A = 4950К/+20
Adobe + whibal + 80A = 4900K/+19
DCamProf + styrofoam + 80A = 5200/+21
DCamProf + whibal + 80A = 5150K/+19
quite close, not completely insane at least
Thanks for this, I had forgot, I needed to look into dngref.c to remember.
lsOther is fine as long as it's not dual-illuminant.
ACR will simply feed the white balance to the (inverted) color matrix so it gets XYZ and converts to a xy chromaticity coordinate which then is through a standard algorithm converted to a blackbody temp and tint. It doesn't need to know which temperature the color matrix was designed at, but it will make more accurate temperatures if the tested white balance is close to the designed.
When dual illuminant ACR needs to know which temperature each matrix was designed for, so it knows how much to mix in from each.
-
yes, _but_ ... I have raw rgb data from the actual target shot (from raw) and the actual target measurements, sprectral, from i1pro2 (which brings its own illumination) combined in .ti3 file ... now that I have the actual illuminant in .sp then I 'd assume that the data in .ti3 file shall be adjusted somewhat to account for that, no ? raw RGB data stays the same, because it is what it is - but now that we have the spectral data of the actual illumination during the shot that shall change how target measurements now are... unless I am missing something very simple... I mean why only CM change shall be enough to account for that ?
The CM will need XYZ reference values for your calibration illuminant, so those will be recalculated from the reflectance spectra and your provided illuminant spectrum.
The FM will need XYZ reference values for D50 so those will not change, they're always calculated for D50, so the calibration illuminant won't matter. This may seems strange like it would be always the same FM regardless light, but don't forget that the camera RGB values are affected by the light you use during test target shooting, and that affects the FM. The task for the FM is convert RGB values that have been shot under the calibration illuminant and then white balanced. But DCamProf does not need to know which light you used for that, the RGB values are already there. And the target space is always XYZ D50 for the FM.
Only if you provide camera SSF curves so the RGB values can be recalculated from scratch the provided calibration illuminant will change the FM result.
Mathematically speaking it can be seen as a problem that FM always makes XYZD50 values, if you shoot under Tungsten wouldn't we want the tristimulus values for Tungsten? Well, the problem is that to convert to RGB or any other space we need to go through D50 (profile connection space) so there would still be a conversion to D50, using some sort of CAT. Then it's better if we make the conversion when designing the profile with full knowledge of spectra.
So a profile will correct the colors to make them look as if it was shot under D50, regardless of actual light used during shooting. For extreme light temperatures the eye will not fully adapt and colors will not appear the same as in D50, so the profile won't do the "correct" job for those temperatures. Is there a way to convert say 2000K color appearance but into D50 XYZ coordinates? I don't really know, maybe CIECAM02 can do that. It can be worth investigating. I suspect that it won't be feasible though, on cannot put on a slight red tint for example as the DNG pipeline requires the FM to map to neutral white. So for now it's up to the photographer's eye and raw converter sliders to simulate any changes in look due to extreme color temperatures.
-
OK, after reading the notes here and there I am coming back to peace of mind then... at least I have lesser work with WB :), so that my exercise with spectrum measurments were not totally wasted
-
OK, after reading the notes here and there I am coming back to peace of mind then... at least I have lesser work with WB :), so that my exercise with spectrum measurments were not totally wasted
Yes providing the spectrum gives the best XYZ reference values when the CM is derived, so it will be as good as it can be at estimating wb temperatures. Of course it will be best at estimating temperatures close to the actual temperature used when profiling, but any reasonable modern camera should have similar matrices over at least some temperature range so it should work for quite some range. I haven't made any specific investigation on how well it works though.
If one shoot under say tungsten but provide XYZ reference values for D50 the CM will certainly be bad at wb temperature guessing, so if one doesn't have a spectrometer so one cannot provide the spectra it's preferable to be reasonable good at guessing the temperature and provide a EXIF illuminant for that when making the profile. Using Adobe's own dual illuminant profiles and make a test shot can be used as an alternative to get some idea if unsure.
-
and some general question, I did not even look into that area, so if it sounds stupid never mind - but given the set of real world patches (through the supplied spectral measurements) data, can software produce a target that shall be a close to "ideal" (what is the "ideal" though ? the next question) one with the given (as a parameter) number of patches ? for example I give a task to software - find me the best target with 24 patches only from the given data for patches (for example I have 200 different patches @ my disposal) that I can use and software then gives me which 24 out of those 200 I shall use to achieve the best result (may be defined as some dE over a big virtual set of patches in your database like Munsell set, etc) ? some kind of automated finding of the best target withing given limitations (# of patches in the target, set of patches that I can only use, illuminant may be and virtual set of patches that I want to aim at in terms of good conversion with that target that software suggests)
-
In order to solve that problem the profiler needs to know the camera's SSF (as the solution will depend on that), but if you already know the camera's SSF you can profile directly against the real spectral data.
On a related subject, I have thought a little about estimating SSFs from a large set of printed patches, but I don't think meaningful precision can be had. There has been some academic work on SSF estimation, but as far as I know no method that reach the level of precision we would be interested of has been done. There is the method of using emissive targets with narrow-band filters, Image Engineering has such systems commercially, but I don't think those are feasible for enthusiast budgets.
-
In order to solve that problem the profiler needs to know the camera's SSF (as the solution will depend on that), but if you already know the camera's SSF you can profile directly against the real spectral data.
that's when I do not... but why it is not possible w/o camera's SSF ? we can imagine some average, idealized SSF at least ?
On a related subject, I have thought a little about estimating SSFs from a large set of printed patches, but I don't think meaningful precision can be had. There has been some academic work on SSF estimation, but as far as I know no method that reach the level of precision we would be interested of has been done.
I modified that matlab script for my camera and IT8 target, just as an exercise
(http://s7.postimg.org/3tyoasm6j/A7_SSF.jpg)
I will be redoing it with some additional measurements though, to see if it becomes better or not
-
So a profile will correct the colors to make them look as if it was shot under D50, regardless of actual light used during shooting.
Re-lighting and chromatic adaptation are different things though. I would have thought that by default a photographer would want to capture the appearance of things as they see them, in which case a chromatic adaptation is exactly the right thing to do to adapt to a different white point while maintaining the appearance.
-
Re-lighting and chromatic adaptation are different things though. I would have thought that by default a photographer would want to capture the appearance of things as they see them, in which case a chromatic adaptation is exactly the right thing to do to adapt to a different white point while maintaining the appearance.
Hmm... maybe I should change how DCamProf does it then. Assuming we have patch spectra and we make a StdA profile it currently does like this: from spectra calculate StdA XYZ values, which is used for CM (as said this is only about tempereture identification, not color correction), and from spectra calculate D50 XYZ values which the actual correction is made with, that is a re-lighting rather than CAT.
I have assumed that the CAT is rather approximative and tries to do the same thing as re-lighting does (as I've thought that the model assumes total adaptation, ie no appearance change), but makes a worse job at it so it is better to relight when possible (if DCamProf doesn't have the spectra it applies CAT02). It's a simple change to make it always do CAT instead, but I need to figure out if it's the right thing or not.
That is the alternative would be that the D50 reference values are always generated by CAT02, first you do the XYZ calibration illuminant ref values by spectra, and then CAT02 on that to get the D50 profile connection space ref values that the color correction works at.
When surfing around I've found papers like this: http://hrcak.srce.hr/file/95370 where the "performance of CATs" is tested, and what they do is that they calculate reference data by "relighting", that is new XYZ values from spectra, and then see how well a CAT matches that, and better match means better performance there. I think that indicated that the current way DCamProf does it is the right way, that is if you can relight using spectra do that rather than using CAT. However what you say indicates that it would be better to do the alternative described above, I'm not really sure what.
My understanding of CAT is that the simpler models try to achieve the same result as a relighting but does not succeed as well as doing an actual relighting, and the more advanced like CAT02 will do the same thing, *unless* you provide some viewing condition parameters, eg dim/dark environment which seems to be adaptations for television and projector viewing (that is how colors appears on screen depending on surrounding environment), I'm not sure if that can be transformed into anything meaningful for camera calibration.
So far I have assumed that relighting is the current best choice for camera profiles.
What do you think?
-
that's when I do not... but why it is not possible w/o camera's SSF ? we can imagine some average, idealized SSF at least ?
Yes, I guess we could. I'm not sure I understand what the purpose is though, could you clarify through a use case? Is it that you have shot a target with like 200 printed patches, and want to make a profile that matches some other spectral database as good as possible (say landscape spectra), and then pick out a subset of those 200 printed patches that would contribute the best to match that landscape spectra, and remove those that won't contribute?
If so, one method could be to just try to find as similar spectrum as possible, I have actually written a prototype of such a function, not for this task though but to compare different targets when I made my profiling target evaluation.
I'm not sure if that would improve the profile though, but one way could be to just expose that spectral matching function is DCamProf as a command so one can filter own .ti3 files and experiment and see what happens.
What can improve a matrix profile is removing colors that are too saturated for the targeted spectra, so the matrix optimizer gets a better chance to match the lower saturation colors.
-
Hmm... maybe I should change how DCamProf does it then. .... (as said this is only about tempereture identification, not color correction) ...
OK, perhaps I'm taking you out of context then. Accurately re-lighting a photographed scene would be no small feat - you'd need a spectral camera for a start, and some way of distinguishing between surface colors and emissive colors - such as photographing it under two different illuminations !
When surfing around I've found papers like this: http://hrcak.srce.hr/file/95370 where the "performance of CATs" is tested, and what they do is that they calculate reference data by "relighting", that is new XYZ values from spectra, and then see how well a CAT matches that, and better match means better performance there.
Well if you want to do re-lighting and all you have is colorimetric data, then doing a chromatic adaptation is an approximation of the effect you really want. And that seems to be all the above paper was exploring. If it was testing the CATs as appearance transforms, then they would have to have done psychovisual testing.
-
OK, perhaps I'm taking you out of context then. Accurately re-lighting a photographed scene would be no small feat - you'd need a spectral camera for a start, and some way of distinguishing between surface colors and emissive colors - such as photographing it under two different illuminations !Well if you want to do re-lighting and all you have is colorimetric data, then doing a chromatic adaptation is an approximation of the effect you really want. And that seems to be all the above paper was exploring. If it was testing the CATs as appearance transforms, then they would have to have done psychovisual testing.
I'm just trying to clarify what a camera profiler should do. We need to get into D50 space as that is the PCS (for both ICC and DCP). What we shoot is test targets, and we have the reflectance spectra, and we have the illuminant spectra. So we can do spectral calculations to get the XYZ values for any illuminant we want. The question that has come up now is if we really should.
I do know that for extreme color temperatures, like dusk in a nordic winter landscape, the white snow will not look white but bluish pink, which is an obvious case of where just white balancing won't work, but AFAIK there's also a small difference between StdA and D65, that is even if fully adapted and only an StdA light source we will see that the light is a bit yellow compared to the same situation under D65. Again, AFAIK color science is designed to work in a range of "reasonable" color temperatures and produce equal colors over those, so a camera profile is expected to be designed such that if you shoot an artwork under Tungsten with a Tungsten profile it will give you the exact same result as if you shoot an artwork under D50 with a D50 profile. This is what DCamProf strives for currently, and therefore it recalculates the XYZ reference values from spectra with the targeted illuminant whenever it can, instead of using CAT.
If however there is a way to model the slight (or large) color appearance changes caused by an illuminant much different from D50, and then convert to XYZ D50 I should at least provide that as an option to DCamProf. I have assumed that CAT is not the answer though, and that there simply is no model for that, just like there seems to be no model for converting between different observers.
-
I'm just trying to clarify what a camera profiler should do. We need to get into D50 space as that is the PCS (for both ICC and DCP). What we shoot is test targets, and we have the reflectance spectra, and we have the illuminant spectra. So we can do spectral calculations to get the XYZ values for any illuminant we want. The question that has come up now is if we really should.
It seems to me that there are two clear parts to this. The first is converting the camera RGB's to XYZ's - i,e. to make the camera image as colorimetric as possible. Having done that, you would then use a CAT to convert your chosen white point to D50 for interchange, since this preserves the appearance.
What you do in the first part is a different matter. If you are striving for the best possible colorimetric profile, then one approach is to key the conversion to various hints as to the original spectral composition of the colors. Obvious hints are the color itself, which you can translate to an assumed spectra of statistically likely surface colors, but the spectral content of the illuminant will affect this, as well as the white point of the illuminant shifting your whole color-to-assumed spectral mapping.
So one practical approach is to create several discrete profiles for different illuminant conditions - which is where you certainly would use spectral re-lighting of your test target - to create these discrete profiles.
Again, AFAIK color science is designed to work in a range of "reasonable" color temperatures and produce equal colors over those, so a camera profile is expected to be designed such that if you shoot an artwork under Tungsten with a Tungsten profile it will give you the exact same result as if you shoot an artwork under D50 with a D50 profile.
It's certainly reasonable that under some circumstances a photographer may want to shoot under incandescent and want the resulting image to look like they shot it under D50. But a different photographer or the same one under different circumstances may want the D50 white point image to present the same appearance as the original under incandescent. The difference is subtle, but its certainly there.
[ The difference may be extremely subtle indeed when forced through the eye of the RGB camera spectral sensitivities. ]
I have assumed that CAT is not the answer though, and that there simply is no model for that, just like there seems to be no model for converting between different observers.
No CAT is perfect, and if you arrive at a consistent set of errors in the current CATs, that can be formulated into a different more accurate and/or flexible CAT, then you certainly have material for a technical paper.
-
Thanks Graeme for the feedback, it's very valuable.
Let's see if I have understood you correctly;
If we would like the Tungsten profile to keep the subtle color appearance difference from a D50 light condition the best current way is to make XYZ reference values for tungsten using spectral data, and then convert to D50 XYZ using CAT02 (with average viewing condition as default) instead of ignoring the calibration illuminant and always generating our D50 XYZ values from spectra using D50 illuminant directly. Is this correct?
Prior to this my humble understanding was that a CAT, including CAT02 (with average viewing condition in/out), did not intend to model those subtle appearance differences but instead just tries to make a result as equal as possible to "relighting" (ie recalculate our D50 values from spectra in our case), and at the same time make some mathematical tradeoffs to make the transform reversible. That is a CAT result is just an approximation of the relighting result and any differences from actual relighting is not due to it tries to model slight appearance differences, but just simply limits in how precise the CAT can be. As far as I see you say that this understanding is incorrect, right?
-
Yes, I guess we could. I'm not sure I understand what the purpose is though, could you clarify through a use case?
ok, here is what I was thinking (may be not sanely enough though)
1) no SSF
2) desire to create a profile from some "target" (see below) that will give me the best color transform for some set of spectral reflectance of some things (like the databases of skin color, etc)
3) spectral data for patches that I can use to actually create a target either physically (cut, glue) or combining raw RGB & patch spectral data from shots & measurements of my existing targets (by removing unwanted data from those)
4) I want to limit myself to some reasonable amount of patches in the "target"
so can software be fed with
1) database of spectral reflectance of things - but I do not have patches for those
2) spectral data for patches that I actually have and so can shoot
3) size of the target in patches, for example 24 patches
4) illuminant spectral data
and output what patches exactly I shall select within the said limit (for example 24) to get the best profile for the said database ?
and then I either make the target to use physically or cut&paste the data from raw RGB & patch spectral data from shots of existing targets or combine both ways
-
a question
I tried to make icc profiles from SSF and used 2 different illuminant spectrums all the way for 2 profiles - one is just regular D50 (-i D50) and one is my measurements of the light I have when I am shooting targets (-i <.sp>) - which is halogen 3200K (nominal) lamp + hoya hmc 80a filter on lens
the difference though is very small (put a lot of RGB C1 color readouts - most are either identical or differ by 1 out of 255 in one or max two channels) between what C1 delivers with both profiles between them (between 2 variants one rendered with D50 profile and one with my illuminant profile - the picture itself is outdoors, ~40.44N latitude, ~4 PM EST, ~April) , is this right ?
this is my light - BLACK one :
(http://s12.postimg.org/6psxox2kt/image.jpg)
Type = File 'CM=SpotRead(A)=Direct+HoyaHMC80A=Average=2015-05-26.sp' spect 0 {Black}
XYZ = 0.945861 1.000000 0.808182, x,y = 0.343445 0.363102
D50 L*a*b* = 100.000000 -3.190716 1.360748
CMYV density = 1.898217 1.864757 1.943857 1.863647
CCT = 5096.603887, VCT = 4905.877034
CRI = 92.4
CIEDE2000 Delta E = 2.871270
Type = File 'CM=SpotRead(P)=StyroFoam+HoyaHMC80A=Average=2015-05-26.sp' spect 0 {Red}
XYZ = 0.949646 1.000000 0.805636, x,y = 0.344664 0.362939
D50 L*a*b* = 100.000000 -2.529025 1.569581
CMYV density = 1.897475 1.866200 1.946574 1.862948
CCT = 5052.127273, VCT = 4879.029968
CRI = 92.9
CIEDE2000 Delta E = 2.660874
-
The CCT for your halogen+filter is about the same for D50, and while the 80A filter adds some bumps and dips it's still a good illuminant so I think it's reasonable to expect that the results should be highly similar.
-
The CCT for your halogen+filter is about the same for D50, and while the 80A filter adds some bumps and dips it's still a good illuminant so I think it's reasonable to expect that the results should be highly similar.
somehow I was expecting that sufficiently different shape (from D50) shall manifest itself more visibly... even CCT is the same.
-
just an image, the difference between a pair of Adobe CMs from Adobe Standard and home-brewed dcp from IT8 printed target (which was actually done by generating SSF from a single raw, then SSF to dual illuminant DCP), the raw from I-R (Sony A7) :
(http://s13.postimg.org/52vex8jg7/delta_mid.jpg)
guessed SSF in question also attached :
-
somehow I was expecting that sufficiently different shape (from D50) shall manifest itself more visibly... even CCT is the same.
I depends on how it lines up with the reflectance spectra and then the SSF, if the bumps here and there in the spectrum doesn't change the integrated value much, then colors won't change much.
I'm also trying to simulate D50 and D65 with halogen plus filters, not really happy with the shape of the spectrum (the 80B filter is way more bumpy than I expected) but your results are encouraging. I'm most worried about that the 80B filter lets through high reds which I guess can cause the accuracy of super-saturated reds on a glossy target go bad.
-
just an image, the difference between a pair of Adobe CMs from Adobe Standard and home-brewed dcp from IT8 printed target (which was actually done by generating SSF from a single raw, then SSF to dual illuminant DCP), the raw from I-R (Sony A7) :
guessed SSF in question also attached :
Really interesting results! Seems like I should consider trying to integrate some SSF guessing function to DCamProf afterall.
-
I depends on how it lines up with the reflectance spectra and then the SSF, if the bumps here and there in the spectrum doesn't change the integrated value much, then colors won't change much.
I'm also trying to simulate D50 and D65 with halogen plus filters, not really happy with the shape of the spectrum (the 80B filter is way more bumpy than I expected) but your results are encouraging. I'm most worried about that the 80B filter lets through high reds which I guess can cause the accuracy of super-saturated reds on a glossy target go bad.
I just measured H3200K nominal halogen lamp ( Ushio JCV120V-1200WCH ) with Hoya HMC80A with i1pro2 (previously I did with ColorMunki)... so here they are (averaged over 10+ tries)
(http://s4.postimg.org/4dtbrf1el/measurements.jpg)
Type = File 'A=H3200K-Average.sp' spect 0 [ Black ]
XYZ = 1.069623 1.000000 0.408425, x,y = 0.431639 0.403543
D50 L*a*b* = 100.000000 17.595772 41.778263
CMYV density = 1.752698 1.938398 2.220658 1.834233
CCT = 3086.897709, VCT = 3070.559182
CRI = 99.4
CIEDE2000 Delta E = 0.774610
Type = File 'P=H3200K+HMC80A-Average.sp' spect 0 [ Red ]
XYZ = 0.946912 1.000000 0.787862, x,y = 0.346249 0.365661
D50 L*a*b* = 100.000000 -3.006881 3.039750
CMYV density = 1.894677 1.865917 1.954723 1.862778
CCT = 5003.925303, VCT = 4813.538610
CRI = 92.2
CIEDE2000 Delta E = 2.984494
-
I've sent a question to Lee Filters and asked if they have some color balancing filters that can reshape tungsten to D50 and D65 without that localized 560nm bump, 650 dip and 700+ peak like the 80-series filters seem to have. They're good at support so I hope to get an answer, my questions tend to be a bit too technical though :)
I got a couple of Solux 4700K lamps and tried them yesterday but I don't get them to meet the specs, they should do a very good smooth 5000K on slight overdrive (really good D50 simulation), but mine gives only 3900K at 12 volts and reaches only 4600K on 16 volts, and then with a bad spectral shape. I'm currently contacting support at the seller to discuss that issue.
Ushio has the 6500K whitestar, but looking at the spec sheet it seems like it has a peaky spectrum, ie it doesn't mimic D65 well, so I don't think that is a good option. The thing with simulating high temps with tungsten is that you need to filter them, which is done in the reflectors of the Solux and Whitestar (the reflectors thus look blue). One can't get more blue (unless increasing voltage), only filter away red the total output of the filtered lamps becomes low, and I guess if one wants a smooth spectrum one needs to filter even more so the output becomes even weaker. I guess you can keep the color and get more power by letting through some green and making a dip in central red and that's why light balancing filters are the way they are. Not ideal though when we want to use them for camera calibration, although it seems to work quite well in practice anyway.
-
Still investigating the CAT or not to CAT issue... difficult to get answers, I get contradicting messages when I read about it. There's correlated color sets, but also those performance tests when they use spectral relighting as reference, such as Bruce Lindbloom's example:
http://www.brucelindbloom.com/index.html?ChromAdaptEval.html
-
...seem's like we're past the compiling stuff in the DCamProf thread by the way... at least for now...
dcamprof v0.7.0 mingw build crashes when invoked with make-testchart command (you can see that "...... done!" is printed, but .t1 file is not generated)
>dcamprof make-testchart -p 24 test.ti1
Generating 24 patches...
................done!
Win8.1x64 event viewer says :
Faulting application name: dcamprof.EXE, version: 0.0.0.0, time stamp: 0x5560a8e9
Faulting module name: ntdll.dll, version: 6.3.9600.17736, time stamp: 0x550f4336
Exception code: 0xc0000374
Fault offset: 0x00000000000f0f20
Faulting process id: 0x22b0
Faulting application start time: 0x01d09a347828b0fd
Faulting application path: D:\argyll\bin\dcamprof.EXE
Faulting module path: C:\Windows\SYSTEM32\ntdll.dll
Report Id: b60e4dbd-0627-11e5-84b5-80fa5b00a892
Faulting package full name:
Faulting package-relative application ID:
any idea that comes to mind right away ?
-
any idea that comes to mind right away ?
Crashes for me as well, a bug. I'll fix to next patch release. If you specify the number of lines/rows you want (say -l 4) it won't crash.
Regarding the CAT or no CAT, read even more now and I will make CAT a user option to next release. A modern CAT like CAT02 does try to render those subtle appearance differences so it's indeed not the same as relighting. However, it's not really clear how good it is at it, and it's a bit of a matter of taste and situation if you'd want to do it or not, so the default will at least in the next release be same as before, but you can then play with CAT if you like.
-
Still investigating the CAT or not to CAT issue... difficult to get answers, I get contradicting messages when I read about it.
It's a topic with lots of inter-twined connections. Here are some of my thoughts on it:
Re-lighting generally has two effects :- it can change the colors of surfaces due to the details of spectral interaction between an illuminant and a colorant, but it also often shifts the white point.
The human eye and brain is always trying to extract the maximum amount of information from the light that hits the retina. One of the mechanisms to do this is chromatic adaptation. A lot of the adaptation seems to be due to the S, M & L w.l. cones independently adapting to a luminance level, explaining the success of the Von-Kries approach to modeling chromatic adaptation, but there are lots of other things going on in our visual path, including in the nerve cells in the retina, and at higher levels in our visual system.
Chromatic adaptation can be seen as a mechanism that helps achived color constancy - our visual system is striving to present an image to our minds that looks the same, independent of what the actual lighting situation is. In this sense, we are trying to "re-light" the scene in our minds. We don't have the spectral information about what's going on though, so we probably use various heuristics, on top of the adaptation that the retina does.
So a chromatic adaptation algorithm (or matrix) is an attempt to model our visual system and minds inexact algorithm for virtually re-lighting a scene. So re-lighting a scene using exact spectral information will arrive at a result that is a lot like what we are trying to do in our eyes and minds when we look at the original scene, but it is not an exact model of it.
-
Graeme,
txt2ti3 when fed with CGATS file with raw RGB data and .ti3 from chartread with spectral data there will not include spectral details in the output, unless I replace "SPEC_" in source .ti3 with "nm", "NM_", "SPECTRAL_NM_", "R_" or "SPECTRAL_"... it is certainly for a good reason, but not convenient !
-
It's a topic with lots of inter-twined connections. Here are some of my thoughts on it:
Re-lighting generally has two effects :- it can change the colors of surfaces due to the details of spectral interaction between an illuminant and a colorant, but it also often shifts the white point.
The human eye and brain is always trying to extract the maximum amount of information from the light that hits the retina. One of the mechanisms to do this is chromatic adaptation. A lot of the adaptation seems to be due to the S, M & L w.l. cones independently adapting to a luminance level, explaining the success of the Von-Kries approach to modeling chromatic adaptation, but there are lots of other things going on in our visual path, including in the nerve cells in the retina, and at higher levels in our visual system.
Chromatic adaptation can be seen as a mechanism that helps achived color constancy - our visual system is striving to present an image to our minds that looks the same, independent of what the actual lighting situation is. In this sense, we are trying to "re-light" the scene in our minds. We don't have the spectral information about what's going on though, so we probably use various heuristics, on top of the adaptation that the retina does.
So a chromatic adaptation algorithm (or matrix) is an attempt to model our visual system and minds inexact algorithm for virtually re-lighting a scene. So re-lighting a scene using exact spectral information will arrive at a result that is a lot like what we are trying to do in our eyes and minds when we look at the original scene, but it is not an exact model of it.
Thanks! I will in the next version make it a user option to either render XYZ for the calibration illuminant and then do CAT02 to get D50 values, or re-render for D50 from spectra directly (like it does now).
When spectral info is missing DCamProf needs to use CAT to get to the right starting point, and in that case it would be best with a CAT that tries to model relighting as close as possible, rather than be based on corresponding color experiments. An extreme example, if we have a reference file without spectra and with D65 XYZ values and we shoot StdA, one would want to convert to StdA with a "spectral" CAT (bradford seems best alternative) and then back up to D50 with CAT02. Not sure if that could make sense though. Now DCamProf always uses CAT02 and will in that case convert from D65 down to D50.
From all the papers I've skimmed it seems like CAT02 is the current best of the established methods to do a "perceptual" CAT, and Bradford is the best at doing a "relighting" CAT.
-
The spectral data of the light source can be rather accurately guessed from white balance coefficients recorded in the raw file (one can use exiv2 or exiftool to get those; I would prefer exiftool as it is easier to patch when a new camera appears), or from external table (necessary for Nikons, as they do not put white balance coefficients into raw, they have colour data in the form of coefficients for unknown white balance calculation formula).
Chromatic adaptation is the worst case scenario in photography, and encouraging its use is far from best practice. However if it is to be used, we first do spectral adaptation of the reference data (what is called "re-lightning" here, a misleading term for a photographer) to the light used in the shot; and after the colour transform is calculated we calculate chromatic adaptation.
I do not see why not to start from normal workflow, based on 3 shots, Ill. A, 5000K, 6500K; and only after that see what is the error introduced by the methods including smaller number of shots.
-
Just released version 0.7.1. It has the new commands average-targets (stacking for noise reduction) and match-spectra (comparing spectra between targets), adds TIFF support for flatfield.
I've now altered the use of CAT a bit. Bradford is used whenever spectral data is missing and we need to remap XYZ reference coordinates to the calibration illuminant, needed for the CM. I use Bradford instead of CAT02 as all papers I've skimmed indicate that Bradford is better than CAT02 for this particular task when you don't want to model corresponding colors.
Then in the case spectral data is available CAT02 can *optionally* (-C) be used instead of regeneration from spectra for the conversion from calibration illuminant to Profile Connection Space D50. The purpose of using CAT02 would be to simulate the subtle color appearance differences, this will work best in the StdA to D65 range as the corresponding color experiments that CAT02 is based on are primarily made in that range.
I have not yet decided if I should change the default to use CAT instead of spectral generation, for now the default is same as the old. In the default case a tungsten profile shot under tungsten light will (ideally) render the exact same result as a D65 profile shot under D65 light, and that result is equivalent to how colors appear under D50. If CAT02 is enabled, the tungsten and D65 result will look slightly different, and is supposed to look closer to how the eye experienced the original color condition.
I have not yet found any experimental data when CAT02 and spectral generation is compared concerning matching the corresponding color experiments. If I find that and see that CAT02 is significantly better at simulating the "real" look, then using CAT02 will be the new default for the next release.
-
I do not see why not to start from normal workflow, based on 3 shots, Ill. A, 5000K, 6500K; and only after that see what is the error introduced by the methods including smaller number of shots.
Concerning DCamProf the camera shots are needed for getting the raw RGB values, but for XYZ they serve no purpose. XYZ values either comes from a manufacturer-provided CGATS reference file, and then often XYZD50 values, or ideally you have a spectrometer and can measure spectral reflectance from the patches.
So making 3 shots is not necessary, if you're making a dual-illuminant profile for StdA and D65 you only need the StdA and D65 shots. For D50 we only need XYZ values and in that case the camera cannot help us.
The question has then been how to generate those D50 values. From corresponding color experiments we know that even with a fully adapted eye colors do not appear exactly the same in StdA and D65. If we want to model that this means that the D50 XYZ values that we need to connect to the profile connection space should be different from StdA and D65, and CAT02 is the current best at modeling that. That is with spectral data available you first generate XYZ for StdA using spectral data and then remap to D50 with CAT02, and the same for D65.
What's still unclear to me is how good the CAT02 is at modeling this. Maybe it's so bad that regenerating the D50 values from spectra (and thus let them be the same for both StdA and D65) would be a smaller error than using CAT02 despite that D65 and StdA will be equalized to D50 look. Before I make CAT02 enabled per default I need to find an answer to that question.
-
> if you're making a dual-illuminant profile for StdA and D65 you only need the StdA and D65 shots
Well, forward matrix is for D50 white point. It is better be computed when the target is physically lit by a light source with CCT around 5000K.
The device response depends on the light spectrum, and white balance does not exactly equalize different light sources even when they are close to SPD of blackbody.
> ideally you have a spectrometer and can measure spectral reflectance from the patches
Not in my experience. Most spectrometers that are in the hands of photographers can't be used to reliably measure a target, especially when it comes to darker patches.
Actually, much more than 3 shots are needed. Black frame needs to be subtracted, division by the saturated frame needs to be performed, flat field needs to be applied. If one is to average 3 shots, he needs 3 blacks and 3 whites too, plus a flat field.
-
Actually, much more than 3 shots are needed. Black frame needs to be subtracted, division by the saturated frame needs to be performed, flat field needs to be applied. If one is to average 3 shots, he needs 3 blacks and 3 whites too, plus a flat field.
Iliah, aren't you going into overkill mode ? because then one can say that you need black frame and saturation frame for a flatfield shot too, no ? and 3 of those then
-
> one can say that you need black frame and saturation frame for a flatfield shot too, no ? and 3 of those then
I think you can find a lot on the topic on astronomers' sites.
-
0.7.1 build for Windows (mingw = dcamprof.exe + libgomp_64-1.dll) : https://app.box.com/s/x14fjbme5oc0lrrjeck6zlry3nxslavy
-
> one can say that you need black frame and saturation frame for a flatfield shot too, no ? and 3 of those then
I think you can find a lot on the topic on astronomers' sites.
True, but their challenge is a lack of photons. That means that each photon counts, and all sources of noise are a handicap. That's why statisticallly weighted averages are taken from many 'exposures', usually some 16 or more per type of correction (read-noise, dark noise, offset noise, and flat-field).
One of the first things a profile shot needs, is proper levels of exposure, preferably uniform, and illuminated (with a suitable spectral output) at such an angle that subject/target/surface reflections play a minor role. The flat-fielding is to eliminate un-even illumination and lens vignetting and light fall-off.
Cheers,
Bart
-
When we add a black traps for accurate flare compensation what good is that if the levels are wrong. Different personal standards exist, but why not to allow for high ones?
-
When we add a black traps for accurate flare compensation what good is that if the levels are wrong. Different personal standards exist, but why not to allow for high ones?
Nothing wrong with high standards, but they should have a meaningful effect.
When diminishing returns no longer have a measurable effect, we've gone too far.
Cheers,
Bart
-
Until one checks, how could he know if the effect is meaningful or not? I went to the trouble, and for me and my customers it is meaningful. So I shared my experience.
-
I haven't seen any indications that astronomy methods need to be applied on camera color profiling, I don't think that level of precision is required. Of what I've seen so far camera profiling is the most approximate among the typical profiling tasks (screen, print, camera). Still I've added stacking in the latest release so one can experiment. Of course it does not hurt to improve the quality of input data, but at some point it becomes overkill and one is doing more work than necessary. From the results I've seen so far flatfield is necessary if you lit the target with one lamp, but if you have a quality "copy" style setup flatfield is overkill. However how large measurement errors one can accept is also a matter of taste, so different users will come to different conclusions.
As a sanity check one can make one profile with "sloppy" measurements and one with all bells and whistles and compare the results. With DCamProf you can also make experiments with SSFs (where measurement errors are nil) and do some noise-adding experiments and see how things start to deviate.
About the D50 whitepoint, the reason FM is D50 is that profile connection space is D50, to be able to get further in the color pipeline we need to get into D50. However, it doesn't necessarily mean that FM must render the color as if the target was lit by D50, you can keep the color appearance of the original illuminant if you want to. If you want to do that depends on the situation of course, if you're making a reproduction profile for Tungsten lights you probably want to make the subject look like it was shot under D50 anyway as it's better for copy applications. If you want to retain the look of Tungsten, to make colors look as they look to the eye under Tungsten, you need to do something else though. This is what the "corresponding color sets" experiments is about which CAT02 is based on. As said I need more data on the quality of its results though before I'll make it the default behavior in DCamProf.
It seems like you are interpreting the FM as it should be lit for D50 as well, that is the camera raw RGB values should be for D50 even if the profile is a StdA. This is not how DCamProf does it currently, RGB values are always for the calibration illuminant. Citing the DNG Spec:
"The use of the forward matrix tags is recommended for two reasons. First, it allows the camera profile creator to control the chromatic adaptation algorithm used to convert between the calibration illuminant and D50. Second, it causes the white bala
nce adjustment (if the user white balance does not match the calibration illuminant) to be done by scaling the camera coordinates rather than by adapting the resulting XYZ values, which has been found to work better in extreme cases."
I've interpreted this like the intention is that RGB values should stem from the calibration illuminant, and how the D50 XYZ values are derived is up to the profile designer ("control the CAT"), that is one can for example choose to re-generate from spectra for a copy profile, or use CAT02 to better preserve actual color appearance.
If we make the FM matrix based on D50 RGB values this means that the matrix will look exactly the same regardless of calibration illuminant, and that does not seem right to me.
-
Well, I have no intention to interfere. Each of us do the things the way we see fit.
-
Until one checks, how could he know if the effect is meaningful or not? I went to the trouble, and for me and my customers it is meaningful. So I shared my experience.
right - but you just said that most of us do not have a spectrophotometer capable to properly measure all of the target with dark patches (but you have such instruments or access to such instruments), so in that case for those who do not (and i1pro2 is not good) - how does black frame and saturated frame helps when we do not have a target properly measured in the first place ? so certainly while utmost precision is a subject of desire - am I to gain ?... personal testing is good (at least as an experience if nothing else), but I 'd love to hear opinions before testing (because it takes a lot of time and I do have a day job - so I have to consider the time) from those who did... so am I going to gain when I can't do better than i1pro2 with a target ?
-
But I explained already on another forum. You do not need dark patches in the target for profiling, instead you make a composite CGATS from 2 or more shots with different shutter speeds; and you scale the spectral reference accordingly.
-
But I explained already on another forum. You do not need dark patches in the target for profiling, instead you make a composite CGATS from 2 or more shots with different shutter speeds; and you scale the spectral reference accordingly.
yes, that was about the black patches in SG - but what if you have a target with darker patches different chromacity wise from lighter patches - then you can't use spectrum (measured with i1pro2) from lighter patches for those, right ? and not only then - w/o actually measuring properly (not possible with i1pro2) darker patches you even can't say that they are of the same chromacity as lighter patches even if manufacturer claims that ! catch22 - no ?
-
I haven't seen any indications that astronomy methods need to be applied on camera color profiling, I don't think that level of precision is required. Of what I've seen so far camera profiling is the most approximate among the typical profiling tasks (screen, print, camera). Still I've added stacking in the latest release so one can experiment. Of course it does not hurt to improve the quality of input data, but at some point it becomes overkill and one is doing more work than necessary. From the results I've seen so far flatfield is necessary if you lit the target with one lamp, but if you have a quality "copy" style setup flatfield is overkill. However how large measurement errors one can accept is also a matter of taste, so different users will come to different conclusions.
Hi Anders,
I think that the averaging of multiple exposures can help in improving (in particular) the darker patch quality, so it's a useful addition. Flat-fielding is more of a must, due to illumination issues and lens vignetting/light fall-off when the target takes up a significant part of the image and/or the use of relatively wide apertures cannot be avoided.
Cheers,
Bart
P.S. I just realized myself that to eliminate most uneven illumination issues, one can shoot the target at a 45 degrees angle, and illuminate the target perpendicular to its surface with a single lightsource. This will eliminate most surface reflection issues, but of course leaves the camera/lens vignetting and light fall-off issues to deal with. Shooting with a longish focal length will reduce the lighting angle effects even further.
-
> darker patches different chromacity wise from lighter patches
Different chromaticity does not matter with a target composed of 5 pigments only.
-
the darker patch quality
but see the note about measuring those patches - yes, you improve raw RGB data - but where you are going to get the measurements of the same accuracy ? unless you are not going to use those patches in the first place or you have a magnitude better measuring device for your target ... it is interesting to know what are the actual limits and how you can actually find out that the patch that you are trying to measure with i1pro2 or colormunki is not the one that you can reliable measure with them... does anybody has any numbers ?
-
> darker patches different chromacity wise from lighter patches
Different chromaticity does not matter with a target composed of 5 pigments only.
you mean target that is printed on a printer with inks, right ? and the higher end targets simply do not (except black patches) patches that are not reliably measured with i1pro2-level instruments by design ?
-
No, I mean ColorChecker series. There were times when 6 pigments were in use, but one of those got into hazardous materials list.
-
A good way to test the performance of a test target is to measure it, make a profile with camera SSFs and then test the resulting profile against real spectral data from a database. I've done a basic test of that type in a neighboring thread.
The X-Rite targets do well, although profiling directly against the desired database you want to match is best of course. The profiling problem is sort of an impossible equation though, if you correct for one set of spectra it may produce contraproductive corrections for another set of spectra. I've seen this when working with SSFs and profiling directly against real spectra. It's not a big problem though, but the point is that one needs to make some sort of average correction for typical spectra, which means that the search for the perfect target is futile. It's not certain that a profile made against real spectra will work better than a profile made against an artifical target.
The number of pigments matters, but I guess also the spectral shape of the pigments. The X-rite CC24 have nice shapes of their spectra (you can plot it to see) probably because the pigments are broad band and more overlapping. Compared to patches printed on my inkjet which have more narrow band pigments that can achieve higher saturation, but end up with less varied spectra anyway. All that is an area worth more investigation though, from my point of view.
-
txt2ti3 when fed with CGATS file with raw RGB data and .ti3 from chartread with spectral data there will not include spectral details in the output, unless I replace "SPEC_" in source .ti3 with "nm", "NM_", "SPECTRAL_NM_", "R_" or "SPECTRAL_"... it is certainly for a good reason, but not convenient !
txt2ti3 is intended to input foreign file formats, not .tiX files. What are you trying to achieve ? - there may be some other way of doing it that is more appropriate.
-
What are you trying to achieve ? - there may be some other way of doing it that is more appropriate.
for example I have a cgats file with raw rgb data
case #1, I have spectral measurements produced by something else in cgats format
case #2, I have spectral measurements produced by argyll chartread (.ti3 has also XYZ and/or LAB, even I do not need to have them, no option to exclude... not a bid deal, but for consistency again it is good to be able to say that I only need spectral data)
case #1, using txt2ti3 to combine the files gives the output that I like : raw rgb from the actual shot + spectral data from the target measurements, leaving whatever software further down the chain to use the spectral data as it can/need
case #2, using txt2ti3 to combine the files gives me raw rgb + but no spectral data included, which kind of makes using chartread more difficult to use in this situation
inconsistent at least ! what I 'd love to have is to use argyll chartread to measure a target and then use txt2ti3 to combine raw rgb data with just spectral data from chartread output... w/o using text editors, etc... yes, I understand the origin how txt2ti3 was intended to be used, but
PS: and if the target measuments were not in 10nm (but for example like -H mode in 3.33x increments), why it is necessary to strip intermediate measurements from the source file and output only SPEC_380, SPEC_390, etc ?
-
Been reading more on CATs. It seems like all CATs, including the linear Bradford, are derived from corresponding color data sets and thus try to model "color inconstancy", that is the subtle color apperance differences that still exist under various light.
There is no established model specifically designed for "relighting", that is predict XYZ coordinates for a different illuminant that gives the same result as re-intergrating the reflectance spectra with the new illuminant, although there are papers describing methods for doing that, such as this: http://www.cs.sfu.ca/~funt/2014_CIC22_MIRZAEI_GaussianPrediction.pdf and those methods make more accurate results than the established CATs. However, as a DCamProf workflow should include spectra in the normal case implementing this type of transform will not be a top priority. 0.7.1 will use Bradford rather than CAT02 when spectral data is missing for relighting, as the test data I've seen in papers show that it performs better for that particular task, but as said it's still not designed for it and there are better models for it.
The other more interesting question I'm trying to find an answer to is how well the CAT02 keeps appearance over illuminant change. The problem DCamProf faces is that profile must be made relative to D50 (the profile connection space), but the profile may have say StdA as calibration illuminant and we may want to keep those color appearances, that is convert to corresponding colors in D50 rather than relight the reflectance spectra. It's very clear that CATs are highly approximate, they will introduce errors. Therefore I wanted to find a comparison with a spectral model of perfect color constancy, that is relighting from spectra. The only paper I've found on this so far is this:
http://www.rit-mcsl.org/fairchild/PDFs/PRO28.pdf
But unfortunately the experimental data is too thin to make any conclusions, it does indicate though that the spectral color constant model might not have larger errors than CAT02 will have. It will indeed be in error by principle, show constant color when we know there should not be constant color, but the actual average DeltaE might be lower than for CAT02.
So I still don't know if the current default (use spectral color constant model) is the right one, or if I should change to CAT02. The investigation continues...
-
but see the note about measuring those patches - yes, you improve raw RGB data - but where you are going to get the measurements of the same accuracy ?
Just like the capture result will get improved precision from averaging, so will the target measurement improve from averaging multiple measurements. The read/electronic noise, which is a larger percentage of dark patch measurements, will reduce by the square root of the number of reads. So averaging 4 reads, will reduce the noise to 50% of a single read, 16 reads will reduce he noise to 25%. How many is enough? That depends on the level of noise and if it produces erratic readings that influence the precision notably.
Cheers,
Bart
-
Just like the capture result will get improved precision from averaging, so will the target measurement improve from averaging multiple measurements. The read/electronic noise, which is a larger percentage of dark patch measurements, will reduce by the square root of the number of reads. So averaging 4 reads, will reduce the noise to 50% of a single read, 16 reads will reduce he noise to 25%. How many is enough? That depends on the level of noise and if it produces erratic readings that influence the precision notably.
Cheers,
Bart
I have colorchecker sg, I am measuring it with i1pro2, I am getting black patches (those along the edge) as L* = 10.x-11.x, I check the data from BabelColor Patchtool (I'd assume 1:1 from ProfileMaker) and from Iliah Borg (from makeinputicc) and I see that those patches there are L* = 6.x-7.x
Do you think that is an error that can be eliminated by many measurements ? because all my black patches like this and that is = 14 measurements, same L*'s... so do I have a different copy of CCSG (some change in manufacturing process, it differs visually from for example the one that you can see in I-R shots - there the name is on top in the middle, my has the name on top, but on the left side and "Digital SG" is black letters with white background) or does i1pro2 simply have an issue with measuring black patches that can't be eliminated by multiple measurements or something else (what) ? lighter patches way better matching L* wise between my measurements and BabelColor|Iliah.
PS: when I am measuring black trap with i1Pro2, I am getting L* reading from perfect 0.0000 to 0.1 ... so error (including operator's too) is there, but that error does not seem to be 1-s of L*
-
Concerning relighting transform; I've compared Bradford and CAT02 with DCamProf's ability to render an artificial spectra from an XYZ coordinate, and then integrate with the new illuminant, the results are that Bradford and CAT02 has very similar results, and going through an artifical spectra is almost always more accurate, often sub-1 Delta E, while the CATs is often up at 2-4. This algorithm is similar to what is described in the paper linked earlier, although much more computationally expensive.
To the next version the default recommendation for relighting transform will be via artificial spectra, the drawback is that it's very slow though. It's still always best to have measured spectra of your target of course, but if you don't have a spectrometer and only have XYZ coordinates in your reference file and you want to calibrate for a different illuminant you need a relighting transform.
I haven't found any standard name for "relighting transform", some call it "color signal prediction", but in most contexts people say just "CAT", which is confusing as all CATs are designed to predict corresponding colors (that is which tristumulus value to use to produce the same appearance), not predict which tristimulus value the same object will have under a new illuminant. Only if you assume that color constancy is 100% perfect they will be the same, and the current established CATs do not.
-
I have colorchecker sg, I am measuring it with i1pro2, I am getting black patches (those along the edge) as L* = 10.x-11.x, I check the data from BabelColor Patchtool (I'd assume 1:1 from ProfileMaker) and from Iliah Borg (from makeinputicc) and I see that those patches there are L* = 6.x-7.x
Do you think that is an error that can be eliminated by many measurements ? because all my black patches like this and that is = 14 measurements, same L*'s... so do I have a different copy of CCSG (some change in manufacturing process, it differs visually from for example the one that you can see in I-R shots - there the name is on top in the middle, my has the name on top, but on the left side and "Digital SG" is black letters with white background) or does i1pro2 simply have an issue with measuring black patches that can't be eliminated by multiple measurements or something else (what) ? lighter patches way better matching L* wise between my measurements and BabelColor|Iliah.
The differences between multiple samples of the same target will gravitate towards its real mean value (somewhere between 10 and 12 in your case), as the random noise cancels out with each additional sampling of the same patch. But I think that the difference between production runs of the SG plays a role here.
PS: when I am measuring black trap with i2Pro2, I am getting L* reading from perfect 0.0000 to 0.1 (including operator's too)... so error is there, but that error does not seem to be 1-s of L*
That suggests that there is no dust/dirt or calibration issue in play here, but rather a variation in the SG targets.
Cheers,
Bart
-
> haven't found any standard name for "relighting transform", some call it "color signal prediction", but in most contexts people say just "CAT"
Dr. Fairchild was using "spectral adaptation". I use sCAT as an abbreviation.
-
sCAT sounds nice, I'll borrow that!
-
Checked 3 SG targets from different manufacturing batches, Spectroscan/Spectrolino vs. i1Pro - it looks like there are no manufacturing variations to speak of. Higher values in case of i1Pro are consistent and seem to be caused by the in-instrument flare.
-
Checked 3 SG targets from different manufacturing batches, Spectroscan/Spectrolino vs. i1Pro - it looks like there are no manufacturing variations to speak of. Higher values in case of i1Pro are consistent and seem to be caused by the in-instrument flare.
Hi Iliah,
Thanks for checking. I do wonder, measuring a black trap apparently gives virtually no signal, and black should not reflect much, I would not expect enough to cause such an amount of internal glare. If a lighter patch is measured in the same acceptance angle of the instrument yes, but with only 'black' in view?
Cheers,
Bart
-
sCAT sounds nice, I'll borrow that!
Oh, realized it's not the same thing. Fairchild's "spectral adaptation" is CAT based on full spectral data, the human adaptation is still modeled just like in an ordinary CAT. In the paper he uses a "color constant" version as reference though, which is the exact same thing as a relighting transform.
-
Let's consider a workflow which was at some point SOP in (at least) one of museums for reproduction work:
- shoot a target at studio light
- measure light spectrum
- re-calculate the target reference for measured spectrum
- convert "light-adjusted" target reference to XYZ
- calculate a matrix to convert "device RGB" to "light-adjusted" XYZ
- use CAT to D50/D65/Ill.A to get matrices needed for a dcp profile
You can easily see there are serious problems with this workflow, and also they are converting to XYZ at a premature stage.
sCAT is just a term, like log (logarithm). It needs qualification as to the method used. What sCAT emphasizes is that the adaptation is based on spectra directly, not just on XYZ convolution.
-
Error kicks in below L*=20, which, in the words of one of X-Rite guys on a seminar, is as deep as one may want to go for _printer_ profiling.
-
Very interesting on the spectrometer performance on dark patches.
DCamProf is not ideal for copy profiles as it only makes a 2.5D LUT, that is a specific chromaticity gets the same correction regardless of lightness. With say an IT-8 photographic target for copying photographs printed in the same media a 3D LUT will probably achieve a tiny bit better result.
I chose to make 2.5D profiles as I think it's better for generic profiles where exposure will vary as well as object spectra.
This means that one does not really need dark patches in the test target, except for highly saturated colors close to the line of purples that must be dark otherwise the saturation cannot be reached. (There's also a possibility that dark patches can add some spectral variation in the target by mixing in more colorants.)
From DCamProf's perspective many of the "skin tone" patches on the Colorchecker SG are redundant as they are (nearly) just darker versions of a lighter patch, and indeed the SG does not make more accurate skintone profiles than the 24 patch colorchecker when I compare with the lippmann2000 data.
-
the 24 patch colorchecker when I compare with the lippmann2000 data.
consider 2 cases :
1) -p cc24 -p skin-white.ti3
2) -p skin-white.ti3 -p cc24
case #1 = 2 patches from CC24 will exclude all the patches in skin-white.ti3 (resulting target = CC24 patches, 24 of them) and case #2 = all patches in skin-white.ti3 will exclude only those 2 patches from CC24 (resulting target = all 78 patches from skin-white.ti3 and 24-2 patches from CC24 = 100 patches)
so is case #2 going to be really (noticeably) better for raw with caucasian skin tone in your opinion ?
-
Very interesting on the spectrometer performance on dark patches.
so may be then add a parameter to dcamprof make-profile (and may be also to dcamprof make-target) to allow (default - nothing excluded naturally) the exclusion (supply L* value for example, everything darker based on supplied target measurements / spectral data or if not XYZ/LAB / gets gutted) of dark patches (yes, we can edit cgats with raw rgb data / target measurements, but a little bit of crude automation might be useful too ?) as "Error kicks in below L*=20, which, in the words of one of X-Rite guys on a seminar, is as deep as one may want to go for _printer_ profiling." ?
-
> caucasian skin tone
Do we have significant differences in hemoglobin, melanin, carotene spectral properties between different races?
-
Normally, one can't reliably print below L*=15. That may be a good default.
-
> caucasian skin tone
Do we have significant differences in hemoglobin, melanin, carotene spectral properties between different races?
only amount (of the same 3 organic chemicals) is different ... are you implying that we shall not bother then with all those skin spectral measurements at all ?
-
Those skin samples are highly correlated, so very few patches are OK, while more than just a few may be useless or even trough things astray.
-
Spectral databases generally has a lot of data with similar chromaticity, including skin databases, although there are racial differences of course. In any case in terms of correction the default setting of DCamProf is to group together nearby patches and make an average correction for those. There's also a distance weighting algorithm enabled per default so it won't hurt matrix optimization either.
Anyway, in the tests I've done the CC24 skin patch gets lippmann skin sub 1 DE for 90th percentile, ie no meaningful improvement in accuracy can be had. The worst targets I've tested in terms of skin tone still gets within 1.5 DE for 90th percentile. It's hard to make any meaningful improvement on that. So no, a skin database will generally not make any meaningful improvements, not for caucasian skin.
"Excellent skin tones" in camera profiles that often comes up in the medium format forums is not so much about accuracy, but about a designed subjective look, and probably designed for foundation makeup rather than real skin. I'm quite sure that an accurate profile will not please the typical portrait photographer used to profiles with a designed look. I'm not a portrait photographer myself though so I can't contribute much when it comes to the subtleties of skin reproduction. DCamProf is not suitable for designing profiles with a look, but is rather for those of us that prefer designing our own look in the raw converter (or photoshop or whatever), with an as accurate as possible starting point.
In terms of accuracy, skin tones does not seem to be that big challenge for modern cameras, maybe there's been some consious design decision concerning camera color filters to make them good at skin tones, I don't know.
-
so may be then add a parameter to dcamprof make-profile (and may be also to dcamprof make-target) to allow (default - nothing excluded naturally) the exclusion (supply L* value for example, everything darker based on supplied target measurements / spectral data or if not XYZ/LAB / gets gutted) of dark patches (yes, we can edit cgats with raw rgb data / target measurements, but a little bit of crude automation might be useful too ?) as "Error kicks in below L*=20, which, in the words of one of X-Rite guys on a seminar, is as deep as one may want to go for _printer_ profiling." ?
Yes I will consider something like that, maybe think a bit more about filtering. When doing SSF profiles one may have lots of patches so it becomes really tedious to filter out one patch at a time manually.
Looked through some target data, and for DCamProf's sake excluding all sub L*20 patches does not seem to be a loss. It's possible to cover any reasonable chromaticities with L*20 and up.
-
In photography, tone is brightness and contrast (gradation), not colour. We can have excellent skin tones on black-and-white photo.
-
so I decided to make SSF out of just __3__ (three) patches from IT8 shot - took most saturated R, G, B that were > ~30 L*... and... with AA7hVFAI00100.ARW from I-R for Sony A7 ( SSF -> StdA & D65 virtual targets with -p cc24 -> dual illuminant .dcp -> ACR ) :
(http://s14.postimg.org/9vvou3ktd/SSF3_1.jpg)
(http://s15.postimg.org/ifem8r763/SSF3_2.jpg)
it is fair to note that it was a matrix .dcp profile : CM + FM and no LUT
-
so may be then add a parameter to dcamprof make-profile (and may be also to dcamprof make-target) to allow (default - nothing excluded naturally) the exclusion (supply L* value for example, everything darker based on supplied target measurements / spectral data or if not XYZ/LAB / gets gutted) of dark patches (yes, we can edit cgats with raw rgb data / target measurements, but a little bit of crude automation might be useful too ?) as "Error kicks in below L*=20, which, in the words of one of X-Rite guys on a seminar, is as deep as one may want to go for _printer_ profiling." ?
I don't immediately accept that L*= 20 is a/the limit, not without some more empirical evidence. A stepwedge. e.g. one from Stouffer like the R2110 (http://www.stouffer.net/Reflection.htm), might reveal a non linearity in the L* readings towards D-Max, or not. If internal glare is in play, which I somewhat doubt because that would require a bright patch and a dark patch at the same time, then the slope would not be close to the expected step delta 0.10 Optical Density . When the i1pro2 sees only a dark patch, then there is not enough light for a significant glare contribution. Maybe it's electronic noise that creates a threshold, but then a light trap would also give a signal.
Cheers,
Bart
-
so I decided to make SSF out of just __3__ (three) patches from IT8 shot - took most saturated R, G, B that were > ~30 L*... and... with AA7hVFAI00100.ARW from I-R for Sony A7 ( SSF -> StdA & D65 virtual targets with -p cc24 -> dual illuminant .dcp -> ACR )
How is that SSF desgined? Is it that matlab script? I have not had the time to investigate that, but my guess is that it uses a database of measured SSFs and then does some fitting against that? This probably means that with as little data as this there's an element of luck involved, that is if the SSF matches well another SSF in the database you can get a good result.
-
I don't immediately accept that L*= 20 is a/the limit
but that's why you shall be able to use whatever number you want as a parameter or not use that and so all patches included - makes it easier to test then... Iliah says that was said by X-Rite people, I think he 'd not/never quote somebody who was just a marketing person, right ?
-
Is it that matlab script?
basically yes, I did just some mods - to read CGATS from rawdigger (instead of using dcraw, etc), use my target illumination spectrum and use my target measurements from external files and as camera's database (certainly - we do not know who good were there measurements with monochromator - what it is what it is) is the same - I made eigenvectors constant... gutted also whatever junk is there.
-
This probably means that with as little data as this there's an element of luck involved
there is only one Sony there... if one to assume that each manufacturer stays along some "brand color" in CFA design (SSF-wise) then it is fitting well w/ the database consisting mostly of Nikon and Canon :
Canon1D Mark III
Canon20D
Canon300D
Canon40D
Canon500D
Canon50D
Canon5D Mark II
Canon600D
Canon60D
Hasselblad
Nikon3dx
NikonD200
NikonD3
NikonD300s
NikonD40
NikonD50
NikonD5100
NikonD700
NikonD80
NikonD90
NokiaN900
Olympus EPL2
PentaxK5
PentaxQ
Phase One
PointGreyGrasshopper
PointGreyGrasshopper2
Sony Nex5N
CGATS/Raw RGB :
1579.63 727.59 279.61
392.85 1445.76 2621.2
246.83 1708.88 406.09
Target (380 : 10 : 730:
0.006771 0.012186 0.019083 0.022492 0.018859 0.015078 0.01285 0.012292 0.013557 0.01626 0.02006 0.02377 0.025309 0.023362 0.020727 0.019255 0.018586 0.018996 0.02305 0.036697 0.070012 0.131161 0.215468 0.306025 0.390381 0.459813 0.513263 0.554692 0.590821 0.622936 0.652642 0.677777 0.69902 0.717879 0.739663 0.765332
0.007337 0.034702 0.090921 0.207428 0.320195 0.375344 0.384487 0.368493 0.33102 0.273935 0.212965 0.157102 0.109939 0.073874 0.051659 0.039648 0.032222 0.027431 0.026041 0.029307 0.035212 0.038126 0.03505 0.03127 0.030014 0.029921 0.0298 0.030836 0.035664 0.047068 0.067224 0.098205 0.137844 0.183507 0.236008 0.29608
0.006708 0.006364 0.008263 0.007945 0.006133 0.004683 0.003775 0.004248 0.005308 0.008668 0.016754 0.033854 0.063985 0.107077 0.154427 0.184376 0.184275 0.158114 0.120697 0.085062 0.053481 0.028621 0.013728 0.00766 0.006021 0.005722 0.005607 0.005918 0.007386 0.011355 0.019664 0.035081 0.058581 0.09012 0.130444 0.180574
-
then there is not enough light for a significant glare contribution. Maybe it's electronic noise that creates a threshold, but then a light trap would also give a signal.
that is beyond my knowledge to discuss, sorry... I can say that with black trap from Datacolor SpyderCube and not in a pitck dark room and with operator errors (the main again being not in a lightless room) I was never getting > 0.1 L* if I make decent effort fitting i1pro2 to the hole
-
Spectral databases generally has a lot of data with similar chromaticity, including skin databases, although there are racial differences of course. In any case in terms of correction the default setting of DCamProf is to group together nearby patches and make an average correction for those. There's also a distance weighting algorithm enabled per default so it won't hurt matrix optimization either.
Anyway, in the tests I've done the CC24 skin patch gets lippmann skin sub 1 DE for 90th percentile, ie no meaningful improvement in accuracy can be had. The worst targets I've tested in terms of skin tone still gets within 1.5 DE for 90th percentile. It's hard to make any meaningful improvement on that. So no, a skin database will generally not make any meaningful improvements, not for caucasian skin.
"Excellent skin tones" in camera profiles that often comes up in the medium format forums is not so much about accuracy, but about a designed subjective look, and probably designed for foundation makeup rather than real skin. I'm quite sure that an accurate profile will not please the typical portrait photographer used to profiles with a designed look. I'm not a portrait photographer myself though so I can't contribute much when it comes to the subtleties of skin reproduction. DCamProf is not suitable for designing profiles with a look, but is rather for those of us that prefer designing our own look in the raw converter (or photoshop or whatever), with an as accurate as possible starting point.
In terms of accuracy, skin tones does not seem to be that big challenge for modern cameras, maybe there's been some consious design decision concerning camera color filters to make them good at skin tones, I don't know.
This Capture 1 tutorial on achieving uniform skin tone confirms what you've indicated about attempting to solve this through a profile due to so many variables that affect skin tone appearance, the effects of makeup base being the tip of the iceberg. It shows that one has to go to the lengths of editing the images color tables manually using Capture 1's Color Editor Tool...
http://blog.phaseone.com/get-uniform-skintones/
-
Adding more highly correlated patches normally does not help, especially with LUTs. If spectrals are known, it is usually better to eliminate highly correlated samples and patches at the very first step.
For repro work, for example, we make targets that consist only of basic pigments used in painting (a profile for every particular painting, if it is for archival and preservation purposes). Lightness and saturation do not matter - they are anyway achieved through the mix of the basic pigment with something like titan white and soot, both being spectrally very flat (hence in repro work 2.5D or 2D profiles are used more often than not if it is a serious museum).
With CC24/CCSG decent profiles can be obtained from 5-6 non-neutral patches + grey step wedge.
-
of basic pigments
how do you find a number of different "basic" pigments if you have a patch (one patch of whatever color) and spectrophotometer then ? is it possible ?
-
There is a reason why for certain measurements Gretag used to offer polarizing filters with Spectrolino. Light is scattered back from both gloss and textured surfaces, causing flare and glare and making readings from very dark patches less reliable for semigloss and gloss substrates, and glossy black/dark pigments.
i1Pro is cost-optimized not for target measurements, but for printer profiling. IMHO it is only very logical in terms of engineering.
-
With CC24/CCSG decent profiles can be obtained from 5-6 non-neutral patches + grey step wedge.
so for dcamprof as a matter of automation for hoi polloi (dcamprof make-dcp/make-icc steps) it is better to find out and select(make it as a command line parameter/option of course) only non-neutral ( or for example with C* from L*C*h* > N ) patches, calculate CM/FM and possibly 2.5D LUT and then proceed (make it again as a command line parameter/option ) to neutral patches and adjust LUT (or even possibly a tone curve) ?
certainly with CC24 there are so few patches that it might be moot point (bare minimum of non neutral patches) - but for CCSG and homemade targets
-
Well, for 1 patch it is possible only with the help of a database. Don't know if such databases are publicly available, and free; but OKUMURA YOSHIO made quite a study in R.I.T. on the topic.
UPD. Found one of his studies online http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.79.799&rep=rep1&type=pdf
-
Well, for 1 patch it is possible only with the help of a database. Don't know if such databases are publicly available, and free; but OKUMURA YOSHIO made quite a study in R.I.T. on the topic.
UPD. Found one of his studies online http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.79.799&rep=rep1&type=pdf
"...The interesting figure of the table is the volume of a inkjet printer (Canon PIXUS iP8600) because the size is not different from the volume of the Pointerʼs color gamut. The result reminds us of an innovative printing technology in which eight dye inks of the printer reproduce almost all the surface real colors introduced by Pointer [Pointer 1980]..."
-
one more unscientific test about dark patches, their measurements, glare on dark patches with glossy print, etc ... this IT8 target was the one measured and few saturated patches from it (actual shot) were used to build SSF profile which was then used in raw conversion of the this shot
(http://s7.postimg.org/gu100e1u3/IT8_1.jpg)
(http://s9.postimg.org/vblw09ssf/IT8_2.jpg)
(http://s13.postimg.org/lthrvwspz/IT8_3.jpg)
(http://s8.postimg.org/wgx4of2qb/IT8_4.jpg)
-
last unscientific experiment for this night, I switched to other targets
CC Passport (1 shot - not averaged, 1 light = illumination : H3200K+80A, flat fielded), SSF from matlab using all patches from that raw, measured illumination (argyll -H mode), measured target (average of 3 measurements of the same target, argyll -H mode)
single illuminant DCP profile for measured illumination and measured target
converting the same raw with ACR using this profle
http://s14.postimg.org/y9noh3y8x/ccpp_3.jpg ->
(http://s14.postimg.org/y9noh3y8x/ccpp_3.jpg)
(http://s17.postimg.org/66e0opvm7/ccpp_1.jpg)
.
(http://s11.postimg.org/rzxlk3s2b/ccpp_2.jpg)
.
so calculated SSF are getting better, no ?
-
Highly correlated spectra should not hurt DCamProf profiling as it's designed to deal with such cases, but on the other hand it will not help much either. They're handled as averaging a bunch of patches into one, unless you reduce the chromaticity grouping distance to zero then it will try to correct for each patch (and result will not be good).
DCamProf currently makes no use of a neutral step wedge (it sees it just a bunch of highly correlated spectra), as it assumes the target shot is linear. Not sure yet if that's a good idea. Shots with glare issues are certainly not linear, but I've thought that those cannot be well corrected anyway so better require a glare-free shot. Even with a target shot made with proper lighting in a dark room maybe there can be some residual glare than needs correction though, or if the spectrometer measurement suffers from glare one needs to correct the other way around. Not sure though if a simple spline curve lightness correction is relevant.
Interesting to know that 2.5D LUTs are used in repro work too, and yes when the lightness adjustment is spectrally flat is should indeed work out fine. If you like DCamProf can make 2D LUTs too by relaxing the L dimension maximally (say -l 1000,0 to make-profile).
-
I've decided to make CAT enabled per default to the next version, as I think most users will expect the look of StdA and D65 differ slightly just like in real life. CATs are far from perfect but from what I've read and some tests I've made it does seem that they are less bad than just doing relighting when it comes to modeling actual appearance.
If you do a D50 profile there will be no CAT of course as it's not needed, and you can still force "color consistent mode" (religthing instead of CAT) which is a good idea if you're making a profile for copy work, or if you just prefer that look. Color consistent means that the profile will make the images look like shot under D50 regardless of calibration illuminant.
-
I've decided to make CAT enabled per default to the next version, as I think most users will expect the look of StdA and D65 differ slightly just like in real life. CATs are far from perfect but from what I've read and some tests I've made it does seem that they are less bad than just doing relighting when it comes to modeling actual appearance.
If you do a D50 profile there will be no CAT of course as it's not needed, and you can still force "color consistent mode" (religthing instead of CAT) which is a good idea if you're making a profile for copy work, or if you just prefer that look. Color consistent means that the profile will make the images look like shot under D50 regardless of calibration illuminant.
'd be nice to have a detailed help section on that in the blog.
-
'd be nice to have a detailed help section on that in the blog.
Yep I'm adding a new fat section on CAT and relighting in the docs to explain the various aspects around their use.
-
For normal profiles I use grey step wedge only to check linearity and diagnose flare and glare. This is an important check, and important warning. Flare can't be effectively eliminated based on a tight group of neutral patches as it changes across the target; but border pattern on SG targets combined with the pair composed of the black and the white patches in the middle allows to have a TPS (I actually use 2 TPSs, one additive, and one multiplicative) to compensate for flare.
Practical recommendation was to use polarizing filters to eliminate the effect, but now it is abandoned as proper target design that includes the interleaved black and white patch pattern and math do a better (much better in fact) job.
-
what I 'd love to have is to use argyll chartread to measure a target and then use txt2ti3 to combine raw rgb data with just spectral data from chartread output... w/o using text editors, etc...
Normally that's all done using scanin. It creates the RGB values from the image and combines it with the reference file. How do you happen to have RGB values not created this way ?
yes, I understand the origin how txt2ti3 was intended to be used, but
I'm reluctant to clutter txt2ti3 with more file formats to recognize that it needs. It's quite complex and hard to test as it is.
PS: and if the target measurements were not in 10nm (but for example like -H mode in 3.33x increments), why it is necessary to strip intermediate measurements from the source file and output only SPEC_380, SPEC_390, etc ?
Sorry, I'm not following you. What tool, input is what ?
-
If you are comparing to CC24 reference, yes, it is getting better; the two sources of error as I see it are still incorrect estimation of red response in the blue part of the spectrum (may be something to do with the pigments in the target you measured) and some vail.
-
I haven't found any standard name for "relighting transform", some call it "color signal prediction", but in most contexts people say just "CAT", which is confusing as all CATs are designed to predict corresponding colors (that is which tristumulus value to use to produce the same appearance), not predict which tristimulus value the same object will have under a new illuminant. Only if you assume that color constancy is 100% perfect they will be the same, and the current established CATs do not.
I can't agree - there is no reason to assume that being able to spectrally accurately re-light a scene is what human mechanisms of color constancy are striving for. They are evolved mechanisms that are useful to us, that bear the same resemblance to re-lighting as any 3-channel white balancing algorithms does.
-
If you are comparing to CC24 reference, yes, it is getting better; the two sources of error as I see it are still incorrect estimation of red response in the blue part of the spectrum (may be something to do with the pigments in the target you measured) and some vail.
as a side note - making a target directly (not through SSF) is better, but with SSF you can can play around with extra options
here is when profile is built directly =
(http://s3.postimg.org/qwkyb9wyr/cc24p_better_1.jpg)
so
the two sources of error as I see it are still incorrect estimation of red response in the blue part of the spectrum (may be something to do with the pigments in the target you measured) and some vail.
I do have operator's errors indeed, still working out on how to shoot targets, but here (see above) I guess the matlab script and the algorithm of building SSF are to blame mostly, no ?
-
I can't agree - there is no reason to assume that being able to spectrally accurately re-light a scene is what human mechanisms of color constancy are striving for. They are evolved mechanisms that are useful to us, that bear the same resemblance to re-lighting as any 3-channel white balancing algorithms does.
The issue of "re-lightning" is mainly for Illum.A and D65 matrices that Adobe are using to estimate colour temperature and tint. Those matrices are not used anymore for calculation and presentation of colour. To estimate the CCT it is better to have spectral calculations when computing those matrices, not CAT.
-
TPS
what is TPS ?
as proper target design that includes the interleaved black and white patch pattern and math do a better (much better in fact) job.
but then why not use an extra flat field (flare field) which is printed checkerboard ? and have 3 shots - target (averaged/bracketed), FF, CB
-
> haven't found any standard name for "relighting transform", some call it "color signal prediction", but in most contexts people say just "CAT"
Dr. Fairchild was using "spectral adaptation". I use sCAT as an abbreviation.
I don't think that's the same thing though.
As I understand it, "re-lighting" is creating a transform of XYZ/RGB values based on the change in values between spectral reflectance values viewed under two different illuminants.
Marks Fairchild's paper on spectral adaptation seems to be an approach to doing a white point change in spectral space, rather than tri-stimulus space.
[ I confess that spurred by some of the discussions here, I contemplated a similar idea to that in Marks paper, based on the idea of taking the original illuminant or spectral emission values, dividing by the black body spectrum which has the source white point, and then multiplying by the black body spectrum which has the destination white point. This was based on the idea of a black body spectrum being the model of the "natural" spectral shape change with a change in color temperature. ]
-
From comparing SSF vs. direct, seems SSF algorithm may need some further improvements; however it may be also something with the measurements, too. The direct transform looks not bad at all (on UK scale, same as "near excellent" in US).
-
Flare needs to be estimated from the actual scene, as it is scene-dependent. TPS = thin plate spline.
-
Normally that's all done using scanin. It creates the RGB values from the image and combines it with the reference file. How do you happen to have RGB values not created this way ?
I am using rawdigger to extract raw values and flatfield them... then I 'd like some automation to combine those with spectral data from chartread to be used further down the /potential/ workflow pipeline
I'm reluctant to clutter txt2ti3 with more file formats to recognize that it needs. It's quite complex and hard to test as it is.
but the issue is not with RD - the issue is that txt2ti3 treats its own (argyll) format differently !!! I have no issues to use txt2ti3 with RD and foreign formats... I want to use txt2ti3 with argyll chartread output... see the irony here ? the RD data is the same in both cases... you already recognize your own format - just make it possible to output the data which txt2ti3 currently does not output
Sorry, I'm not following you. What tool, input is what ?
chartread with -H outputs 3.3nm data... txt2ti3 when used with with such spectral data (mutated from chartead output into foreign format for the sole purpose of txt2ti3 being able to include spectral data in the output when combining it with raw RGB) includes only 10nm steps... why no option to preserve 3.3x data ? I can certainly just use a text editor, etc... but again it will be easier to use less steps.
-
Flare needs to be estimated from the actual scene, as it is scene-dependent.
but are we not estimating/correcting flare for the actual scene of shooting target for profile building - so that it will help with creating a profile too, no ?
-
Oh, realized it's not the same thing. Fairchild's "spectral adaptation" is CAT based on full spectral data, the human adaptation is still modeled just like in an ordinary CAT. In the paper he uses a "color constant" version as reference though, which is the exact same thing as a relighting transform.
I think you mis-interpret the testing in that paper. The "color constant" version was simply one of the 3 other algorithms the spectral CAT was sanity checked against in a lightweight psychophysical experiment. The following more precise Computational Model Testing used CAT02 as the reference, not the "color constant" computation, although it fared well - but note the caveat about metamerism, which is the principle reason I don't think it should be the default in camera profiling.
-
I can't agree - there is no reason to assume that being able to spectrally accurately re-light a scene is what human mechanisms of color constancy are striving for. They are evolved mechanisms that are useful to us, that bear the same resemblance to re-lighting as any 3-channel white balancing algorithms does.
I probably wasn't clear, I don't suggest relighting as an alternative to CAT, although I'm interested in it's performance which is lightly touched upon in Fairchild's spectral adaptation paper. So I don't think we are in disagreement.
I don't think it's desirable to use a CAT when relighting is what you want to do. When do I want to do relighting? Say you make a profile for tungsten light, then I need XYZ reference values for tungsten, the values for the observer under this particular light for the particular target. This is calculated from spectra when possible, ie relighting. Then we need D50 XYZ reference values for the PCS, these are then created by applying CAT to those tungsten reference values to keep the color appearance.
That is CAT is used when moving to the PCS, and relighting is used when we need values for the actual calibration illuminant.
In 90% of the use cases, perhaps 95% of ICC profile use cases, the user will make a D50 profile and have a file with D50 reference values, and in this case CAT/relighting is not used.
-
I don't think it should be the default in camera profiling.
Color constancy will not be the default in the next release. Default will be relight to calibration illuminant, use CAT to get back to PCS.
There are three scenarios for relighting in DCamProf: 1) recalculation from spectra (ideal), 2) recalculation from simulated spectra (using DCamProf's ability to render a spectrum from an XYZ coordinate), 3) using Bradford CAT. The last is "wrong" as CAT is not relighting, but among the CATs the linear bradford seems to be the least bad at the task, it performs worse than the spectral simulation though.
For the CAT, CAT02 is used.
-
From comparing SSF vs. direct, seems SSF algorithm may need some further improvements; however it may be also something with the measurements, too. The direct transform looks not bad at all (on UK scale, same as "near excellent" in US).
and extra data to illustrate direct profiling (w/o going through SSF) results from the prev. post
attched PatchTool compare report :
(http://s16.postimg.org/jyp10k16t/cc24p_better_2.jpg)
-
For normal profiles I use grey step wedge only to check linearity and diagnose flare and glare. This is an important check, and important warning. Flare can't be effectively eliminated based on a tight group of neutral patches as it changes across the target; but border pattern on SG targets combined with the pair composed of the black and the white patches in the middle allows to have a TPS (I actually use 2 TPSs, one additive, and one multiplicative) to compensate for flare.
Practical recommendation was to use polarizing filters to eliminate the effect, but now it is abandoned as proper target design that includes the interleaved black and white patch pattern and math do a better (much better in fact) job.
Thanks for sharing the "tricks" I really appreciate it.
-
If we go a step back for a moment... If we shoot a target 2 times, once with a light source filtered to 5000K, and once with a light source filtered to 6500K, we will have 2 sets of device values that are not in linear proportion to each other. Same, if we take 2 shots under daylight sunny and daylight overcast. The usual point here is that the results should not look the same anyway due to human perception. However it is a general statement, applied backwards, to justify already existing math. Contrary to scanners where the light spectrum is fairly stable, and viewing prints under D50 lights in booths, photography deals with a much more diversity in light sources, and constant complaints on wrong colour is not because photographers and clients are being capricious, but because there is a problem there indeed. This problem stems mostly from the nature of CATs.
For now, the only publicly available way to tame CATs in wrong places is to create ad hoc profiles, that is to profile for the light is the scene (easy in studio, very difficult anywhere on location). What happens when an ad hoc profile is created usually is that the CAT is an organic result of computing a transform between a shot taken under studio lights and target reference being D50/2°. Effectively it is what we here refer to as "re-lightning". But in this case we try to use not "some" lights, but lights as close to D50 as possible - and that helps to keep adaptation to minimum. Try the same with an arbitrary light - and the results are not usable. This alone demonstrates the problem we are having with CATs, "re-lightning", and profiling.
-
but are we not estimating/correcting flare for the actual scene of shooting target for profile building - so that it will help with creating a profile too, no ?
Sorry, I lost you here... The target is the subject in the shot for profiling, and the flare/glare needs to be estimated from the target and eliminated as much as possible from the target (smoothness here is the paramount, so a spline is a good way to do it; better to have a larger error than to sacrifice smoothness), not from some other object or other scene, as flare and glare are scene-dependent and position-dependent (varies with the scene and depends on the coordinates in the scene), right?
-
Interesting indeed, it seems like I may need to revisit the use of CATs once more... I'll let it stay in the way described in the upcoming release though, at least for a few days ;)
How well CATs really work is indeed a bit of a mystery, hard to figure out from just reading papers. From reading, guessing and some limited numerical testing I've assumed that they are quite okay. We'll need to do some practical testing and judgements by eye too I guess. If they mostly produce useless results, I will have to return to color constancy as the default mode.
However if I understand Iliah correctly, you really need to make profiles with D50 lights to get good results, that is it's better to use a D50-designed profile even if it's used under tungsten light?
-
> In 90% of the use cases, perhaps 95% of ICC profile use cases, the user will make a D50 profile and have a file with D50 reference values, and in this case CAT/relighting is not used.
That's exactly what happens, and that's the wrong way of doing things. If the light in the scene is not accounted for, the transform is going to be wrong.
-
Sorry, I lost you here... The target is the subject in the shot for profiling, and the flare/glare needs to be estimated from the target and eliminated as much as possible from the target (smoothness here is the paramount, so a spline is a good way to do it; better to have a larger error than to sacrifice smoothness), not from some other object or other scene, as flare and glare are scene-dependent and position-dependent (varies with the scene and depends on the coordinates in the scene), right?
Iliah, do you know how much of a problem glare is in a proper copy setup in a dark room? The camera outside "the family of angles" etc? Currently I assume that glare is negligible in this situation, but I'm not sure if that really is the case.
-
Sorry, I lost you here... The target is the subject in the shot for profiling, and the flare/glare needs to be estimated from the target and eliminated as much as possible from the target (smoothness here is the paramount, so a spline is a good way to do it; better to have a larger error than to sacrifice smoothness), not from some other object or other scene, as flare and glare are scene-dependent and position-dependent (varies with the scene and depends on the coordinates in the scene), right?
yes and what is wrong with replacing b/w patches in the target (for example the one that does not have them in the first place) with b/w patches for a 2nd shot under the same light, camera position, etc... just like you do flatfielding... remove the target, place checkerboard target... advantage of checkerboard target is that you can make smaller patches (for example) and they will cover the whole area of the original target - unlike SG where you have those patches along the edge and in the center
-
>you really need to make profiles with D50 lights to get good results, that is it's better to use a D50-designed profile even if it's used under tungsten light?
Depends on what we consider acceptable. To create an Illum.A or Studio Tungsten (3200K) profile, I shoot under that light, use target spectral reference measured (or re-calculated) to the light spectrum in the scene, and after that use some (s)CAT to make a profile with D50 WP. Incidentally, Spectrolino measurements are taken with Illum.A light source, and D50 reference is the result of spectral re-calculation. For creating D65 reference, they have a filter attachment.
-
yes and what is wrong with replacing b/w patches in the target (for example the one that does not have them in the first place) with b/w patches for a 2nd shot under the same light, camera position, etc... just like you do flatfielding... remove the target, place checkerboard target... advantage of checkerboard target is that you can make smaller patches (for example) and they will cover the whole area of the original target - unlike SG where you have those patches along the edge and in the center
I see. The substitution patches need to be of the same size as patches on the target, made on the same substrate, and using the same pigments. There are minor geometry changes involved, like different number and locations of white patches will cause different flare.
-
Iliah, do you know how much of a problem glare is in a proper copy setup in a dark room? The camera outside "the family of angles" etc? Currently I assume that glare is negligible in this situation, but I'm not sure if that really is the case.
From measurements I took from the shots made in well-equipped studios by seasoned photographers, typically flare is robbing 1/2 of a stop of dynamic range from and SG target, and the toe portion (non-linear shadows, non-linearity being more than 3% allowed by some museum instructions) is about 1 stop long. That poses a problem, as SG target is only 7 stops DR.
-
I see. The substitution patches need to be of the same size as patches on the target, made on the same substrate, and using the same pigments. There are minor geometry changes involved, like different number and locations of white patches will cause different flare.
I am sorry - why same pigments for example, you calculate (in your setup) flare corrections using pigments from B/W patches and applying to patches of different pigment composure, no ? so why demand different from checkerboard flarefield ?
-
typically flare is robbing 1/2 of a stop of dynamic range from and SG target... That poses a problem, as SG target is only 7 stops DR.
you mean that when you measure SG with a really good instrument (no issues with dark patches) it is 7 stops, you do a shot and from raw rgb is 6.5 stops on the same targer (between black and white patches), right ?
-
you mean that when you measure SG with a really good instrument (no issues with dark patches) it is 7 stops, you do a shot and from raw rgb is 6.5 stops on the same targer (between black and white patches), right ?
Exactly.
-
I am sorry - why same pigments for example, you calculate (in your setup) flare corrections using pigments from B/W patches and applying to patches of different pigment composure, no ? so why demand different from checkerboard flarefield ?
Same pigments - same reflection. It is indeed a "field" problem, if one wants to recreate a magnetic field on a flat surface, it is simpler to use the same magnet sizes and same sizes and spacing than to change the distances.
-
Same pigments - same reflection.
I see what I am missing - you mean that reflections from other patches shall be present exactly, including color patches that we in my suggestion remove to replace with flarefield, right ? but are those reflections a bigger problem vs everything else - walls, ceiling, floor, etc... but then those are again reflecting from patches... catch22 >:( !
-
From measurements I took from the shots made in well-equipped studios by seasoned photographers, typically flare is robbing 1/2 of a stop of dynamic range from and SG target, and the toe portion (non-linear shadows, non-linearity being more than 3% allowed by some museum instructions) is about 1 stop long. That poses a problem, as SG target is only 7 stops DR.
Maybe it's clearer to make a distinction.
Semi-specular surface reflection of of a target is just reflection. This is an issue that has to do with surface smoothness and angle of illumination. Glossy targets have more specular reflecting surfaces, and semi-matte targets have a more Lambertian diffuse reflection. So observing the angle of illumination versus the position of the camera/optical axis can eliminate the reflection of the lightsource better on a glossy target, but it requires a controlled environment (no other bright reflecting or light emitting surfaces that can reflect their presence to the camera lens). Diffuse reflecting surfaces will always add more environment and direct lightsource reflection (desaturation) to the patch reflection.
Reflection inside an optical system can be veiling glare (lack of perfect anti-reflection coating due to angle and wavelength and internal reflection e.g. due to non-blackened edges of lenses or barrel surfaces) which adds a uniform amount of non-image forming light mostly noticeable in the darker tones. Flare is a pattern, a (multicolored) image of the light source as formed by reflections between lens surfaces, and is more local. This is an optical issue that can also get worse near brighter areas in the scene or image, which suggests that white and black patches of targets are best not positioned directly next to each other.
So surface reflection is target and lighting related, internal lens / instrument reflection or Glare (maybe even flare patterns) is an optical phenomenon.
And then there is electronic noise added to the signal that the sensor (camera or spectrometer) records. The signal itself is already noisy (photon shot noise) which is worse when fewer photons are available (e.g. coming from darker patches). Truely random noise can be reduced by averaging multiple images/captures/readouts.
Cheers,
Bart
-
basically yes, I did just some mods - to read CGATS from rawdigger (instead of using dcraw, etc), use my target illumination spectrum and use my target measurements from external files and as camera's database (certainly - we do not know who good were there measurements with monochromator - what it is what it is) is the same - I made eigenvectors constant... gutted also whatever junk is there.
CGATS is not the best option. Switch to CSV to have pixel coordinates too, you will need those for flare/glare compensation splines - tpaps in matlab. And to assess linearrity and field flatness, too.
-
you will need those for flare/glare compensation splines - tpaps in matlab.
haha, I think I need some math ed. before that...
-
haha, I think I need some math ed. before that...
I will help you with relevant code if you will need it; but it is so straightforward I doubt you will need any help.
-
I'll add some glare correction feature to the flatfield command. I need to read up a bit on how to best model glare first though. I had already started with some spline stuff, but got some issues concerning spectral flatness etc, which I need to investigate a bit first. Oh it would be nice to have Photo Research spectroradiometer now.
-
Just in case, I apply flare/glare removal stage before flat field, and the goal of the flat field is mostly to get uniform white balance across the target.
-
> it would be nice to have Photo Research spectroradiometer now
Maybe some University will be interested :)
-
Oh it would be nice to have Photo Research spectroradiometer now.
but are you not working in University or something like this yourself ? some vo... beer and voila...
-
I will help you with relevant code if you will need it; but it is so straightforward I doubt you will need any help.
that you, that shall wait as I still need to polish the basics...
Iliah, a question, as you saw a lot of targets, the difference between QP202 (big one) and QP203 (small one) cards... QP203 looks like patches are painted on a substrate and not that glossy at all and QP202 looks like they are cut and glued and somewhat of lesser quality (like those printed paint samples from any hardware store) and glossy (and substrate glossy too)... and measurements are quite different (while it seems that I remember from somewhere that you mentioned that spectralwise they are similar - I think from references supplied with makeinputicc)... can you comment on that ?
attached are the measurements :
-
Done a little bit of glare correction coding, and now a little bit of review of test shots. I'm not yet having satisfactory results, it breaks more than it fixes.
In a glossy homemade target shot indoor in a dark room with one StdA lamp outside family of angles, I get 5.57 stops between brightest and darkest color (not black, but super-deep purple, had no blacks in that target) with SSF virtual process (ie glare free) and 5.41 stops in the real shot, that is a 0.16 stop difference. Not too big DR difference, but the darkest patch is 1/3 stop too bright compared to what it should be. Measurement error in the instrument (if it's glare there too) may contribute to a smaller number than it should be.
Anyway, from the initial glare correction code tests it seems like the glare error may be too small compared to the uncertainties in the glare correction model.
Currently the code searches for neutral patches in the target by looking how flat the spectra is, then makes a reference value from provided illuminant spectrum+greenSSF (an "typical" SSF if no SSF is provided), and compares that with the actual value. This serves as input to a spline contrast correction curve. The intermediate points are estimated with TPS. Illuminant mismatch, SSF mismatch, deviation from spectral flatness, distance to nearest control point all contribute to error. However, I suspect the simplistic model of glare, ie just a contrast spline curve, contributes the most error. I'm not satisfied with that model, but haven't come up with anything better.
I've not given up yet though, I have a few things left to test.
I does look tempting though to just skip glare correction and instead relax lightness correction in ultra-saturated violets-purple as that's where the errors seem to be located.
-
My glare correction method is based on comparing contrast between the neighboring black and white patches on the shot of the target to the contrast between the same patches in the reference measurement of the target. It is done on all 4 channels independently. The correction coeff. is than placed at the coordinates between the patches, and a spline is constructed. I use 2 splines, one to compensate for the overall veiling glare (bias), uniform additive model; and the second in multiplicative model, to correct for scene-dependent flare and glare - that one is not uniform.
-
the contrast between the same patches in the reference measurement of the target.
and contrast using XYZ is something like log2(Y from white patch / Y from black patch), right ?
-
> contrast using XYZ is something like log2(Y from white patch / Y from black patch), right ?
Contrast is Y1/Y2; but if you want it to convert to photographic stops, log2(Y1/Y2) or log2(Y1)-log2(Y2). For counter-flare splines, I use just contrast, not log2
-
> contrast using XYZ is something like log2(Y from white patch / Y from black patch), right ?
Contrast is Y1/Y2; but if you want it to convert to photographic stops, log2(Y1/Y2) or log2(Y1)-log2(Y2).
yes, stops... thank you.
PS: out of curiosity - what the best (min) difference you achieve yourself with your own setup between the contract from measurements and the contract from your shots, before any math. corrections... ? for example I checked my shots and it seems that so far the best (in stops) difference is somewhat around what Torger mentioned (~0.17)... how small you can realistically get by setup (target, light, isolating everyting else that might throw a bad reflection to target/lens, hood, angle of light, etc) before you need to start using corrections with the data ? the mere fact that you are correcting says that you do not achieve zero, do you ? so it is interesting how far that 0.17 is from what you can do in a natural way.
-
attached are the measurements :
it seems I forgot about the troubles with new RAR format - here are zip files in case the issue still persists
-
> what the best (min) difference you achieve yourself
For a 7-stops SG target, lens with concave rear element, same setup:
D4s - slightly less than 0.2 EV reduction
Df - 0.3 EV reduction
D750 - 0.5 EV reduction
lens with convex rear element - approximately +0.1 EV, making D4s 0.3 EV and so on
For an 11-stops target, it starts with 0.9 EV and goes to 1.2 EV
With mirrorless camera, flare is very lens-dependent, but with 100mm Zeiss I can reliably get to 0.1 EV reduction on SG.
MF back on SG - got 0.43 EV reduction.
But if the target is shot out of studio, on location, as you've seen on the other forum, things with flare are not so controlled.
-
Simply unrar'ed on Windows. I e-mailed QP guys for an explanation. My pair of 202 and 203 were very close; and the site also gives one reference for both.
-
Simply unrar'ed on Windows. I e-mailed QP guys for an explanation. My pair of 202 and 203 were very close; and the site also gives one reference for both.
my QP203 is 3 years old and my QP202 is just couple month old.... the difference of newer QP202 is very big vs your files (black patch specifically), it does not look like an operator or instrument error to me (I have one more set of measurements with i1pro from 3 weeks ago and colormunki measurements from 6 weeks ago), like black patch L* = 20.4x (ColorMunki) ... 20.9x (i1Pro2) vs L* = 29.x in your files... just a note.
-
Could be the QP203 hasn't aged well. Has it seen much light?
Brian A
-
Could be the QP203 hasn't aged well. Has it seen much light?
Brian A
the issue is with a new (2mo old) QP202, not with QP203.
-
the issue is with a new (2mo old) QP202, not with QP203.
Have you got it from them directly or via reseller?
-
Have you got it from them directly or via reseller?
QP202 from B&H ... do you think B&H is enterprising and making 'em in their basement ? QP203 was from QPCard directly back in 2012.
-
QP202 from B&H ... do you think B&H is enterprising and making 'em in their basement ? QP203 was from QPCard directly back in 2012.
No - was worth a check though. I got my 203 from re-seller in UK and had to replace it - it was a bad batch apparently.
-
No - was worth a check though. I got my 203 from re-seller in UK and had to replace it - it was a bad batch apparently.
but yours was small QP203, not big QP202 ?
-
Done some more testing on glare and glare reduction algorithms. Core of the problem is that there are many unknowns when modeling glare so it's hard to make accurate results, and the more glare it is the harder it is to make any sane correction.
One of my conclusions is that (semi-)glossy targets produce so much glare when shot outdoors that a glare correction algorithm is bound to fail. You easily halve the dynamic range of the target in diffuse outdoor light. With that kind of input it's really hard to make good results, much better to use a matte target.
So for DCamProf the rule is that if you have a glossy target you must shoot it indoor in controlled conditions. I've done so in attached photo, lit a glossy target with a single light in a dark room, the uneven light corrected with flatfield correction (which can unlike glare be modeled which high accuracy, so really one light is ok). A minor amount of glare can be measured, but comparing the SSF results (glare free) with the target generated profile the differences are so small that my conclusion is that in the context of all inaccuracies there is in camera profiling the glare issue can be ignored. This assumes photographers will be able to follow instructions when it comes to shooting a target, which I guess is not always the case in real life. I'll be adding a "glare detection" log in the make-profile step so it least is noted when suspicious input is given.
(If there is a significant amount of glare this is seen as the SSF profile generating sane and accurate colors while the target profile will make dark patches too dark or if large amount of glare, the whole target too contrasty and saturated.)
Remaining case is matte targets shot outdoor, there I see both visible effects of glare, but fortunately also low enough glare to be able to make a reasonable correction. I shall do a little bit more testing on that.
-
One of better lighting setups is what Imaging Resource are using for their hVFAI shots of SG target. Checking those shots with and without applying of flare correction, the colour transforms using flare correction are 2 dE2000 more accurate.
-
the colour transforms using flare correction are 2 dE2000 more accurate.
2 ! I certainly need to try to write a code in Matlab then... at least I can do this during the daylight time :D
-
One of better lighting setups is what Imaging Resource are using for their hVFAI shots of SG target. Checking those shots with and without applying of flare correction, the colour transforms using flare correction are 2 dE2000 more accurate.
Thanks for the tip, I played around with animaging resource image:
http://www.imaging-resource.com/PRODS/canon-5d-mkiii/E5D3hVFAI000050.CR2.HTM
with the reference data from here: http://www.babelcolor.com/main_level/ColorChecker_content/Digital%20ColorChecker%20SG.txt
Profiling fails miserably, some dark colors pull in opposite directions compared to lighter colors. By excluding the darker, many which are redundant in terms of chromaticity anyway, a good profile is had (sanity-checked comparison with the colorchecker24 result in the same image). Can the dark color errors only be due to glare, or is the reference file bad?
(compared to the reference data it does show a quite large amount of glare, the black patches are ~1.1 stop too bright in the test shot.)
-
Can the dark color errors only be due to glare, or is the reference file bad?
as far as I know BabelColor Patchtool reference file is what was in ProfileMaker... so it is from the same company that makes CCSG... so if they keep the process steady since 2005 it is more or less accurate (within their tolerances) and you can also compare with what Iliah supplied with makeinputicc...
-
> Profiling fails miserably
That is exactly so. Now, if you will have a look at their lighting setup, http://www.imaging-resource.com/ARTS/TESTS/HMI.HTM and some more here http://www.imaging-resource.com/ARTS/TESTS/SLMULTI.HTM you will see they do try, and from seeing what others do, Imaging Resource is among the best.
I was looking at their Canon 6D target, and flare mounts to 0.83 EV easily, dark patches (not just black, but all the dark ones, including pure blue and red) are not very usable; and plus they underexpose the targets not looking at the histograms of raw data (though they do have RawDigger and FastRawViewer).
I'm afraid it is close to impossible to make folks shoot targets properly, so we either ignore the patches where flare is about 10% of raw values and higher, or we issue warnings and refuse calculations unless explicit override is specified, or we try to take flare off (you can see it is mostly additive / bias flare, with some scene-dependent flare non-uniformity); or we do all of the above.
-
I just made a little sanity check, I shot a glossy printed paper white+black patch beside eachother, 7.89 stop difference on the Y channel according to the colormunki, and I got 7.28 stop on the G channel with the camera, shot in a dark room. The CCSG has a bit less DR than my baryta paper, about 7.0 stop, so the difference should be smaller there.
So it should be possible to shoot the CCSG target a fair bit better than imaging resource does, I guess they have too much light in the room.
With DCamProf, as the complex command line tool it is, I can live with putting extra requirements on shooting glossy targets.
Concerning target design, for DCamProf's 2.5D LUT making as bright colors for a given chromaticity is the way to go. The CCSG does not have an optimal design in that regard.
-
Yes, they could have controlled the light better - and it is of course not about the amount of light per se, but about light missing the target, shielding the lens using proper hoods; cleaning the lenses, closing the viewfinder, etc. But we deal with what we have.
As to the amount of light, here is something to take into account. In studio we need profiles for the light we use for production shots. If it is with flashes, it is the same power as in production shots, or spectrum is different. Same with hot lights. So general additive flare is inevitable in such shots. The larger are the softboxes and the more diffused the light is, the more of the light misses the target and scatters back to the lens. As the actual production shot usually occupies the area larger than the target the flare and glare on the target are not characteristic to the scene. Scene needs different linearization.
SG target is suboptimal, as is CC24 as both are based on pigments that were created for quite a different set of spectral response curves (film), not having deep overlaps (like substantial red channel response to the blue-green range of spectrum) like CFAs do. But those are the only readily available. On top of that, CC24 has nothing useful for flat-fielding, even being matt it is still not quite to the challenge of profiling outside the studio.
-
There is a reason why for certain measurements Gretag used to offer polarizing filters with Spectrolino. Light is scattered back from both gloss and textured surfaces, causing flare and glare and making readings from very dark patches less reliable for semigloss and gloss substrates, and glossy black/dark pigments.
i1Pro is cost-optimized not for target measurements, but for printer profiling. IMHO it is only very logical in terms of engineering.
what are the chances that this thing will work = http://www.ebay.com/itm/381275256818 ? $50+ so I took a chance.
-
After having compared CC24 results with results of profiling directly against spectral databases I'm not particularly concerned with the spectral shapes of it. Every color can be varied in infinity in terms of spectral shapes, and it seems to me that the soft shapes the CC24 has (and also the CCSG) is a good design that works as well as one can expect. If you make the spectral shape perfectly match one color it will mismatch another. The next level is to design targets with material of what you're going to shoot, which indeed can be feasible for reproduction. Or use SSF and spectral data, but it's then only performing better for that particular set.
If we're really making a reproduction profile (a use case I'm not so much into, I made DCamProf to make generic profiles primarily), then we shouldn't really try to correct glare in the target shot, right? In that case what's important is that the shooting condition stays exactly the same, so any glare in the target shot is the same glare as when we shoot the artwork or whatever. The profile's LUT will correct the glare, if it's a 3D LUT, and it probably should be for this reason.
-
what are the chances that this thing will work = http://www.ebay.com/itm/381275256818 ? $50+ so I took a chance.
Lucky you - I missed out on one local recently...
-
Experimented more with glare reduction algorithms on the Imaging Resource 5Dmk3 shot of the CCSG. Glare reduction can indeed make significant improvements to the result, but it's still worse compared to the result I get when I simply exclude the dark patches when profiling. So from that result the recommended workflow would be to plot and check for the bad ones, create an exclude list for those and then make a new profile.
There's still a risk that the reference data file is not matching the dark patches very well though which makes the result worse than it otherwise would be. It would be interesting to have a CCSG target and measure it with a spectrometer, and then shoot it with lower amounts of glare.
I'll move on to check closer what glare reduction can do on matte targets on outdoor shots.
-
I'll move on to check closer what glare reduction can do on matte targets on outdoor shots.
Hi Anders,
You can shoot glossy targets out-door if you e.g. put them flat on the ground, but shoot them from a 45 degree angle (squaring the target is then done in software, and the resampling (if even needed) can be simple because we need uniform colored patches, not edge and other fine detail).
Since the glossy surface acts like a mirror, you need to put a sheet of black material perpendicular to the target's surface on the other side, so that only the 'black reflection' is what the camera picks up from the surface, which leaves the patch color itself relatively pure. A glossy target will pick up less ambient light reflections (because of the specular reflecting surface that only mirrors the other side at 45 degrees) than a semi-glossy or diffuse target surface (which picks up a more diffuse image of its surroundings, and thus desaturates/pollutes the colors), even with a black plane on the opposite side of the shooting angle.
These surface reflections (which can be colored from nearby walls or trees) need an additive (or subtractive) neutralization model, where as the optical vignetting and light fall-off needs a multiplicative neutralization model
Cheers,
Bart.
-
You can shoot glossy targets out-door
Really interesting technique, but I doubt that it will work well in practice. Even in an ideal indoor situation there is some residual glare issues, and there can be glare issues in the measurement too, those values 6-7 stops down are hard to get precise. Residual issues will be larger outdoor.
Semi-glossy/glossy targets contribute nothing within the normal range of colors, they only increase risk of measurement error. For super-saturated colors they do contribute a little, but I'm quite sure that the residual error in a good outdoor setup will still be large enough to make the profile perform worse on those colors than a profile based on a matte target.
I'm satisfied with the "indoor controlled light pitch-dark room only" recommendation for semi-glossy targets, and then my results indicate that you don't need glare reduction if you do it as good as possible.
When I get back home I'll continue with tests on matte targets and see what conclusions I make from those. I'm not sure I will come to the same conclusions as Iliah, but camera profiling is an "organic" approximate process so it's natural that we can come to different conclusions on how to relate to measurement issues.
My results so far indicate that glare reduction works only in a narrow window. If glare is small, glare reduction only make things slightly worse or no significant improvement, if glare is large results will improve, but results will improve even more if just excluding the dark (=most glare-affected) patches. It seems like matte targets outdoor can be an in-between case where useful results can be had.
If making a commercial easy-to-use software users expect good results even if sub-optimally used. I've seen some bad reviews of the ColorChecker SG which probably are all glare-related. So I do understand the approach of using glare reduction to do the best of what the software get. With DCamProf I have the luxury to say "sorry, junk in, junk out", so I will only include glare reduction if it can provide significant improvements in important use cases. Actually there's only one on the list as I see it -- shooting matte targets outdoor. I'll report back on results.
-
Lucky you - I missed out on one local recently...
come on - you have a shiny new monochromator setup now !
-
> After having compared CC24 results with results of profiling directly against spectral databases I'm not particularly concerned with the spectral shapes of it
In real-world photography the blue-purple shifts as well as red being too orange or too magentish are being a plague. The wrong rendition of reds touches upon yellows and causes problems in skin rendition too. In my experience it is target design issue as well as CAT issue. As I make my own targets, I can see it. One of the practical way to minimize those shifts without making new targets is profiling based on spectral response. Using spectral transforms to the latest possible stage and recording light spectrum in the scene is also one of the things I always considered better suited for profiles for photography.
Semigloss and gloss targets were appearing at the time when profiling libraries were designed with the premises that the wider is the range of colours the more accurate will be the estimation of red, green, and blue chromaticities. That started full force with CC DC, and it turned out not to work well even in studio, so Gretag made a special reference with glossy patches being removed from reference data and ignored in the device data. The other hypothesis, including patches for a LUT-oriented cube, also proved to be wrong. Another trouble is that the large white patch in the middle caused flare on the surrounding patches, and CC DC died giving place to CC SG. From the point of view of profiling, SG differs from Classic only in reflectance. But Classic also has no patches for accurate flatfielding and flare estimation. In all other aspects classic has advantages over SG even in studio. That is why we included flatfielding in RawDigger, to make Classic more usable. I always maintained that the targets need to be matt, and that in studio Classic is more practical than Passport - given one can mount the Classic geometrically flat.
I think diagnostics of flare and glare and eliminating patches with excessive amounts of those is a good working approach. Indeed, GIGO.
Profiling and linearization are different tasks, solved at different stages. I do not rely on profile LUTs for linearization (one of the reasons is that CMMs introduce too much noise and rounding errors when working with small numbers; and may even cause posterization). As to glare, I have photographic ways of reducing it, as well as object-level sampling and processing data accordingly, so in my own work glare is not an issue when it comes to profiles. In any case, flare in the final scene can't be characterized based on the shot of device-level targets like ColorCheckers being shot for profiling.
-
what are the chances that this thing will work = http://www.ebay.com/itm/381275256818 ? $50+ so I took a chance.
How could I possibly know the chances? :)
This thing is important:
How do I reset my Spectrolino?
To reset your Spectrolino, hold down the measurement button on the top of the instrument for approximately 10 seconds. During this time you will hear a sequence of four tones, then two tones in rapid succession that are repeated after a pause. Continue holding the measure button until you hear a fifth long tone, indicating that the reset is complete.
-
But Classic also has no patches for accurate flatfielding and flare estimation.
can't you DIY greyscale patches around the edges of CC Classic yourself by mounting into some frame ?
-
On a different subject, just found this: http://www.liulabs.com/Daylight.htm a D65 simulator that actually seems to work like they should work.
Unfortunately they have no put-in-basket button. I don't know what it costs, but hopefully cheaper than buying an X-Rite Spectralight viewing booth, which also is based on precisely designed filtered halogen lights.
(I'm about to give up on Solux + standard color temperature adjustment filter, the spectrum is not nice enough, and my Solux lamps are too low temperature)
LED-based daylight simulators are the future, but I don't know if there is anything affordable yet. Filtered tungsten seems to be the current best.
-
> just found this: http://www.liulabs.com/Daylight.htm a D65 simulator
I saw that, but never checked it out.
Booths are adding a lot of flare.
Not sure the precision of design of the filters is that much needed, or, better to say, is placed where it needs to be. Wide and smooth spectrum light source, like a clear sky or xenon flash, normally is enough to reconstruct spectral transmission of a camera with good accuracy.
> Filtered tungsten seems to be the current best.
That's what I'm saying, too.
-
can't you DIY greyscale patches around the edges of CC Classic yourself by mounting into some frame ?
You can, but you need to maintain thickness and use same surface materials. There were times when Gretag were selling N-type paper in sheets. Now you can cut a Kodak grey card or something similar and make a border around Classic.
-
Made some glare testing of matte targets outdoor.
Typical dynamic range loss seems to be 0.14 - 0.2 stops, reducing range from 4.8 towards 4.6 stops. The loss happens in the darkest patches. In worse cases it may rise to 0.3, and if you have very dark colors in the target you may get a small visible difference. In a CC24 with a 0.14 stop difference the visible difference with glare correction (that is linearization) enabled is to my eyes negligible. I can see on the histogram that something happens to the image when I change profile, but it's very hard to see any changes in colors in the image.
Now when I've coded and integrated the linearization, with TPS and all, I will include it in the next release. But the use case seems very narrow to me. Either you have so little glare that it makes no visible difference, or you have so much glare that it's still bad after linearization, and redoing the shot or excluding problematic patches is a better choice. The inbetween case seems to be when there's been say 0.3-0.6 stop loss in dynamic range.
-
On a different subject, just found this: http://www.liulabs.com/Daylight.htm a D65 simulator that actually seems to work like they should work.
Mailed and asked. You can buy a D65 MR-16 lamp from them directly, 299 USD + shipping and handling. I would have preferred $60 or so, but well, they truly have something unique if it performs to spec, and for that price I expect it to.
-
How bad do you think D65 fluorescents are by the way? The best seem to follow the D65 curve quite okay, but with those nasty narrow peaks in there. I don't like fluorescents for that reason, but I don't really know how big a problem those spectral spikes are.
-
Mailed and asked. You can buy a D65 MR-16 lamp from them directly, 299 USD + shipping and handling. I would have preferred $60 or so, but well, they truly have something unique if it performs to spec, and for that price I expect it to.
Interestingly, their patent (https://www.google.co.uk/patents/US7378784?dq=7378784&hl=en&sa=X&ei=S1VxVfnyE8imU7aEgMgH&ved=0CCAQ6AEwAA) quoted on the site suggests that this is really just a SPD modifying filter sandwich on top of halogen lamp.
-
Interestingly, their patent (https://www.google.co.uk/patents/US7378784?dq=7378784&hl=en&sa=X&ei=S1VxVfnyE8imU7aEgMgH&ved=0CCAQ6AEwAA) quoted on the site suggests that this is really just a SPD modifying filter sandwich on top of halogen lamp.
Yes, the principle has been known for very long time, I haven't read the patent in full so I don't know what's actually new in theirs, but I'm suspecting that the new thing is that the package is small enough so you can just clamp filters directly onto the lamp.
The Spectralight viewing booth use the same technique (but filters are away from the lamps), but liulabs product is the only one I've found where you can buy lamps separate, and 300 bucks is a bargain compared to buying a viewing booth.
The challenge is in designing the filter with a precise response to get a good spectral match. We're paying basically 300 bucks for that filter, I guess the lamp is a standard $3 MR-16.
I've experimented with Solux lamps and standard color balancing filters, and while you easily get 6500K, the spectra is not remotely close to follow a D65 shape.
Halogen can't do UV, so below 400 you don't get much output. For camera profiling this should not be a problem though.
From my experiments the main reason for getting this type of thing would be to profile glossy targets, and for me there's some added value in repeatability when developing the software. Matte targets can be profiled in free outdoor daylight without too much glare issues.
-
Just released 0.7.2.
Not too large code additions, most of the work since last release has been research and testing. There's some more stuff in the docs.
CAT is now enabled per default for reasons discussed earlier in this thread. If you prefer color constant behavior for non-D50 profiles use the "-C" flag. Previously "-C" meant enabling CAT but now that's inverted. Before 1.0 is released there may be some of these "turbulent" changes.
Linearization is now available as a part of the testchart-ff command. I won't be using it myself much as in my tests my own results gets best if I minimize glare in the first case and then just ignore any residual glare rather than applying an approximate correction.
make-profile can now write .icc and .dcp directly (then you get default settings), useful for quick trial-and-error loops when testing how the look changes with parameters etc...
-
0.7.2 build for Windows (mingw = dcamprof.exe + libgomp_64-1.dll) : https://app.box.com/s/7jsnnhl7orq1nlptk0w9k9qts107ow1f
-
I have measured some FL lights, including the top of the line ones used in booths. It was not something I would use, but here is what you may want to look at - DPR "new Studio Scene" is shot under D55 Kino Flo bank, raw files available here: http://www.dpreview.com/reviews/image-comparison
-
Interestingly, their patent (https://www.google.co.uk/patents/US7378784?dq=7378784&hl=en&sa=X&ei=S1VxVfnyE8imU7aEgMgH&ved=0CCAQ6AEwAA) quoted on the site suggests that this is really just a SPD modifying filter sandwich on top of halogen lamp.
Quite so.
-
Interestingly, their patent (https://www.google.co.uk/patents/US7378784?dq=7378784&hl=en&sa=X&ei=S1VxVfnyE8imU7aEgMgH&ved=0CCAQ6AEwAA) quoted on the site suggests that this is really just a SPD modifying filter sandwich on top of halogen lamp.
actually it is seen right on the photo of the lamp too...
(http://www.liulabs.com/daylight1.JPG)
-
actually it is seen right on the photo of the lamp too...
All you see is light and filter which does not per se lead to any conclusion. Patent reading is quite useful anyway...
-
All you see is light and filter which does not per se lead to any conclusion. Patent reading is quite useful anyway...
sure, but they also write clearly right below
"1. Incandescent lamp with colored glass filters,
2. Xenon arc lamp with colored glass filters, and "
w/o boasting anything special about the lamps, I'd assume manufacturing of the lamps themselves is a way more difficult (and $$$) industrial-wise process than procuring the right set of precut to the shape glass blanks to assemble them in layered filter for a small shop like dr. Liu
-
sure, but they also write clearly right below
"1. Incandescent lamp with colored glass filters,
2. Xenon arc lamp with colored glass filters, and "
Yes and that gives you loads of useful info ;)
I was not criticising them or anything, just making the point that it is interesting to read how it all works. Evidently its not that interesting for everyone (it was for me :))
-
It's very surprising that it hasn't come any LED solutions, other than the superduperexpensive.
I've seen a research paper when they made a good D65 simulator with just 8 LED channels (if you can solder it's probably a quite easy DIY project), you don't really need 20+ like the high end programmable have. But all products I've found are like 3 or 4 channels which means poor spectral properties.
If anyone know about a LED D65 simulator, please post.
-
Yes and that gives you loads of useful info ;)
I was not criticising them or anything, just making the point that it is interesting to read how it all works. Evidently its not that interesting for everyone (it was for me :))
so practical outcome for torger is - measure his smooth halogen light, download some databases (spectral data) of filters (organic or glass) and find a combo of 2 that will filter closest enough to D65...
-
so practical outcome for torger is - measure his smooth halogen light, download some databases (spectral data) of filters (organic or glass) and find a combo of 2 that will filter closest enough to D65...
I asked Lee filters about that, they have loads of filters, but they didn't manage to come up with a solution that works from their standard products :-\
Here's a paper showing how to build a D65 simulator based on Lee filters (there is a full text link there on the page):
http://www.degruyter.com/view/j/msr.2011.11.issue-5/v10048-011-0025-y/v10048-011-0025-y.xml
the result is not that good though, about the same as my Solux with an 80B filter
-
I've seen a research paper when they made a good D65 simulator with just 8 LED channels (if you can solder it's probably a quite easy DIY project), you don't really need 20+ like the high end programmable have. But all products I've found are like 3 or 4 channels which means poor spectral properties.
If anyone know about a LED D65 simulator, please post.
And that research project didn't have spikes anywhere in a8 channel LED spectrum? Do you have the link to the publication?
-
And that research project didn't have spikes anywhere in a8 channel LED spectrum? Do you have the link to the publication?
Desperately trying to find it again, but I haven't found it :-( I'll post if I find it.
-
I asked Lee filters about that, they have loads of filters
but you asked about _1_ filter - vs the stack of filters (2-3)...
-
Desperately trying to find it again, but I haven't found it :-( I'll post if I find it.
I suppose this is the one where (http://colour.org.uk/Li%202007.pdf) the simulator is cited but I cannot find a free version of the cited publication (the payable article can be obtained here (http://www.ingentaconnect.com/content/ist/cgiv/2006/00002006/00000001/art00049?crawler=true))
-
I suppose this is the one where (http://colour.org.uk/Li%202007.pdf) the simulator is cited but I cannot find a free version of the cited publication (the payable article can be obtained here (http://www.ingentaconnect.com/content/ist/cgiv/2006/00002006/00000001/art00049?crawler=true))
may be to write to that author ( ccdcli@leeds.ac.uk ) and ask for a copy ?
-
Yes it was that, mixed up with this: http://file.scirp.org/Html/14-7600116_16626.htm
-
Look at second sheet here http://www.graham.auld.me.uk/406/Calculating%20CRI-CAM02UCS-v2.xls
-
*Disclaimer: the product in the link below is the result of a research project I work on...so I am slightly biased.*
Here you go: http://ledmotive.com/
10 LED channels, fully programmable and with spectral feedback. Although the website shows more commercial, general illumination applications, we use the light engines in several scientific studies. Multichannel LED light sources are rarely cheap as the electronics to drive the LEDs and especially the optics to ensure an homogeneous beam profile are quite advanced.
An alternative I use myself in a homemade viewing booth are spotlights with UV-pumped white LEDs. Their spectrum is smooth and continues without the pronounced blue spike of standard white LEDs.
Keep up the good work!
Wim
-
Thanks for the interesting link.
Indeed I was suspecting that it may be quite expensive to make a multichannel LED, more than those 300 bucks for a filtered halogen.
Could you provide a link to a UV-pumped white LED? I have never heard of them before.
Surfed a bit, the principle seems to be to have a UV led (producing invisible ultraviolet light) and then put in some phosphors there that absorbs the UV an re-emit at a smoother wider spectrum. This links shows what to expect:
http://www.open-photonics.com/featured-technologies/high-cri-leds
Better than fluorescents it seems, but we can't have D65 I guess?
-
http://yujiintl.com/high-cri-led-lighting/?gclid=CM6ut6-v-cUCFTHHtAodoGkAyw
Filtered halogens could be more accurate, but the MR16 halogen replacement I tested offered a much higher illuminance and better beam profile than Solux (not to mention they consumed 10x less power). Their mid-power SMD LEDs are excellent!
-
http://yujiintl.com/high-cri-led-lighting/?gclid=CM6ut6-v-cUCFTHHtAodoGkAyw
Filtered halogens could be more accurate, but the MR16 halogen replacement I tested offered a much higher illuminance and better beam profile than Solux (not to mention they consumed 10x less power). Their mid-power SMD LEDs are excellent!
Cool, I had just surfed up that exact site and was about to post a link :). A package for 3 5600K lamps is $125 though, so it's not for free. Cheaper than Liulabs $300 D65 simulator though.
(For the application of camera profiling I think a directional spot is a good thing, easier to get nothing but the target lit to minimize any glare issues.)
-
Hmm, I bought a couple of MR16 spots from them about a year ago. Maybe worth asking if they still have them?
-
These guys also have this kind of thing, http://high-cri-led.com/high-cri-led/led-lighting/high-cri-mr16.php
their latest product brochure was released today, and it has some 6500K products in it, on the web I find only 5600K max though.
-
Interesting! I will contact them and see if I can get some more info. They seem to use the standard blue pump + phosphors to achieve a high CRI. This means the spectrum most likely has a large blue spike at 450-460nm...not something you's want. Let's see.
-
Experimented with some outdoor shooting techniques to minimize glare and flare issues.
Setup as shown in attached image; cut out the side of a cardboard box, cover it with black cloth, and put the glossy target at the bottom. Then shoot the target at an angle from the side, so all reflections is supposed to come from the blackness of the box. You need perspective correction to make the target scannable, and you also need to use the flat-field correction function to even out the light.
I have also lengthened the lens shade. The standard lens shade is designed to not vignette on the largest aperture, so when you shot at f/8 you can have a considerably longer lens shade and reduce any flare. The viewfinder is made light tight (important!). I'm using a 85mm lens as it has few lens elements which is good concerning flare.
I'm not shooting a real target here, just a glossy printer target which has both black and white patches so I can test dynamic range.
7.89 stops: spectrometer reference measurement
7.28 stops: target shot in a dark room with a tungsten light (note: not same lens used here, possibly a little bit better result can be had)
7.19 stops: outdoor with the shown setup using polarization filter
7.15 stops: outdoor with the shown setup using no polarization filter.
6.7 stops: outdoor as shown but without the cardboard box
6.3 stops: outdoor with the shown setup, being sloppy with the viewfinder
Polarization filter does not seem to make any significant difference in this type of setup, so it's probably best not using it as I think they're not fully spectrally flat.
It seems like it may be possible to shoot semi-glossy targets outdoor and get good results using this type of technique, if you're careful when setting it up. At some point I'll test it in full.
-
All,
I have followed this discussion from its start and am constantly amazed at what the internet allows a few motivated people to do. I just love these kinds of globehopping collaborative projects.
In reading the latter part of the thread and the problems with flare and glare, a question keeps popping into my head; Why do you need to measure a paper target? Why not take a variable light source like the LEDMOTIVE or a homemade alternative (RGB+ LEDs and a ColorMunki?) point it at the camera and take photos as you vary the light? I guess it would look more like a version of display profiling. On that thought, maybe instead of LEDs it could even use a display for the source?
Andy
-
There are two ways that are currently in use; one is to use bandpass filters over a halogen or xenon stabilized light; the other is to use a target made of gel filters or an IT8 / HCT (http://hutchcolor.com/) target recorded on a slide film and positioned and spectrally measured right on the lighttable (a more stable variation of the same is to use single filters mounted into slide frames in a multiple shot technique, lighttable or slide projector). Light source must have smooth spectrum, and I have not yet seen LEDs that provide such spectrum.
-
All,
I have followed this discussion from its start and am constantly amazed at what the internet allows a few motivated people to do. I just love these kinds of globehopping collaborative projects.
In reading the latter part of the thread and the problems with flare and glare, a question keeps popping into my head; Why do you need to measure a paper target? Why not take a variable light source like the LEDMOTIVE or a homemade alternative (RGB+ LEDs and a ColorMunki?) point it at the camera and take photos as you vary the light? I guess it would look more like a version of display profiling. On that thought, maybe instead of LEDs it could even use a display for the source?
It's probably harder and more expensive than we'd like to make a home-made programmable multi-channel LED light-source. I'm all for such a project though, but I'm not the right person to do it.
Targets should present relevant types of spectra. Homemade inkjet printed targets have some limitations discussed earlier, X-rite's targets have better spectra, but I think you can use a inkjet super-saturated target as a complement to X-Rite's matte CC24 for example. If you do copy work, such as copy paintings, making an own target by painting patches with the same colors as used in the work you copy will be a good idea.
The core of the problem is that cameras are not colorimetric devices, their SSF do not match the standard observer CMF, that is they register colors in a significantly different way than humans. This means that to make a profile that works as good as it can, we need to make it for the exact reflectance spectra that will be in the scenes we shoot. Unless we do copy work this is not really possible, and then we instead need to do it for some average spectra, then smooth rounded spectra are much better than spiky spectra as it's more similar to natural colors and works smoother over a wider range of similar colors.
Unfortunately displays are very limited in terms of spectra, it's just a red, green, and blue peak which you can mix. This can recreate tristimulus values that work well for a human, but as the camera sees colors otherwise those spectra will not make up for good profiling colors.
It's the same reason why we're so picky about the artificial light sources. The mismatch between human and camera color response makes spiky spectra unpredictable. If you have photographed in low CRI light sometime you have probably noticed that the colors come out quite different from the camera than the eye saw them, and this is a demonstration of the effect of having different color response. The spikier spectra the more the differences are seen. Therefore we want to do camera profiling with full spectrum lightsources with smooth spectra, using colors with smooth wide reflectance spectra.
Using transmissive targets is interesting though. In Image Engineering's CamSpecs product they use transmissive targets with several narrow bands to measure the camera's SSF. And I think if one is going to make some elaborate DIY project with hardware stuff, I think one should try to look into measuring SSF.
-
> Using transmissive targets is interesting though
And much simpler than one may think. Bonus - they can be measured as mounted on the light, right before shooting, with a German-made, half-the-PR-price JETI device http://www.jeti.com/cms/index.php/instruments-55/radiometer/specbos-1201 ; and additional focus optics is available if the patches are less than 4mm. This makes cheap gel filters very practical, and allows to virtually eliminate scene flare.
-
half-the-PR-price JETI device http://www.jeti.com/cms/index.php/instruments-55/radiometer/specbos-1201 ;
that is still several $Ks...
-
> that is still several $Ks...
Yes, but it has no useless bells and whistles; while for a regular user i1Pro is ok in transmissive setups.
-
I have been involved in modifying a drum scanner (Howtek) in a project over at the large format forum. I started out alone with better PMTs and amps and then worked with others who generously helped to fix some of the issues that came up that were out of my "comfort zone". One of the guys I was working with is using a RGB led as his source now instead of the halogen lamp that was originally used. He is reporting good results and is technically sharp enough that I believe him.
Of course, a bayer array and a drum scanner really work on the same principle, its just that one is two dimensional and one is a point. This is what led me to believe that a more direct approach might work.
Andy
-
The difference is that film spectral transmission allows for relatively narrow spectrum light sources (see colour separation filters), while the response of a camera sensor does not.
-
> that is still several $Ks...
Yes, but it has no useless bells and whistles; while for a regular user i1Pro is ok in transmissive setups.
so Torger can still find a use for his solux lamp by sticking gel filters along with a filter to raise CCT up on top of it and using his colormunki to measure the light directly and then shooting it directly with a camera in M mode to keep exposure the same... or alternatively find a way to mount a gel in front of styrofoam and measure the illuminated styrofoam (tele mode with colormunki) and then shoot it too, gel by gel... flare, etc will not matter much there too... now you just need to find a minimal set of gels to get results
-
Using transmissive targets is interesting though. In Image Engineering's CamSpecs product they use transmissive targets with several narrow bands to measure the camera's SSF. And I think if one is going to make some elaborate DIY project with hardware stuff, I think one should try to look into measuring SSF.
It would be less expensive to build proper monochromator setup to do that. I have completed mine (it needs to be calibrated) and overall it cost less than the 6500K light you were looking for. Used monochromator (from manufacturing like Luxtron/Lumasense/Xinix - like this one for example (http://www.ebay.com/itm/Monochromator-XINIX-Lumasense-Luxtron-Model-1108-Microscopy-Spectography-Testing-/301592903145?pt=LH_DefaultDomain_0&rmvSB=true&nma=true&si=W7DyBAIXW7srNNGZOow9wjIiayI%253D&orig_cvip=true&rt=nc&_trksid=p2047675.l2557)), used halogen light source with fiber optic feed (plenty on eBay) and used integrating sphere (even a simple one like this (http://www.ebay.com/itm/PHOTONIS-XP3112-PB-TUBE-HOUSING-UV-filter-integration-sphere-/271166072389?pt=LH_DefaultDomain_0&rmvSB=true) will do). Also a way to measure illumination level in a sphere - arduino + photodiode + 30mins of coding. All of that put together on a bench in a dark room gives a decent enough setup to measure SSF.
-
It would be less expensive to build proper monochromator setup to do that.
That's super-cool, I'd love to see some photos when you have it all working!
-
> It would be less expensive to build proper monochromator setup to do that.
I doubt "less expensive" part (unless we are chasing D65, which even an advanced user will not be doing, and which is unnecessary as obtaining spectral response is a much better way to do things), but much more precise, much more stable, and much more convenient if a lot of cameras and lenses are to be characterized.
Light spheres are easy DIY project.
-
That's super-cool, I'd love to see some photos when you have it all working!
Sure - I'll make the setup more permanent first and will do a calibration run (with spectrophotometer and photodiode measurements to make a correlation table). I have not got much spare time lately so it all goes a bit slow at the moment. At the beginning of the week I completed light source and put it all together on a table in a dark room for a quick run - walked through the spectrum with i1Pro measuring every 10nm (thanks Graeme for Android version to make it really convenient with the phone) and all seems to work fine.
-
> It would be less expensive to build proper monochromator setup to do that.
I doubt "less expensive" part (unless we are chasing D65, which even an advanced user will not be doing, and which is unnecessary as obtaining spectral response is a much better way to do things), but much more precise, much more stable, and much more convenient if a lot of cameras and lenses are to be characterized.
I meant less expensive when constructed from used parts off eBay - getting all parts shiny new is prohibitively expensive for home enthusiast. My setup was half the price of new Colorchecker SG in total.
Light spheres are easy DIY project.
I did construct a sphere using paint and was about to attempt coating it with barium sulfate when a cheap one came along made with spectralon.
-
>My setup was half the price of new Colorchecker SG in total.
That is what I would expect :) But wait for calibration before celebrating ;)
All in all, you know I was suggesting this setup for quite some time.
-
>My setup was half the price of new Colorchecker SG in total.
That is what I would expect :) But wait for calibration before celebrating ;)
All in all, you know I was suggesting this setup for quite some time.
I am far from celebrating - the real experiments are only just beginning...
-
As many seems to be reading this thread it may be worth mentioning that spending lots of money to get the perfect D65 simulator "just because" it's not a wise thing to do.
When you make dual-illuminant DNG profiles for general use the defacto standard (=Adobe standard) is indeed to make them for the StdA and D65 pair. This means that in order for the EXIF lightsource tags to match the truth you need to profile under those illuminants. However, they don't really need to match. The only thing they're used for is when deciding the matrix mix, ie if the temp is 4700K 50% of the 2850K StdA matrix will be mixed with 50% of the 6500K D65 matrix. To know the mix you need to know the temperature and to now the temperature you need to know the mixed matrix, a hen and egg problem solved by an iterative loop. But only after mixing the CM is used to derive the actual temperature/tint that is shown in the white balance tool.
To summarize, you can mismatch the actual calibration illuminant and its tag with quite wide margin before it will have any real effect. Therefore it's perfectly feasible to use real daylight as "D65" even if you make a dual-illuminant profile. (On single illuminant profiles the EXIF lightsource tag has no effect at all. ICC profiles are always single illuminant.)
Having a simulator adds convenience though, especially if you make many profiles or experiment a lot. It would not need to be a D65 simulator though, you could have some other average daylight.
It's still some advantage to *know* what light that has been used as calibration illuminant, if not the spectra at least the temperature. This is needed to make the DNG profile estimate light temperatures properly. (The CAT is also affected by that, but you can of course enable color constant behavior)
I have a hunch that for dual-illuminant profiles it's an advantage to get a bit above 5000K to get some more distance from StdA. At D65 we have very limited options on artifical light sources with nice spectral properties though. The Liulabs expensive filtered halogen does not have much competition. I'm curious about the UV-pumped white LEDs though, seems like Yuji is about to release one. With those it seems to be that the higher temperature the worse spectrum, so I'm not so sure the 6500K will look that good, but probably better then a Solux with 80B filter.
-
> I am far from celebrating
Not that far...
> - the real experiments are only just beginning...
That never ends ;)
-
> contrast using XYZ is something like log2(Y from white patch / Y from black patch), right ?
Contrast is Y1/Y2; but if you want it to convert to photographic stops, log2(Y1/Y2) or log2(Y1)-log2(Y2). For counter-flare splines, I use just contrast, not log2
I have a question... I flatfielded rawdigger samples from a CC24PP shot and I am getting the following (no WB, no scaling, no gamma/gamma = 1) :
19 D1 5442.64 13193.07 8222.41
20 D2 3414.98 8434.33 5383.07
21 D3 2123.34 5291.81 3394.74
22 D4 1127.10 2778.43 1783.99
23 D5 515.34 1304.47 855.50
24 D6 179.69 439.28 287.96
so contrast is (in stops) :
R = log2( 5442.64 / 179.69 ) = 4.920724585
G = log2( 13193.07 / 439.28 ) = 4.908495689
B = log2( 8222.41 / 287.96 ) = 4.835620983
CC24PP spectral data converted to XYZ / Y :
D6 = 3.128
D1 = 92.588
log2 (92.588/3.128) = 4.887512805
comparing with flatfielded RD it is too good to be true, so where is the error might be then ?
FFD data :
19 D1 0.997140 0.999829 0.999334 0.998824
20 D2 0.997535 0.999202 0.999851 0.999649
21 D3 0.997811 1.000000 0.999594 1.000000
22 D4 1.000000 0.999567 1.000000 0.999630
23 D5 0.994782 0.993781 0.993994 0.996559
24 D6 0.975756 0.977245 0.975554 0.977036
unflatfielded samples
19 D1 5427.07 13184.19 8216.93
20 D2 3406.56 8429.48 5382.27
21 D3 2118.70 5291.81 3393.36
22 D4 1127.10 2777.32 1783.99
23 D5 512.65 1298.17 850.37
24 D6 175.34 429.24 280.92
scene is :
(http://s17.postimg.org/xl20jn5wf/DSC06299_OK.jpg)
-
A quick question to any Yuji high CRI LED owners out there, it would be nice if you could post a spectrum measurement, for example of the 5600K. The "marketing" spectrums shown on web sites often make things look better than they really are...
-
Frankly, I do not see much marketing looking at the 5600K SPD on page 7 of http://www.yujiintl.com/files/high-cri-led-datasheets/VX5730-YUJI.pdf
It is not pretty for the purpose of D-series simulation for profiling. Visual comparison with a D-series SPD is not very telling, it is close to what Dr.Fairchild calls "A Metric for the Aesthetic". Kino-Flo Celeb lights are made of Yuji LEDs. An independent and reputable study of different light sources (including Celebs http://www.gtc.org.uk/media/fm/TLCI_charts_April_2015/Kino_Flo.zip ) by Alan Roberts (former BBC research engineer) is at http://www.gtc.org.uk/tlci-results/tlci-results-new-format.aspx (SPDs are presented as TIFFs, you need to download files to see those).
-
CC Classic was designed to cover 5 stops, flare and glare at this dynamic range is negligible. Try underexposing it by 2-3 stops and see how the error increases.
-
I've discovered some silly bugs in v0.7.2, CAT is not run when it should and opposite, and flatfield not working properly. I will soon release a patch to fix those. I'll do a little bit more testing first.
-
CC Classic was designed to cover 5 stops, flare and glare at this dynamic range is negligible. Try underexposing it by 2-3 stops and see how the error increases.
but it did not much, took EV-3 (vs the raw in the post above) exposed raw from the same series and
19 D1 673.49 1642.2 1017
20 D2 423.69 1044.29 666.82
21 D3 263.83 654.65 421.1
22 D4 140.29 344.66 221.82
23 D5 64.214 163.55 107.25
24 D6 22.477 55.471 36.186
R = log2( 673.49 / 22.477 ) = 4.905135127 vs 4.920724585
G = log2( 1642.2 / 55.471 ) = 4.887752297 vs 4.908495689
B = log2( 1017 / 36.186 ) = 4.812744228 vs 4.835620983
drop by 0.02
and still strange vs measured target log2 (92.588/3.128) = 4.887512805
-
but it did not much, took EV-3 (vs the raw in the post above) exposed raw from the same series and
19 D1 673.49 1642.2 1017
20 D2 423.69 1044.29 666.82
21 D3 263.83 654.65 421.1
22 D4 140.29 344.66 221.82
23 D5 64.214 163.55 107.25
24 D6 22.477 55.471 36.186
R = log2( 673.49 / 22.477 ) = 4.905135127 vs 4.920724585
G = log2( 1642.2 / 55.471 ) = 4.887752297 vs 4.908495689
B = log2( 1017 / 36.186 ) = 4.812744228 vs 4.835620983
drop by 0.02
and still strange vs measured target log2 (92.588/3.128) = 4.887512805
Nope, not that way ;)
8 EV - log2(13193.07/55.471) for G - about 0.11 EV
Always disliked the practice of shooting targets on complex backgrounds instead of dark grey and black; having foreign objects in the scene (even if they are not showing in the frame); semi-gloss or gloss targets (SG has 5.5 stops between the "same" 2 patches, and that already breaks IR shots), shallow hoods or no hoods...
-
Nope, not that way ;)
8 EV - log2(13193.07/55.471) for G - about 0.11 EV
that is if you are using your approach of bracketing shots and combining target measurements with data (spectral or otherwise) adjusted to match underexposed raws... than naturally means using the data from D1 from the EV0 and the data for D6 from EV-3...
then measured target will be (using Y from spectral data divided by 2^3) log2( 92.588 / 0.391 ) = 7.887512805 and then is not 0.11EV, but ~zero, no ? unless you state that i1pro2 can't even measure ColorChecker Passport black patch (not CCSG) correctly...
-
> then measured target will be (using Y from spectral data divided by 2^3) log2( 92.588 / 0.391 )
I disagree with 0.391. Measured Passport with my i1Pro, Y on black patch is 0.31
-
> then measured target will be (using Y from spectral data divided by 2^3) log2( 92.588 / 0.391 )
I disagree with 0.391. Measured Passport with my i1Pro, Y on black patch is 0.31
you mean you disagree not with Y = 0.391, but with Y = 3.128 from measurements ( 0.391 was derived from Y = 3.128 when I divided it by 8 = 2^3 stops to match EV-3 exposure )... well , I averaged 3 measurements a week ago (attached) - but I will repeat it again.
-
I have Y = 94.79 for the white, Y = 3.1 for black, log2(94.79/3.1)+3 = 7.9344
However, in practical photography any variation below 1/6 EV (0.17 EV) is considered irrelevant. The best shutters are guaranteed to be within 1/12 EV, apertures within 1/10 EV accuracy and repeatability.
-
The best shutters are guaranteed to be within 1/12 EV
by the way - it sounds logical that exposure time component of exposure shall be more precise with longer exposures, no ?
-
Just released a quick update 0.7.3 out to fix some bad bugs introduced in 0.7.2
-
0.7.3 build for Windows (mingw = dcamprof.exe + libgomp_64-1.dll) : https://app.box.com/s/bh3zagil9imxff5h3voxozgtgcwzmupa
-
Been working with some elaborate workflows, seems to be one feature left to do before I can get profiles that look as they should; somehow one need to take the film curve into account. The commercial profiles do, and even some matrix-only profiles seems to be deliberately desaturated to not oversaturate when a film curve is applied. On the other hand it may just be a side-effect of using only matte targets when profiling which typically seems to lead to a slightly under-saturated profile.
This has nothing to do with reproduction style profiles which are designed to look accurate with no film curve applied. Not sure if it's suitable to rely on standard color science models at all in this case.
I think one can let highlights desaturate with the lowered S-curve contrast, but maybe one want to keep saturation in shadows, if so I need to introduce a 3D look LUT. Possibly it will look good if just compensating for the increased midtone contrast all over, that's probably what I will try first.
Adobe has a slightly different curve formula than a pure RGB curve, may need to take that into account too. If we only look at linear midtone contrast we won't though as it then is no different from a pure RGB curve.
-
Some measured spectra of the Yuji LEDs and LEDMOTIVE multichannel source:
1. High CRI blue-pumped white LED (CCT = 5800K)
(http://s3.postimg.org/6fdv475dv/Blue.jpg) (http://postimage.org/)
2. Violet-pumped white LED (CCT = 5000K)
(http://s3.postimg.org/4q4rw4poj/Violet.jpg) (http://postimage.org/)
3. Violet-pumped LED, different phosphors (CCT = 5600K)
(http://s3.postimg.org/i5rslky6b/MR16.jpg) (http://postimage.org/)
4. LEDMOTIVE source emulating D65 spectrum (CCT = 6504K)
(http://s3.postimg.org/t2t4ackxv/Ledmotive_D65.jpg) (http://postimage.org/)
Hope this helps!
Wim
-
Thanks! They're not really UV-pumped it seems, but on the edge of the visible range, where both the eye and a camera has some sensitivity left. Not sure how much it will disturb though.
-
manual has a small typo error - here is a piece of text :
Here follows a description of each command available.
make-target
dcamprof make-profile <flags, with inputs> <output.ti3>
Make a target file which contains raw camera RGB values paired with reference XYZ values, and (optionally) spectral reflectance. The file format is Argyll's .ti3, with some DCamProf extensions.
-
> They're not really UV-pumped it seems, but on the edge of the visible range, where both the eye and a camera has some sensitivity left. Not sure how much it will disturb though.
If it does not cause response in the red channel it does not matter much, not adding to metameric failures. But at least with Nikon cameras based on Toshiba sensors it does.
-
Thanks! They're not really UV-pumped it seems, but on the edge of the visible range, where both the eye and a camera has some sensitivity left.
Might make for more fluorescent whitener response than a "real" light source.
-
Experimented with some different methods to adjust saturation when using the profile with a film curve. Adjusting chroma of the XYZ reference values via CIECAM02 Jab/JCh seems to work well. JCh is more linear concerning hue (that is less problems of changing hue when chroma is scaled) and overall more perceptually uniform than Lab/LCh, and a suitable adjustment seems to be in the 1-2 units range, that is very small, so hues stay stable enough. I was thinking of making a Munsell corrected space (like Bruce Lindbloom's UP Lab http://www.brucelindbloom.com/index.html?UPLab.html), but as adjustments are so small and often with more saturated colors than exist in Munsell's book of colors it's not worthwhile, and as said Jab is already a bit better than Lab.
Instead of adjusting the XYZ reference values the other alternative is to apply the curve on the RGB values, but then matrix fitting becomes badly nonlinear and I don't think the result is better. Just like the curve itself this type of adjustment is more about look-and-feel than accuracy.
I'll include possibility to adjust chroma it in the next release, and voilà DCamProf has its first subjective look-and-feel adjustment.
This is a good read regarding perceptually uniformity of various color models by the way: http://www.handprint.com/HP/WCL/color7.html
-
any ideas how to come up with some good heuristic to speed up (or rather try in a proper directions) trying possible combinations within 28 measured SSFs to "calculate" the best "virtual" SSF for my camera ? a simple brute force is too slow because calling dcamprof executable hundreds of thousands times (to test how SSF profile generated by Matlab matches /dcamprof test-profile/ the real measured and shot target) is not an option...
so far brute force (I can only try all combo up to 5 cameras out of 28 on my notebook in several hours) gives me the around the following match :
(http://s11.postimg.org/dt49bqnn7/match.jpg)
not bad, but I want better.
dcamprof test-profile ("dcamprof test-profile -i ill.sp -C -B ccpp.ti3 ssf-profile.json", where ccpp.ti3 has real/actual raw RGB and real/actual measurements) gives for this profile :
Native LUT patch match
average DE 0.67, DE LCh 0.25 0.35 0.44
median DE 0.62, DE LCh 0.27 0.39 0.38
p90 DE 1.24, DE LCh 0.45 0.60 0.95
max DE 1.31, DE LCh 0.68 0.79 1.23
results of conversion in ACR (process 2010, etc, with WB off D2 patch ) are naturally worse than math and not matching dcamprof test-profile 1:1
-
Nelder-Mead simplex can do such stuff, but it requires lots of finetuning of error function and start values to get good results.
It would be interesting to integrate an SSF estimation function into DCamProf but it's probably not happening for at least a few months. I'm planning to code less during the summer which is starting here now :-)
-
Nelder-Mead simplex can do such stuff, but it requires lots of finetuning of error function and start values to get good results.
It would be interesting to integrate an SSF estimation function into DCamProf but it's probably not happening for at least a few months. I'm planning to code less during the summer which is starting here now :-)
what might be the optimization parameters, I am using max dE as the top one, then with max dE being equal I am thinking shall I consider an average/mean dE or go for maximum chromacity error (not paying attention to luminocity errors) ?
-
what might be the optimization parameters, I am using max dE as the top one, then with max dE being equal I am thinking shall I consider an average/mean dE or go for maximum chromacity error (not paying attention to luminocity errors) ?
For Nelder-Mead to find a good minimum it needs a quite "well-behaving" error function, if it's too random it's hit and miss. That is if it changes a parameter it must see some sort of trend if it got better. Using max dE may be worse than mean for this reason. I don't have any general good guide though, trial and error is the method I've used.
-
How could I possibly know the chances? :)
This thing is important:
How do I reset my Spectrolino?
To reset your Spectrolino, hold down the measurement button on the top of the instrument for approximately 10 seconds. During this time you will hear a sequence of four tones, then two tones in rapid succession that are repeated after a pause. Continue holding the measure button until you hear a fifth long tone, indicating that the reset is complete.
so I shall find out tonight... whether I can measure the black patches on CCSG or not
(http://s23.postimg.org/liwxr2e7v/20150609_125138.jpg) (http://s23.postimg.org/5n83nrnnf/20150609_125254.jpg)
-
Just released 0.7.4. A very small update adding the subjective chroma adjustment possibility.
-
I'm working on a camera profiling article, still much work left, but here's a preview of an elaborate workflow to make a Dual-Illuminant DNG profile based on shots of a CC24 and a custom home-made inkjet printer target made on semi-glossy paper.
It also show a little how I work iteratively using plots as guides. I also use RawTherapee for visual evaluations when comparing the look of profiles.
# 1. Make a custom target to top of the CC24 with some super-saturated colors
dcamprof make-testchart -l 15 -d 14.46,12.26 -O -p 210 custom-target.ti1
printtarg -v -S -iCM -r -h -T300 -p A4 custom-target
# 2. Print custom target to a semi-glossy OBA-free paper
# 3. Scan the target with a spectrometer
chartread -v -H -T0.4 custom-target
# 4. Create reference file
spec2cie -v -i D50 custom-target.ti3 glossy.cie
# 5. Setup light and measure spectrum, save to light.sp for later use
spotread -a -H -x
# 6. Shoot CC24 target, glossy target, and white card.
# 7. Crop export and convert to cc24.tif, glossy.tif and ff.tif
# 8. Apply flatfield
dcamprof testchart-ff cc24.tif ff.tif cc24-ff.tif
dcamprof testchart-ff glossy.tif ff.tif glossy-ff.tif
# 9. Scan values
scanin -v -dipn cc24-ff.tif ColorChecker.cht cc24.cie
scanin -v -dipn glossy-ff.tif custom-target.cht glossy.cie
# 10. Merge targets into one, letting CC24 have priority
dcamprof make-target -p cc24-ff.ti3 -a cc24 -p glossy-ff.ti3 -a glossy -d 0.03 combo.ti3
# 11. Make a bunch of preliminary profiles to evaluate saturation
when used with a film curve. Possibly you will want to
desaturate the profile a little. Do the evaluation visually
and subjectively in a raw converter.
dcamprof make-profile -i light.sp combo.ti3 chroma.json ; dcamprof make-dcp -t acr chroma.json chroma-0.dcp
dcamprof make-profile -i light.sp -k -1 combo.ti3 chroma.json ; dcamprof make-dcp -t acr chroma.json chroma-1.dcp
dcamprof make-profile -i light.sp -k -2 combo.ti3 chroma.json ; dcamprof make-dcp -t acr chroma.json chroma-2.dcp
dcamprof make-profile -i light.sp -k -3 combo.ti3 chroma.json ; dcamprof make-dcp -t acr chroma.json chroma-3.dcp
dcamprof make-profile -i light.sp -k -4 combo.ti3 chroma.json ; dcamprof make-dcp -t acr chroma.json chroma-4.dcp
# 12. Using the found chroma adjustment make a preliminary
# profile, dumping plots. Exclude glossy from the matrix
# optimizer to get as good base match as possible for the
# important normal colors represented by cc24
dcamprof make-profile -r dump1 -w cc24 0,1 -w glossy 0,0 -i light.sp -k -1.5 combo.ti3 preliminary.json
# 13. Start gnuplot (cd dump1; gnuplot -background gray) and plot
# target and LUT, plus LUT stretch vectors and DE vectors.
# Use 'set view equal xyz' and 'set view equal xy' to turn
# on/off scaling of lightness axis, must be turned off if
# error vectors are viewed in 3D.
gnuplot> splot 'nve-lut.dat' w l lc "beige", 'gmt-locus.dat' w l lw 4 lc rgb var, \
'gmt-adobergb.dat' w l lc "red", 'gmt-pointer.dat' w l lw 2 lc rgb var, \
'target-nve-lutve2.dat' w vec lc "black", 'target-nve-lutvm.dat' w vec lw 2 lc "olive", \
'targetd50-xyz.dat' pt 4 lc rgb var, 'targetd50-xyz.dat' using 1:2:3:5 w labels offset 3
# 14. Look in the plot for patches that pull in opposite directions and cause a
# bad bend in the LUT. Add those (typically one or two) to an
# exclude.txt and render
dcamprof make-profile -r dump1 -x exclude.txt -w cc24 0,1 -w glossy 0,0 -i light.sp -k 1.5 combo.ti3 preliminary.json
# 15. Make matrix-only and full correction profiles for sanity
# check comparisons later
dcamprof make-dcp -n "Canon EOS 5D Mark II" -t acr preliminary.json no-relax.dcp
dcamprof make-dcp -n "Canon EOS 5D Mark II" -t acr -L preliminary.json matrix.dcp
# 16. Relax the LUT (primarily for the glossy class) to improve
# smoothness. Re-render and replot for each change.
# a) It may be worthwhile to lock the matrix before changing DE k
# weights: save profile to separate file matrix.json and
# provide -m and -f parameters.
# a) Try relaxing DE weight even for the important CC24, setting
# it to at least 1 leads often to some relax without much loss
# in accuracy.
# b) Try changing CIEDE2000 k weights, 4,1,1 good start (less
# weight on lightness)
# Example result after iterating:
dcamprof make-profile -r dump1 -x exclude.txt -f matrix.json -m matrix.json \
-w cc24 2,1,4,1,1 -w glossy 4,0,4,4,1 -i light.sp -k -1.5 combo.ti3 final-1.json
# 17. Make final DCP, sanity check it by comparing it with
# no-relax.dcp and matrix.dcp. It should have better high
# saturation correction than matrix.dcp, and not lose too much
# accuracy compared to no-relax.dcp
dcamprof make-dcp -n "Canon EOS 5D Mark II" final-1.json final-1.dcp
# 18. Repeat steps 5 - 17 for the second illuminant
# 19. Merge to a dual-illuminant profile, here using a default
# film curve
dcamprof make-dcp -n "Canon EOS 5D Mark II" -t acr final-1.json final-2.json final-dual.dcp
-
0.7.4 build for Windows (mingw = dcamprof.exe + libgomp_64-1.dll + PDF manual / copy of Torger's blog) : https://app.box.com/s/ujo64khq6blhsxpxgmxcczg8jm19vtiz
-
so I shall find out tonight... whether I can measure the black patches on CCSG or not
Swiss engineering... cables !!!
(http://s23.postimg.org/6591hdlgr/gm3.jpg)
it seems to be working... __HOWEVER__ my CC SG black patches are still reading in line with i1pro2 data... which is L ~10-11... what that might be now ? do I have defective Spectrolino ? light trap it reads many times perfectly 0.00xxx for L/a/b (more consistent than i1pro2, but then it fits the light trap hole more tightly)
-
It was designed for old Macs, with round serial (modem/printer port on PowerMacs, pre-G3; page 69, bottom drawing of http://www.xrite.com/documents/literature/gmb/en/spectrolino_serial_5_en.pdf - get it before it is gone, it contains serial protocol commands). Makes for 1 data cable and a power cable through "T-adapter" (power injector). Mine is a bit later model, so-called "purple", and power injector is combined with serial, does not need T-adapter.
L*=10..11 instead of L*=6..7 is seriously high. Is the target clean? Any dust in Spectrolino (on your shot the measurement head was without a "N" filter)?
-
http://www.xrite.com/documents/literature/gmb/en/spectrolino_serial_5_en.pdf - get it before it is gone, it contains serial protocol commands).
thank you for the link !
Makes for 1 data cable and a power cable through "T-adapter" (power injector). Mine is a bit later model, so-called "purple", and power injector is combined with serial, does not need T-adapter.
the strange thing is that while Mac "T-cable" (that black thing) is working for power connection, PC "T-adapter" (silver box labeled "Gretag Adapter Mini-DIN Part 36.19.58") is not (it passes the data, but not taking in the power, so I have to use the one for Mac = hence so many connections)
L*=10..11 instead of L*=6..7 is seriously high. Is the target clean?
it is - at least black patches are, those were the first thing I measured when I switched the whole thing on (I did that 10 sec pressing button reset first with Spectrolino with "U" ("N") filter and on a calibration tile)
Any dust in Spectrolino (on your shot the measurement head was without a "N" filter)?
if not -how to clean the lens (w/ filter off it) ? just with a soft cloth ? no isopropyl alcohol ?
-
I fully measured CC Passport so far (attached for posterity)
1) D6 patch has "Y" = 3.116 - which is close to the number you mentioned above in this topic
2) measurements, it seems to me, are very consistent between themselves (I did 5 in a row, calibration before each, and averaged) - seems better than with i1pro2 (but again that might be an operator error when I was using i1pro2 for Passport)
*** done, 1.32 -> 1.68 *** 3) it seems that it is possible to upgade firmware for Spectrolino, my is from 1998 and v1.32 (firmware)
-
Things I use to clean:
http://www.edmundoptics.com/lab-production/cleaning/compressed-air/dust-off-compressed-air/1902/
http://www.edmundoptics.com/lab-production/cleaning/gloves-brushes-swabs/cotton-tipped-swab-applicators/2922/
http://www.edmundoptics.com/lab-production/cleaning/lens-cleaners-pouches/edmund-lens-cleaner/1367/
Since Passport was measured within the specs as it seems, the measurements of the SG target may be wrong because the target is out of specs.
-
Since Passport was measured within the specs as it seems, the measurements of the SG target may be wrong because the target is out of specs.
seems strange that spectrophotometer manufacturers always include a white calibration tile , but not something like a "black" calibration tile... so what might be the way to find the truth ? why reading light trap (which gives something like 0.0x ) does not answer the question if the device functions properly ? I took a piece of sufficiently black velvet and Spectrolino reads it as L = 7.08 and it visually looks to me way darker than black patches on my CCSG, but then it is not even semiglossy
-
Spectrophotometer calibration tiles are very expensive, a set can easily be more than a grand. It is cheaper to send the device to a calibration service (not necessarily X-Rite). When you read black trap there is no stray light, so the only thing you are getting is that the noise is sufficiently low. Velvet allows for some light from the sides to get to the spectrophotometer. You may also want to e-mail X-Rite with the question - why the black patches are read abnormally high. Maybe they will replace your target, or say something useful. And of course try cleaning.
-
Velvet allows for some light from the sides to get to the spectrophotometer.
so I switched all lights in the room and it's past midnight in rural area -> result is XYZ: 0.732348 0.748855 0.654355, D50 Lab: 6.764379 0.415899 -0.691388 ...
-
so I switched all lights in the room and it's past midnight in rural area -> result is XYZ: 0.732348 0.748855 0.654355, D50 Lab: 6.764379 0.415899 -0.691388 ...
Just so I don't miss what happened, so you got lower values now, those 6.7 instead of 11 earlier? I guess I should in the docs recommend reading glossy targets in dark rooms then, to avoid stray light issues?
-
Just so I don't miss what happened, so you got lower values now, those 6.7 instead of 11 earlier? I guess I should in the docs recommend reading glossy targets in dark rooms then, to avoid stray light issues?
Or shoot them at an angle, facing a black background (which is mirrored by target surface).
Cheers,
Bart
-
Or shoot them at an angle, facing a black background (which is mirrored by target surface).
We're discussing the spectrometer measurement, not the photo :)
-
seems strange that spectrophotometer manufacturers always include a white calibration tile , but not something like a "black" calibration tile
They do - it's the white calibration tile. When you don't turn the illuminant on, you get black.
-
Just so I don't miss what happened, so you got lower values now, those 6.7 instead of 11 earlier? I guess I should in the docs recommend reading glossy targets in dark rooms then, to avoid stray light issues?
no, I have or have access now to 3 different spectrophotometers
1) ColorMunki Design
2) i1Pro2
3) Spectrolino/Green
my "issue" is - can I measure the black patches on my own CCSG or I can not... a $1000 question !
so far it seems all 3 are measuring them (including operator errors too) within L* = upper 10s - lower 12s
Iliah and CCSG data from PM are saying those shall be ~7s
so my next try this evening will be - I will took a piece of black velvet (which Spectrolino apparently measures as darker than black patches on my CCSG - that is what I was referring to by saying " result is XYZ: 0.732348 0.748855 0.654355, D50 Lab: 6.764379 0.415899 -0.691388" - about that velvet w/o any ambient light), switch the light off and measure it with all 3 devices to see if there is a difference between CM, i1Pro2 and Spectrolino ...
and I will try to write a WTF letter to X-Rite too...
-
They do - it's the white calibration tile. When you don't turn the illuminant on, you get black.
good catch, "black"
-
Just speculating, maybe they've changed the formula on newer CCSG targets, having made the black patches brighter? That would reduce the glare problem and make the target easier to use.
-
Black tile is 1% reflection +/-1% accuracy. In "no light" measurement mode spectrophotometer is usually hard-pressed against the white tile, and the tile itself is usually installed in a holder that fits the spectrophotometer head, to ensure no stray light. It is not the same as calibration against flat black tile.
-
Just speculating, maybe they've changed the formula on newer CCSG targets, having made the black patches brighter? That would reduce the glare problem and make the target easier to use.
Checked Robin Mayers measurements, target production dates 2005-2007 (same generation as mine), he used Spectrolino with D65 filter, and got L*=7.5..9.5
-
Checked Robin Mayers measurements, target production dates 2005-2007 (same generation as mine), he used Spectrolino with D65 filter, and got L*=7.5..9.5
as I noted earlier in this thread mine is some new design (I mean where logo is printed - certainly not from 2007 or earlier) and I do not have access to other CCSGs around to check...
here is how it looks :
(http://s28.postimg.org/3kqkomqn1/ccsg.jpg)
may be X-Rite decided to make patches lighter so that i1pro level devices could measure them ? just a wild guess... I am yet to email to xrite, trying to fix a notebook after yet another series of W8.1 patches.
PS: what was the point to use D65 filter with Spectrolino to measure the target for him ?
-
Just speculating, maybe they've changed the formula on newer CCSG targets, having made the black patches brighter? That would reduce the glare problem and make the target easier to use.
but at the moment I can't check - unless somebody with both CCSG can do this, I simply do not have anybody around with the old CCSG (not even new and not even with passport... but plenty of guns though - appalachia).
-
Here goes X-Rite support :
Dear D....,
In an effort to resolve your support Case, the following comment has been added. Please review this comment, and respond as indicated.
Comment:
There are two essential things to keep in mind here.
The first one is that you can't compare LAB readings from different models of spectrophotometers and expect to see exact matches. Your i1PRO2 is the newest one of the ones you've indicated you own, and will be using the current XRGA calibration standard. So please use that device for your measurements.
The second point that is relevant is that the target measurements done in ProfileMaker were done with older formulations of the ColorChecker targets, and were also done with a different spectrophotometer than any of the ones you are testing with.
I hope that this helps to explain.
We would like you to help us continuously improve the support we provide. Click on this Survey Link to provide your input (less than 1 minute to complete).
Thank you,
B.... W....
Technical Support Rep - Level 2
so... "the target measurements done in ProfileMaker were done with older formulations of the ColorChecker targets"
anybody with any spectrophotometer and both old (old as in www.imaging-resource.com test shots) and shiny new CC SG out there willing to test ?
-
Since you already have their ear, can you please ask them where we can find the reference for the new formulation of Digital ColorChecker SG?
This one http://xritephoto.com/ph_product_overview.aspx?ID=938&Action=Support&SupportID=5158 from 2012 indicates low Lab values for the target.
-
Since you already have their ear, can you please ask them where we can find the reference for the new formulation of Digital ColorChecker SG?
already did and CSR did an evasive maneuver suggesting me ColorChecker Classic data ("...Yes, we do post color values for the ColorChecker Classic 24 patch target...")... so I even asked if X-Rite 'd be so kind as to suggest the official/semi-official range for black patches only... waiting
-
This one http://xritephoto.com/ph_product_overview.aspx?ID=938&Action=Support&SupportID=5158 from 2012 indicates low Lab values for the target.
Iliah - but this is __exactly__ the data from Profile Maker - just converted from Spectral to LAB and rounded... but I will use this in my next move (as it has the date = 2012 - so when did they allegedly change the formulation then ?).
-
Iliah - but this is __exactly__ the data from Profile Maker - just converted from Spectral to LAB and rounded... but I will use this in my next move (as it has the date = 2012 - so when did they allegedly change the formulation then ?).
Yes, that data is the same as in PM5, I know that. I do not know of SG formulation ever being changed. XRGA was introduced in 2010, before the page with the link to the reference was edited. You have Spectrolino which is not XRGA-enabled by default (ColorPort 2.x may add an option to account for XRGA w/Spectrolino spectral measurements), and the measurements should conform to pre-XRGA era. I do not know of any newer official or semi-official references, but I know out-of-spec batches of targets may happen. In some regards it may be even beneficial to have higher black values (less flare), and now that you have Spectrolino and know to measure the targets in the dark environment you may take advantage of that.
-
e2:e4
....On the back of your Digital Color SG there will be a date with the Edition month and year. What is your Edition information on this target...
January 2014 Edition
waiting for the next move
-
ColorPort 2.x may add an option to account for XRGA w/Spectrolino spectral measurements
I never managed to get it working on my Windows 8.1 x 64.... i1pc.bin crashes
-
I keep VMs Windows XP64 and Windows 7.
-
I keep VMs Windows XP64 and Windows 7.
argyll does not use XRGA corrections right ? so if one uses argyll spotread/chartread for example then the reported (by the said utilities) data from old gear and new gear will be consistent - correct ?
-
argyll does not use XRGA corrections right ? so if one uses argyll spotread/chartread for example then the reported (by the said utilities) data from old gear and new gear will be consistent - correct ?
As far as I know i1Pro XRGA support is in the driver, so it may depend on what driver is in use. I guess Graeme is the person to ask ;)
-
PS: what was the point to use D65 filter with Spectrolino to measure the target for him ?
Many industrial measurements standardize on D65 and the 10 degree observer. While you can compute this using reflectance data, you won't get quite the same results when there is fluorescence present, whereas the D65 filter will gives a more accurate result.
[ Interestingly one of my Spectrolino's came with a UV filter. I guess it was an option, since it isn't mentioned in the manual, nor did one come with the other Spectrolino I was given. ]
-
argyll does not use XRGA corrections right ? so if one uses argyll spotread/chartread for example then the reported (by the said utilities) data from old gear and new gear will be consistent - correct ?
Yes, ArgyllCMS doesn't currently support XRGA conversion (X-Rite have never publicly revealed what that is, which is pretty poor service to their customers in my view), so you get whatever is native from the device. I believe in the case of the i1pro2, that is XRGA. My current understanding though, is that XRGA involved some massaging of the wavelength calibration and spectral reference values, so it is unlikely to have any significant effect on the L* value.
-
I believe in the case of the i1pro2, that is XRGA.
BabelColor Patchtool though gives an option to select one of 2 modes of operation for i1pro2 (XRGA = M0/M1/M2 and non-XRGA = M0 only) - so either the data comes non corrected from hardware to PC/MAC side software or there is a way to tell the device to switch back to non XRGA mode...
-
BabelColor Patchtool though gives an option to select one of 2 modes of operation for i1pro2 (XRGA = M0/M1/M2 and non-XRGA = M0 only) - so either the data comes non corrected from hardware to PC/MAC side software or there is a way to tell the device to switch back to non XRGA mode...
There is a dll that comes with the X-Rite driver & other programs that does is used for the conversions to/from XRGA.
-
I keep VMs Windows XP64 and Windows 7.
Iliah how do you manager to get the instrument connected in a virtual machine ?
-
Iliah how do you manager to get the instrument connected in a virtual machine ?
That probably depends on the VM you are using. For Fusion (VMWare) it is automatic, when an instrument is plugged in, it pops up with a question 'what to do' - to connect to host or to VM. This question can be switched off, too, if one of those two actions is set as default in menu item "Virtual Machine" -> USB and Bluetooth -> Advanced USB options. I also set USB 2.0 there for the guest.
-
That probably depends on the VM you are using. For Fusion (VMWare) it is automatic, when an instrument is plugged in, it pops up with a question 'what to do' - to connect to host or to VM. This question can be switched off, too, if one of those two actions is set as default in menu item "Virtual Machine" -> USB and Bluetooth -> Advanced USB options. I also set USB 2.0 there for the guest.
may be USB settings were the issue, I put USB 1.1 compatibility now in VMWare and can connect to i1pro2... however with Spectrolino the ColorPort v2.0.5 software only shows SpectroScan as an option - which is Spectrolino with an automated measuring table, which I don't have... does it work with Spectrolino alone ?
-
may be USB settings were the issue, I put USB 1.1 compatibility now in VMWare and can connect to i1pro2... however with Spectrolino the ColorPort v2.0.5 software only shows SpectroScan as an option - which is Spectrolino with an automated measuring table, which I don't have... does it work with Spectrolino alone ?
I tried and - no, it does not work for me, sorry. Only with Spectroscan, seems only because of the automatic positioning on the calibration tile. All the rest I was able to do manually.
-
I tried and - no, it does not work for me, sorry. Only with Spectroscan, seems only because of the automatic positioning on the calibration tile. All the rest I was able to do manually.
OK, N/A... on the other subject - I compared a new black velvet sample measuring with i1pro2 and spectrolino : i1pro2 gives L* = ~2.9-~4.0 on various spots and spectrolino ~2.9-~3.5 on various spots... so I guess if both of them do measure noticeably lighter black patches on SG and their data is similar then so it is... new targets do have the black patches lighter... I did not hear back from X-Rite so far... so will be trying to remind them again, albeit w/o big hope.
-
OK, N/A... on the other subject - I compared a new black velvet sample measuring with i1pro2 and spectrolino : i1pro2 gives L* = ~2.9-~4.0 on various spots and spectrolino ~2.9-~3.5 on various spots... so I guess if both of them do measure noticeably lighter black patches on SG and their data is similar then so it is... new targets do have the black patches lighter... I did not hear back from X-Rite so far... so will be trying to remind them again, albeit w/o big hope.
Yes, it seems they need to publish a new reference for SG. Or recognise bad batches and either replace, or publish references. One other thing - if you have polarasing filter for Spectrolino, try measuring with the filter on, using MeasureTool from ProfileMaker 5.0.10. It should not make a difference, theoretically, but does not hurt to try.
Very sorry I confused you with Spectrolino / ColorPort.
-
One other thing - if you have polarasing filter for Spectrolino, try measuring with the filter on, using MeasureTool from ProfileMaker 5.0.10. It should not make a difference, theoretically, but does not hurt to try.
it actually makes hell lot of difference, while w/ U filter it gives L* = 11.x, w/ P filter it gives L* = ~1.8-1.9 (???!!!)
Very sorry I confused you with Spectrolino / ColorPort.
n/p - some knowledge gained
-
> it actually makes hell lot of difference, while w/ U filter it gives L* = 11.x, w/ P filter it gives L* = ~1.8-1.9 (???!!!)
Was it with MeasureTool? Does the resulting CGATS say pola filter is in use?
-
> it actually makes hell lot of difference, while w/ U filter it gives L* = 11.x, w/ P filter it gives L* = ~1.8-1.9 (???!!!)
Was it with MeasureTool? Does the resulting CGATS say pola filter is in use?
you asked to use ProfileMaker MeasureTool, so it was it... yes, filters were detected
INSTRUMENTATION "Spectrolino"
MEASUREMENT_SOURCE "Illumination=D50 ObserverAngle=2° WhiteBase=Abs Filter=Pol"
strangely MeasureTool does not output spectal data - LAB/XYZ only
-
> strangely MeasureTool does not output spectal data - LAB/XYZ only
It should, if asked to - click on Configuring and choose spectral.
What L does it show for white patches with pola filter? What does it show on black and white patches without a pola filter?
-
> strangely MeasureTool does not output spectal data - LAB/XYZ only
It should, if asked to - click on Configuring and choose spectral.
but it does not... checkbox is on, but it does not save the spectral data (I am talking about the mode when it measures Test Chart).
white patch, L* = 98.*
grey patch L* = 48.*
black patch L* = 1.9
PS: for example I measured some 3 patches again (white-grey-black) and here is the output... no spectral data
LGOROWLENGTH 1
Measurement_mode "patch"
CREATED "6/13/2015" # Time: 21:58
INSTRUMENTATION "Spectrolino"
MEASUREMENT_SOURCE "Illumination=D50 ObserverAngle=2° WhiteBase=Abs Filter=Pol"
KEYWORD "SampleID"
KEYWORD "SAMPLE_NAME"
NUMBER_OF_FIELDS 8
BEGIN_DATA_FORMAT
SampleID SAMPLE_NAME XYZ_X XYZ_Y XYZ_Z LAB_L LAB_A LAB_B
END_DATA_FORMAT
NUMBER_OF_SETS 3
BEGIN_DATA
1 A1 91.25 94.85 75.05 97.97 -0.36 2.71
2 B1 16.33 16.98 13.81 48.24 -0.25 0.53
3 C1 0.21 0.21 0.18 1.89 0.30 -0.15
END_DATA
-
but it does not... checkbox is on, but it does not save the spectral data (I am talking about the mode when it measures Test Chart).
white patch, L* = 98.*
grey patch L* = 48.*
black patch L* = 1.9
Nice measurements, overall. Do you save data explicitly, File - Save As; or do you use "save measurements"?
-
Nice measurements, overall. Do you save data explicitly, File - Save As; or do you use "save measurements"?
the dialog window says : "export lab"
(http://s9.postimg.org/r60bpsmrj/pm1.jpg)
PS: got... it - there is a difference between that option and what is done through : File - Save As ... RTFM !
-
Nice measurements, overall.
well white and grey may be - but black ?
-
the dialog window says : "export lab"
Please try File - Save As. You have "spkt" mark, so it should be saving spectral data, 'cause this mark indicates the measurements were taken in spectral mode.
-
As to black, take measurements of those 15 neutral patches in the middle and let's check how linear they are. I would also measure the whole target and check deltaE with the reference.
-
Please try File - Save As. You have "spkt" mark, so it should be saving spectral data, 'cause this mark indicates the measurements were taken in spectral mode.
yes, I found that File->Save As indeed saves the spectral data... thank you, I was thinking that both options are the same and did not bother to try.
-
yes, I found that File->Save As indeed saves the spectral data... thank you, I was thinking that both options are the same and did not bother to try.
No, they are quite different. It was mentioned in manual somewhere, but not in the Help file they have with the program.
-
As to black, take measurements of those 15 neutral patches in the middle and let's check how linear they are. I would also measure the whole target and check deltaE with the reference.
OK, I will do this now and post then
-
As to black, take measurements of those 15 neutral patches in the middle and let's check how linear they are. I would also measure the whole target and check deltaE with the reference.
here is the full target, taken with P filter
-
here is the full target, taken with P filter
It looks like the instrument needs cleaning, after all.
-
Got my Solux lamps working finally by powering them properly. Look at plots in the neighboring thread:
http://forum.luminous-landscape.com/index.php?topic=100698.msg830255#msg830255
I think the Solux 4700K looks good for simulating D50 and even D55 when pushed into overdrive.
-
Got my Solux lamps working finally by powering them properly. Look at plots in the neighboring thread:
http://forum.luminous-landscape.com/index.php?topic=100698.msg830255#msg830255
I think the Solux 4700K looks good for simulating D50 and even D55 when pushed into overdrive.
I had Solux bulbs powered from their own power source, and they do not simulate D50. When powered to a higher voltage through a high grade stabilized variable voltage power supply, the time drift of SPD is quite significant. Once I measured the SPD with 2nm resolution, and it was spiky.
Solux 4700/36, measured with i1Pro, 4 times, averaged, 10nm, beam integrated over the surface:
380 1.16572925E-3
390 1.91144715E-3
400 2.50905682E-3
410 3.10437335E-3
420 3.64064472E-3
430 4.19029733E-3
440 4.51817922E-3
450 5.01060858E-3
460 5.41420793E-3
470 5.68507239E-3
480 5.91422338E-3
490 6.13182224E-3
500 6.40437938E-3
510 6.66856067E-3
520 6.87941955E-3
530 7.24500837E-3
540 7.71248853E-3
550 8.06061924E-3
560 8.29440542E-3
570 8.48474540E-3
580 8.45067855E-3
590 8.48383829E-3
600 8.30854755E-3
610 8.03543720E-3
620 7.96562620E-3
630 7.86271039E-3
640 7.66951498E-3
650 7.79212918E-3
660 7.72498781E-3
670 7.72686955E-3
680 7.82623328E-3
690 7.64010241E-3
700 7.34899333E-3
710 7.22256443E-3
720 7.06195412E-3
730 6.93874108E-3
That is less than 4300K CCT.
-
When powered to a higher voltage through a high grade stabilized variable voltage power supply, the time drift of SPD is quite significant.
what might be the source of the drift then... assuming that you have some lab grade power supply and voltage then can be assured within what ? like < 0.1% or something like this ?
-
beam integrated over the surface
does it mean that you were illuminating some surface (like styrofoam) and measuring off the reflected light ? or something else ?
-
does it mean that you were illuminating some surface (like styrofoam) and measuring off the reflected light ? or something else ?
Matt spectralon panel. That gives much better idea of the beam profile.
-
How could the spectrum be spiky? Shouldn't it be physically impossible when you have halogen? As far as I understand the Solux is nothing else than a standard halogen lamp with a blue-filtered reflector.
Indeed, you really need a variable power supply to get a color temperature you want. It's graded to 4700K but far from all lamps reach that, halogens vary quite a bit naturally and I would not think they have put 4700K in the center of the normal distribution. Lamp fixtures also seem often seem to underpower the lamps bit, and then with an unsuitable lamp shade (especially if using the unpainted older lamps) you lose even further due to mixing in unfiltered light.
I'd be surprised if Solux halogens would drift more than any other halogen, as it is a, well, halogen lamp.
-
what might be the source of the drift then... assuming that you have some lab grade power supply and voltage then can be assured within what ? like < 0.1% or something like this ?
Possibly, accelerated aging; or overheating. Anyway, Solux are using Eiko Q50MR16/CG/47/36 neodymium bulbs, and metal-doped filters produce not very natural SPDs.
-
> How could the spectrum be spiky? Shouldn't it be physically impossible when you have halogen? As far as I understand the Solux is nothing else than a standard halogen lamp with a blue-filtered reflector.
If you look into how the reflector is made...
-
I only have the Colormunki instrument to measure with that claims 3nm at high resolution mode, no spikes turn up there (spikes do turn up very clearly on a fluorescent source for example).
This makes me think that even if there are spikes they are narrow enough to have negligible impact in camera profiling applications. What we're measuring in profiling is the camera's spectral integration, and a "worried" spectrum is no problem unless it risks shifting the energy to such an extent that it shifts the integration result. Wouldn't you agree?
-
Additionally, it seems to me that a typical photographic flash or xenon arc lamp would be worse concerning spikiness, and certainly shot-to-shot variation, here an example flash spectrum (wavelength is in Angstrom not nm, divide by 10):
(http://www.astrosurf.com/buil/us/spectro10/flash.gif)
I'm just presenting the "evidence" I base my own recommendations on. As I thought I had enough info on my hands to be able to recommend the Solux on overdrive as a D50 simulator for camera profiling, any claims that suggests that it's actually no good is very interesting to me.
-
> a "worried" spectrum is no problem unless it risks shifting the energy to such an extent that it shifts the integration result.
I simply found that the results of colour transforms I'm getting with 5000K using normal filters over halogen lamps are better compared to Eiko/Solux, and I can use regular 3200K halogen lamps and cheap high quality filters; both components easily replaceable. And I do not need to deal with uneven distribution of CT across the beam.
-
I only have the Colormunki instrument to measure with that claims 3nm at high resolution mode, no spikes turn up there (spikes do turn up very clearly on a fluorescent source for example).
This makes me think that even if there are spikes they are narrow enough to have negligible impact in camera profiling applications. What we're measuring in profiling is the camera's spectral integration, and a "worried" spectrum is no problem unless it risks shifting the energy to such an extent that it shifts the integration result. Wouldn't you agree?
a small note... for example CMF that you feed into DCamProf being spiky does not affect much the profiles it generates...
-
Ok, thanks for sharing.
I just did a quick drift test when running it 15.8 volts at 5500K. It's not free from drift, the first 10 minutes (started cool) it drifted about 0.3DE, but when warm it seems to stabilize a bit and it drifted 0.08DE the following ten minutes (plot attached). That drift is so small it can be some related to measurement too.
Anyway, I'm going to recommend to work with lamps that's been on for 10 minutes or so, and then work relatively swiftly, should take only 5 minutes or so to complete a series of test target shots, and it that time it should be stable enough.
-
> I'm going to recommend to work with lamps that's been on for 10 minutes or so
Usual is 15 minutes to 20 minutes. Now, after the bulbs are stabilized, one needs to measure the spectrum; and that defeats the purpose of having D50 simulator. To be clear, that is the case with any halogen bulbs except lab grade sources, and even with those it is still better to measure the spectrum. Point of the exercise is to get as close to 5000K blackbody as possible the cheapest way.
-
Usual is 15 minutes to 20 minutes. Now, after the bulbs are stabilized, one needs to measure the spectrum;
one might imagine a spectrophotometer used may be in ambient mode right when you shoot your target... for example make a hole behind something where you target is mounted to place spectrophotometer there... now you just need to sync things somehow.
-
My strive for a "D50 simulator" should not be taken too literally, the purpose I have is a smooth spectrum, no significant broad peaks or dips, no significant spikes, with an overall distribution similar to D50. For camera profiling there's no real value in exactly matching D50 as D50 doesn't really exist in nature either. The purpose of this is to have an alternative to going outside using actual daylight. Shooting glossy targets with minimized glare is a lot easier to do with this type of light source than using real daylight.
Now I'm pushing it to 5500K to get a little bit more spacing from 2850K in a dual-illuminant scenario.
Compared to a 3000K halogen lamp with an 80B filter all indications I get is that the overdriven Solux makes a better job. I'm sure though that Liu labs lamp is even better, but I doubt it will present any significant gain, except if we want to get closer to 6500K which the Solux can't do.
It's far from a practical setup, I have only one lamp, flatfield correction is required, temperature stabilization is required, it's preferable to measure it. I wouldn't use this for copy work or anything like that, but to shoot a target with flatfield-capable software it seems to be a good setup.
The value of actually measuring the spectrum is there when we want to make a DNG profile that can do precise whitepoint estimation, otherwise it's not that important when we make a generic "daylight" profile.
-
On a different subject; I'm going to have to look more into how to subjectively deal with the contrast curve. The global desaturation trick did not work out as well as it first seemed, it may work for highly saturated colors but leave low saturation colors too desaturated. Possibly an increasing onset depending on saturation would work, or I need to figure out some different strategy.
-
I simply found that the results of colour transforms I'm getting with 5000K using normal filters over halogen lamps are better compared to Eiko/Solux, and I can use regular 3200K halogen lamps and cheap high quality filters; both components easily replaceable. And I do not need to deal with uneven distribution of CT across the beam.
Iliah, do you use 80A, 80C Lee gels for correction? I had a go with Lee CT blue gels (full and half) briefly and for my halogen source they do add strange dip at the green area. Have not evaluated various lamps yet only the one that came with my source but it does not have this without filters.
-
Iliah, do you use 80A, 80C Lee gels for correction? I had a go with Lee CT blue gels (full and half) briefly and for my halogen source they do add strange dip at the green area. Have not evaluated various lamps yet only the one that came with my source but it does not have this without filters.
I use 8x series.
For the lights they make:
Lee 283, 201, 281, 202 (these are the main ones)
Given the sensitivity of sensor green channels, a dip in the green area may be a good thing.
for use with MR16 and PAR bulbs, framed glass, better greens:
Lee B64, LD201, LD202
-
I use 8x series.
For the lights they make:
Lee 283, 201, 281, 202 (these are the main ones)
Given the sensitivity of sensor green channels, a dip in the green area may be a good thing.
for use with MR16 and PAR bulbs, framed glass, better greens:
Lee B64, LD201, LD202
Thanks, mine was 201 (Full CT blue) that had this dip. I tried 202 as well and it was a lot better and smoother. For now my only use for them was an attempt to even slightly blue areas of the spectrum for monochromator input hence the questions.
-
Thanks, mine was 201 (Full CT blue) that had this dip. I tried 202 as well and it was a lot better and smoother. For now my only use for them was an attempt to even slightly blue areas of the spectrum for monochromator input hence the questions.
Lee are publishing pretty accurate Y-curves on their site.
For the purpose of making the light to monochromator more even I would be looking at 281 and 202, maybe double layer.
-
Lee B64, LD201, LD202
those who love smooth graphs shall like this one = B53 Blue 3 - it is a black line on the graph
(http://s4.postimg.org/guhyfng31/chart.png)
-
those who love smooth graphs shall like this one = B53 Blue 3 - it is a black line on the graph
(http://s4.postimg.org/guhyfng31/chart.png)
With dichroic filters, the curve is angle-dependent. If the sampling is closer, such filters usually show the transmission curve is not very smooth https://www.thorlabs.com/newgrouppage9.cfm?objectgroup_id=986
-
With dichroic filters, the curve is angle-dependent. If the sampling is closer, such filters usually show the transmission curve is not very smooth https://www.thorlabs.com/newgrouppage9.cfm?objectgroup_id=986
but then @ any angle that curve shall be still smoother than others, because it has a better "starting point /at 0 degrees/", no ?
-
but then @ any angle that curve shall be still smoother than others, because it has a better "starting point /at 0 degrees/", no ?
The problem here is to have good mixing after the filter to have reasonably constant SPD across the beam. The setup starts to be bulky and expensive (compared to a single fresnel lens).
-
Lee are publishing pretty accurate Y-curves on their site.
For the purpose of making the light to monochromator more even I would be looking at 281 and 202, maybe double layer.
Thanks will try this combination (I have a Lee swatchbook and that is enough for light source fiber optic lead).
-
This tone curve thing is giving me an headache...
I've attached two images to demonstrate. Both are using the same profile. The first with linear curve, and the other a typical S-curve, Adobe's default. The curve brighten the picture a lot so I have reduced the exposure of the second to make it easier to compare.
The first has rather accurate colors when I compare the calibrated screen with the real thing, as expected. The second has not only increased saturation quite far, but also pushed the reds into some other hue. I know one reason, the curve is applied in Prophoto space (as DNG pipeline does), pushing green to 0 in sRGB. Adobe Lightroom (used here) has applied some sort of gamut mapping though so it's not as bad as it could be. I've tried to apply the curve in sRGB space instead when using RawTherapee, better result. Even without space clipping there are saturation increases and hue shifts though.
In theory the tone curve could been seen as just a compensator for the Hunt and Stevens effects (eg the dimmer screen needs more contrast and saturation to perceptually match a brighter real scene), but it's generally going way over the top. I'd like it not to be the profile's job to deal with side effects of the tone curves, but anyone that's going to use default Adobe curves with DCamProf profiles are going to think that the profiles suck, and indeed they kind of do.
Currently I'm currently out of ideas how to deal with this. For myself using RawTherapee I can apply contrast using a CIECAM02 lightness curve and have good realistic results, but that's not what the general user will be doing, so I guess I need to come up with something. I'm not totally out of ideas, but there are only the painful ones left...
There are a zillion of tone curve models out there in the research world, which all are better than a simple RGB curve (ICC profiles) or Adobe's RGB curve twist (DNG profiles), but DCamProf can just make the profile, not affect which tone curve that's being used, and current color pipelines are locked to simplistic RGB curves.
-
It seems like this tone curve thing is what makes camera color "difficult".
As an RGB curve is so massively bad at doing anything perceptually sane, you need to make a profile that rather than making accurate colors make a precompensation for the tonecurve errors, so you end up at something decent.
The commercial profiles seems to be a mix of subjective look and compensation of tone curve errors. It differs a lot how saturated reds are dealt with for example.
I shall continue with some testing though. If I'm lucky a good compensation can be some gradual pre-desaturation of colors, that is low saturation colors are kept almost unchanged while high saturation colors are desaturated more, and out of sRGB/AdobeRGB colors even more. If that works the profile can stay 2.5D.
In the painful case I need to make the profile 3D with the sole purpose of compensating the tonecurve's wrongdoings. In a way that could be a cleaner design though. In that case I'd keep subjectivity out of the design process and the native profile format, it would always be linear, and tonecurve compensation would only be an option in the make-dcp command. It would then just add a 3D LookTable on top which compensates for the tonecurve. It would be quite simple to make a "mathematically exact" compensation this way, if you know what the end result should be. A 3D LUT would be less robust though, if you change the curve the compensation would no longer match, but maybe that's not a big issue.
With ICC profile you cannot add a separate look-table on top, but one can merge everything into the same 3D CLUT, for the user it will be no difference.
To do the 3D way I need to decide on what the desired tonecurve look should be... work in progress...!
-
Hi Torger
Many of the technical details in this thread are far above my head. Yet I dare to come up with a little comment. I attach 3 screen shots. 1-shows your linear jpeg opened in PhotoLine. 2-with an S-curve in RGB, which shows the same color shift as yours. And 3- the same S-curve, but applied in HSV, using the V channel only. No color shift (visually judged). It seems to me the solution to the curve problem should not be searched in the profile. But I can see, it is something the USER would have to do...
Best regards - Hening.
-
Hi Torger
Many of the technical details in this thread are far above my head. Yet I dare to come up with a little comment. I attach 3 screen shots. 1-shows your linear jpeg opened in PhotoLine. 2-with an S-curve in RGB, which shows the same color shift as yours. And 3- the same S-curve, but applied in HSV, using the V channel only. No color shift (visually judged). It seems to me the solution to the curve problem should not be searched in the profile. But I can see, it is something the USER would have to do...
Thanks for the feedback. Indeed, the easy way out is to say "the profile gives you accurate colors, but the raw converter messes it up with its ancient RGB curve, you need to use some other method to add contrast". I can certainly live with that myself, but in terms of making profiles for wider use it's probably not so good strategy. The sad thing is that a neutral accurate profile will in Lightroom, Capture One etc produce less realistic colors (at least for the saturated ones) than some of the bundled ones, as the bundled ones are designed with the tone curve in mind. Unless you lock it to a linear curve, but for a decent look you need to add at least a little contrast.
Designing for a specific curve and contrast makes the profile less generic and means that you cannot use it for reproduction work, so it's really not a nice option. But the fact is there -- most current raw converters use primitive color pipelines from the 1990s and that makes it impossible to have one profile with neutral accurate color that can be used with different amounts of contrast at still produce realistic colors.
People tend to think that you can use a profile with any "film curve" (in Capture One you can select it separately), but the profiles are clearly designed for the default film curve. Capture One's profiles produce too desaturated results when used with a linear curve, because they're not designed for that. DCamProf's profiles produce accurate results with linear curve, but then instead too saturated and sometimes visibly hue-shifted results when used with a curve.
I don't have a finished answer on what's the state of the art contrast curve today, but preliminary I've got very nice results with a CIECAM02 lightness curve. CIELab lightness which in theory would be the same thing as CIECAM02 lightness is not good at all, it desaturates color, not realistic result. The reason being that the CIECAM02 is simply a more recent and better color appearance model.
What raw converters should be doing is have a modern color pipeline designed for digital photography rather than being nostalgically stuck with "film looks", in that way the profile would not be involved in tone reproduction or tone mapping at all but you would control that in the raw converter and you could choose both realism and make a subjective look. The only raw converter that allows this type of workflow I know of is RawTherapee (the oldschool way is the default though). My goal with DCamProf is to work with popular widely used raw converters though so I need to adapt to their workflows.
-
I wonder what Adobe were thinking when they made the DNG pipeline apply the tone curve in prophoto space. To me that seems to be a rather bad idea. Any fairly saturated color will be clipped due to this, and Lightroom "repairs" it with some sort of gamut mapping. It makes the look really unpredictable so I understand that people need to handtune their profiles.
Capture One applies their RGB curve in some smaller space, which it is seem to depend on camera. That's probably a better approach, but still not well-defined.
-
I wonder what Adobe were thinking when they made the DNG pipeline apply the tone curve in prophoto space. To me that seems to be a rather bad idea. Any fairly saturated color will be clipped due to this, and Lightroom "repairs" it with some sort of gamut mapping. It makes the look really unpredictable so I understand that people need to handtune their profiles.
Capture One applies their RGB curve in some smaller space, which it is seem to depend on camera. That's probably a better approach, but still not well-defined.
A small correction, they don't apply it in just ProPhoto - they apply it nearly at the end of the pipeline in linear ProPhoto. That is before converting to gamma corrected ProPhoto.
-
A small correction, they don't apply it in just ProPhoto - they apply it nearly at the end of the pipeline in linear ProPhoto. That is before converting to gamma corrected ProPhoto.
Indeed, I am often a bit sloppy when talking about color spaces. Used to floating point since ten years gamma has lost its use, so I meant prophoto primaries in linear space. I shall try to be clearer, it's not the first time I confuse people with my sloppy color space terminology :-)
-
[...] The only raw converter that allows this type of workflow I know of is RawTherapee [...]
Hm - I use(d) Iridient that way. And if memory serves me, one could do it in ACR as well. I use(d) the linear camera-own profile both as an in and out profile. This requires a gamma matrix profile, which is built with the assumption of a linear sensor response. This should be theoretically OK with the 5D2. And I have naïvely done the same with the a7r, and become first now aware of that the compressed raw means the sensor is not linear. I have not examined this any further yet.
edit: Ooops, no, ACR does not allow you to choose a linear out profile.
-
And I have naïvely done the same with the a7r, and become first now aware of that the compressed raw means the sensor is not linear. I have not examined this any further yet.
For all intents and purpouses the raw is linear. The tonal curve applied to compress (lossy compression) the raw. Raw converters then apply it in reverse to unpack the raw data. The raw values are linear because otherwise you would not be able to white balance it easily.
-
Oh thank you! That is certainly good news to me!
-
Oh thank you! That is certainly good news to me!
and when things are getting non linear you hence have WB issues - like deeeeeep shadows.
-
and when things are getting non linear you hence have WB issues - like deeeeeep shadows.
And hiiiighlights? Obviously this is the problem - but what do I make of this? Is the response of the sensor+raw converter linear, or not? Does the raw converter apply white balance before or after linearising? Is this the correct question? The practical question for me is: Can I make a gamma matrix profile for the a7r based on the assumption of linear sensor response? And can I continue to use this both as the in and out profile? Or do I need to make 2 LUT profiles? What I want is a linear output profile.
Are we side tracking the thread, Torger?
-
Is the response of the sensor+raw converter linear, or not?
you are not dealing with "sensor", you are dealing with the numbers that firmware writes in raw file... so for profiling purposes you try to get those numbers in a "linear fashion", means you expose so that any non linearities can be avoided ... that is you do not underexpose and you do not get close to clipping - based on how particular camera behaves here and there.
Does the raw converter apply white balance before or after linearising?
naturally after
-
thank you for clarifying!
-
>> Does the raw converter apply white balance before or after linearising?
> naturally after
Yes, if linearization is a part of raw unpacking and decoding :)
No, if you mean full well limited cameras, like some Panasonics are.
There are more woodpeckers than one can possibly count.
-
No, if you mean full well limited cameras, like some Panasonics are.
you mean that with Panasonic you can have each sensel data recorded in raw to the max possible DN /no adjustment by firmware to so max limit/, so you have non linear data if you take more then one sensel at once for your purposes - they /sensels/ all will have (might have) different well capacities... hence non linearity.
-
you mean that with Panasonic you can have each sensel data recorded in raw to the max possible DN /no adjustment by firmware to so max limit/, so you have non linear data if you take more then one sensel at once for your purposes - they /sensels/ all will have (might have) different well capacities... hence non linearity.
_Some_ Panasonic cameras at the lowest ISO setting are full well limited, so each sensel charged close to max (within 1 stop of saturation) is non-linear, Panasonic has the linearity limit in makernotes, and we informed Phil of that tag - so it is covered in exiftool.
-
Searched some more on tonecurve stuff, and it's not that easy to find any obvious solution to the problem. In the research world this is called "tone reproduction operator", and they have since 10+ years left the simplistic one global tone curve approach and are more into spatially variable operators, that is what we usually call tonemapping. A profile can't do scene-dependent corrections so we can't use that kind of stuff.
The traditional RGB tone curve is not only about creating a subjective look, but also to fix appearance issues, mainly Stevens and Hunt effects (contrast and colorfulness appears higher in a real bright scene than if reproduced with lower luminance on a screen). Digging down you also have Helmholtz-Kohlrausch effect and Bezold-Brücke hue shift effects. All these effects are scene-dependent of course, but you could say a fixed tone curve roughly matches a sunny day, and it's probably good approach. It will make duller scenes look more contrasty than the real deal but that's probably how people will want to make it look anyway.
The big name raw converters are in any case not designed for scene-dependent appearance modeling, we have this fixed tonecurve to relate to. And then my idea is that I want a tone curve which changes contrast, but colors appears to be of the same hue (and saturation) as before. It's there the current raw converters with their RGB curves fail miserably, making a linear colorimetric profile produce bad colors. But a profile can always compensate to give you any result you'd like, and it's exactly what the big names do (with varying success). My plan is to make it possible to make tone curve compensation with DCamProf profiles as well, but using some good appearance model rather than relying on golden-eye subjective hand-tuning which many commercial raw converters seem to be doing.
-
Hi,
What about just applying tone curve to the luminance channel?
My take is really that raw conversion should result in a scene referred image and different kind of "looks" could be applied that image. I don't know if this is in conflict with DCP-based processing.
Best regards
Erik
Searched some more on tonecurve stuff, and it's not that easy to find any obvious solution to the problem. In the research world this is called "tone reproduction operator", and they have since 10+ years left the simplistic one global tone curve approach and are more into spatially variable operators, that is what we usually call tonemapping. A profile can't do scene-dependent corrections so we can't use that kind of stuff.
The traditional RGB tone curve is not only about creating a subjective look, but also to fix appearance issues, mainly Stevens and Hunt effects (contrast and colorfulness appears higher in a real bright scene than if reproduced with lower luminance on a screen). Digging down you also have Helmholtz-Kohlrausch effect and Bezold-Brücke hue shift effects. All these effects are scene-dependent of course, but you could say a fixed tone curve roughly matches a sunny day, and it's probably good approach. It will make duller scenes look more contrasty than the real deal but that's probably how people will want to make it look anyway.
The big name raw converters are in any case not designed for scene-dependent appearance modeling, we have this fixed tonecurve to relate to. And then my idea is that I want a tone curve which changes contrast, but colors appears to be of the same hue (and saturation) as before. It's there the current raw converters with their RGB curves fail miserably, making a linear colorimetric profile produce bad colors. But a profile can always compensate to give you any result you'd like, and it's exactly what the big names do (with varying success). My plan is to make it possible to make tone curve compensation with DCamProf profiles as well, but using some good appearance model rather than relying on golden-eye subjective hand-tuning which many commercial raw converters seem to be doing.
-
The big name raw converters are in any case not designed for scene-dependent appearance modeling, we have this fixed tonecurve to relate to. And then my idea is that I want a tone curve which changes contrast, but colors appears to be of the same hue (and saturation) as before.
Check RPP out
-
Check RPP out
Only possible on Mac ...:(
Cheers,
Bart
-
What about just applying tone curve to the luminance channel?
My take is really that raw conversion should result in a scene referred image and different kind of "looks" could be applied that image. I don't know if this is in conflict with DCP-based processing.
Current raw converters are a bit of a mixture of old film techniques and new color appearance models. White balance or tone curves do not exist in color appearance models like CIECAM02. In color science speak white balance is a simplified chromatic adpatation transform, and an RGB tone curve is an over-simplified tone reproduction operator. Color science is busy with making realistic reproductions, even emulating eyes night vision etc (meaning that you need to know exposure level too), and taking view conditions into account. To make it state-of-the-art is very complex. Documentary video seems to be the natural applications of these types of ultra-realistic reproductions.
The first part of DCP could be said to be scene referred, but the tone curve is certainly output referred. As I want DCamProf to be a profile maker also useful for the big name raw converters I need to adapt to their workflows. Personally I'd surely prefer that a profile only did the scene referred part, and the raw converter took care of all output modeling, but that's not how it works in Lightroom and Capture One for example. They expect a curve to be there, and if the profile is linear scene referred only there are no tools to add contrast to adjust the look without having color shift effects.
In photography we seem to settle at rendering D50 bright sunny scenes fairly realistically in an average viewing condition, and then accept as a bonus the extra "pop" we get when we apply the same tone reproduction and colorfulness on other conditions. I'm fine with that. Problem is that the RGB tone curve is as discussed a very bad tone reproduction operator which don't even make the sunny scenes realistic but makes color pop like crazy, and in some circumstances cause hue shift as well. Bundled profiles are semi-compensated for that, by hand I suppose for more exclusive cameras.
I'm trying to find an automated way to compensate a profile so it can be used with RGB curves without getting crazy results.
Applying contrast to the luminance channel is a good idea, but then the next question is which model of luminance should we use? Using Lab Lightness results in a desaturated look. I'm currently experimenting with using CIECAM02 lightness and that is a better perceptual model than Lab and gives better results. However, as we apply lightness contrast to simulate a brighter outdoor condition (Stevens effect) we still get a desaturated look due to the Hunt effect, to match realistically we need to add some saturation too. It's there I am now. It seems like this problem hasn't really been solved before as those that work with CIECAM02 don't work with traditional "film curves".
A color scientist would just say it can't be done without knowing all scene parameters and viewing condition parameters, but I'm looking into using that sunny D50 scene as a reference, I think if it works for that scene we'll have a suitable "neutral" look that photographers can identify with. The bright outdoor sunny scene is really the only condition that mandates increased contrast and saturation.
-
Only possible on Mac ...:(
no, both RPP and Iridient do work in OSX and OSX does work in VmWare (yes, EULA says you can't, but Apple can take a hike) and VmWare does work on PC/Win...
-
RawTherapee is also a good experimental box for trying out "scene referred" workflows, in RT you have the CIECAM02 mode, where you can enter scene conditions, viewing conditions and all, simulate light dark adaptation and partial chromatic adaptation etc.
You can also apply contrast in Lab, CIECAM02 J or Q, or use their own custom "weighted standard curve" which is a quite good curve when it comes to keeping hue/saturation constant.
As said though we don't want to keep saturation constant as we have the Hunt effect to think about, if we just add contrast the impression is that we reduced saturation even if it was kept constant. We could also argue that hue should not be kept constant either due to the Bezold-Brücke effect, but I think that is overkill. So my current idea is to add contrast and saturation in some sane way and keep hue constant.
I can recommend this link for those interested in those color appearance effects: http://www.handprint.com/HP/WCL/color4.html
-
Small demonstration.
Same profile on all images (a matrix-only profile). First image is with linear curve (this is the reference), the others have the same contrast curve. One is CIECAM02 J lightness only (second image), hue and saturation/chroma constant. The other (third image) is the same, but with a subjective increase in chroma.
If you really just stare at the pure colors and compare you'll probably see that the constant hue/chroma matches the linear the best. If you look at the picture "globally", the constant hue/chroma looks slightly desaturated compared to the original linear curve, and then the one with slightly increased chroma may look more "right".
For reference I've added a 4th image which is a pure RGB curve in sRGB space (which here makes a better result than doing it in ProPhoto space like DNG). With sRGB and this color I don't think the RGB curve pulls it off that much, maybe a little towards yellow (hue shift is worse if done in Prophoto due to clipping). The obvious difference is that it's grossly saturated, way over the top.
It's quite subtle differences we're working with here, so it can be quite hard to compare with the images one below the other, especially since screens vary in color and contrast slightly depending on viewing angle. Downloading and flipping through them in an image viewer makes it easier.
I'm thinking that DCamProf users will want results similar to image 3, that is contrast in lightness, but with some added saturation.
-
For the sake of completeness, here's one with the RGB curve applied in Prophoto space, here without gamut mapping which makes the hue shift more evident.
-
I think it's quite clear from the little demonstration that you cannot see the profile and curve separately. That's why I'm skeptical about the practice I see many do in Lightroom and Capture One of changing to linear curve to get "more accurate color" out of the bundled profiles. Most likely the bundled profiles are designed for the default curves, and if they are they have changed the color to a big extent to work with that RGB curve, meaning that color may very well become less accurate when used with a linear curve.
I see on the forums that curves are by many seen as some minor thing you add on top to get a little contrast, but it makes a big difference in global color appearance, and for some colors hue shift is quite noticable. Striving for utmost precision in targets and measurements when making an own profile is all thrown out the window if we add a standard curve on top and think we still have accurate color.
We can always fall back on the argument that accurate color is not important, as we will subjectively tune the look to something that pops anyway, and I have no problem with that view. However, the vision I have with DCamProf is to provide to photographers that want it a neutral starting point to add their artistic intent on top, rather than getting a random starting point.
For my local runners club I shoot running competitions from time to time, and it's with bundled profiles impossible to get those supersaturated sports apparel look the same hue as in real life, which I think is remarkable that we can't do in 2015. And although I can agree that maybe it's not that important, it feels silly that my expensive gear and software can't do it. Although I add a subtle subjective look to my images, having the wrong hue on peoples clothing is not a part of my artistic intent, it just becomes like that because profiles and raw converter is not up to it. The film legacy have shaped raw converter design into making it more difficult to get realistic color than it had to be, and I'd say the lack of a proper tone reproduction operator is 95% of the problem.
If we want to do reproduction photography we're fine though, because in that case we won't be using a curve and then any raw converter with a colorimetric profile will make accurate color.
-
Only possible on Mac ...:(
And your point here being? I pointed to RPP because it is one of the few converters that does not use RGB of any kind as its internal working space - it uses UPLab (http://www.brucelindbloom.com/index.html?UPLab.html). It is quite possible that tonal curve is applied to the L channel there (Iliah can confirm/deny this as I am not entirely sure).
-
I think it's quite clear from the little demonstration that you cannot see the profile and curve separately. That's why I'm skeptical about the practice I see many do in Lightroom and Capture One of changing to linear curve to get "more accurate color" out of the bundled profiles.
The approach is still perfectly possible. Matrix only profile + raw development adjustments in raw converter + assigning LUT colour correction profile in Photoshop to bring colour to accurate values. Having said that it is not that easily achievable in LR.
-
And your point here being?
Hi Alexey,
The point being that I'd love to try/use RPP (as would undoubtedly many others), but without creating a MAC OS environment or having a MAC, Windows users cannot use it. So suggestions that can potentially solve issues, and that only work on a 'limited' number of setups are partial solutions at best.
One of the DCamProf benefits is that it can be (and is) ported to many OS platforms, thanks to Anders' making the source code available for recompilation into complete working binaries.
I pointed to RPP because it is one of the few converters that does not use RGB of any kind as its internal working space - it uses UPLab (http://www.brucelindbloom.com/index.html?UPLab.html). It is quite possible that tonal curve is applied to the L channel there (Iliah can confirm/deny this as I am not entirely sure).
Yes, that may be useful, but I cannot test it on my Windows machines (unless I purchase and invest time in setting up a virtualization environment).
But thanks for the suggestion anyway, maybe Mac OS users can do something with it (or maybe one day there will be an RPP Windows binary?).
Cheers,
Bart
-
I have not had that good results of applying tone curve in L in Lab, the look becomes to desaturated, but you can of course subjectively add in some saturation.
In the posted experiments above the curve is actually always run through RGB first, but only to check at what luminance it gets at then it's done in CIECAM02 J. I did that so the curve would look the same luminance-wise to be easier to compare with the RGB result.
-
I think it's quite clear from the little demonstration that you cannot see the profile and curve separately. That's why I'm skeptical about the practice I see many do in Lightroom and Capture One of changing to linear curve to get "more accurate color" out of the bundled profiles.
Hi Anders,
I do not think that the Linear curve users (I am one of them) in Capture One do it to get "more accurate color", but rather to have better control and rendering over e.g. highlights. With the default (until one changes that) film curve, there is a significant loss of highlight detail compared to less of a 'shoulder' roll-off designed to prevent hard-clipping.
Also, with my Exposed to the Right (ETTR) shots (as verified with Rawdigger, free of important highlights clipping), the default film curve will seemingly clip my (EOS 1Ds3) highlights by close to 1 stop. So I would need to underexpose (with more overall noise as a result) if I wanted to use the Film curve as it is encoded in Capture One.
Therefore, I much prefer using the C1 controls to adjust the tone curve (or even the profile with the ColorEditor>Save as ICC), but starting from a neutral rendering, so I can better see where the real issues are. That choice of curve shape is not because of the color 'accuracy', and C1 has excellent tools for creating pleasing colors, assuming they were not clipped and reasonably good to begin with.
Cheers,
Bart
-
Hi,
I made some test in an IT-8 chart and I got better accuracy in terms of Delta E (CIEDE 2000) with the linear than with default curve in C1.
I absolutely agree with Bart's observation that C1 default curve seems to yield overexposed images. I guess this may explain the reason MFDs are said to provide more DR in the highlights, the default processing and the rated ISO makes us underexpose.
Best regards
Erik
Hi Anders,
I do not think that the Linear curve users (I am one of them) in Capture One do it to get "more accurate color", but rather to have better control and rendering over e.g. highlights. With the default (until one changes that) film curve, here is a significant loss of highlight detail compared to less of a 'shoulder'.
Also, with my Exposed to the Right (ETTR) shots (as verified with Rawdigger, free of important highlights clipping), the default film curve will seemingly clip my (EOS 1Ds3) highlights by close to 1 stop. So I would need to underexpose (with more overall noise as a result) if I wanted to use the Film curve as it is encoded in Capture One.
Therefore, I much prefer using the C1 controls to adjust the tone curve (or even the profile with the ColorEditor>Save as ICC), but starting from a neutral rendering, so I can better see where the real issues are. That choice of curve shape is not because of the color 'accuracy', and C1 has excellent tools for creating pleasing colors (assuming the were not clipped, and reasonably good to begin with).
Cheers,
Bart
-
If you make delta E comparisons with curve you need to disregard from lightness, as the curve by nature will introduce large errors. Possibly the linear curve would show less incorrect color anyway, as the C1 profiles are created for a look its hard to know without testing. It's hard to test a curved look for accuracy as saturation need to be added for compensation of perceptual effects. Standard delta E tests don't work for testing appearance modeled color.
I'm sure though the profiles are designed to be used with the default curve. As they use an RGB tone curve they must choose to design for a specific, can't maintain color in both.
-
Doesn't negative exposure slider settings work in C1 to bring in highlights? Haven't tried recently so I don't know.
I have noted that C1 cuts away quite some highlights per default, it varies depending on camera though
-
Hi,
I presume you are right on all issues. For me, the liner curve have mostly been a way to get rid of Delta E errors induced by he non linearity of the L-component.
What I think I have seen on the C1-profiles that they are over saturating blues and may be greens, so they actually go out of gamut for say Adobe RGB on the IT-8 target.
What I have seen is very large deviation on blue patches, like Delta E (CIEDE 2000) around 17 that were widely exceeding Adobe RGB gamut, but still looking very good on screen.
:-) Erik :-)
If you make delta E comparisons with curve you need to disregard from lightness, as the curve by nature will introduce large errors. Possibly the linear curve would show less incorrect color anyway, as the C1 profiles are created for a look its hard to know without testing. It's hard to test a curved look for accuracy as saturation need to be added for compensation of perceptual effects. Standard delta E tests don't work for testing appearance modeled color.
I'm sure though the profiles are designed to be used with the default curve. As they use an RGB tone curve they must choose to design for a specific, can't maintain color in both.
-
The point being that I'd love to try/use RPP (as would undoubtedly many others), but without creating a MAC OS environment or having a MAC, Windows users cannot use it. So suggestions that can potentially solve issues, and that only work on a 'limited' number of setups are partial solutions at best.
Everyone decides what instruments they need - that includes both cameras and equipment to process the images. I was pointing out that the instrument does exist out there - whether you want to make an effort to use of course is up to you. To me it is just a matter of the choice - same as choosing the camera or lens.
Anyway I am not sure it does exactly what Anders is trying to achieve - perhaps Iliah can clarify it.
-
If you make delta E comparisons with curve you need to disregard from lightness, as the curve by nature will introduce large errors. Possibly the linear curve would show less incorrect color anyway, as the C1 profiles are created for a look its hard to know without testing. It's hard to test a curved look for accuracy as saturation need to be added for compensation of perceptual effects. Standard delta E tests don't work for testing appearance modeled color.
I'm sure though the profiles are designed to be used with the default curve. As they use an RGB tone curve they must choose to design for a specific, can't maintain color in both.
C1 CE edition was supplied by profiles (and curve files) supposedly for reproduction work... both profiles and curves can be used in a regular edition (unfortunately it seems that Lab readouts can't - or at least I did not find a way)
-
I absolutely agree with Bart's observation that C1 default curve seems to yield overexposed images.
I think that curve files in C1 do carry both curves and exposure correction commands inside .frv files.
-
And your point here being? I pointed to RPP because it is one of the few converters that does not use RGB of any kind as its internal working space - it uses UPLab (http://www.brucelindbloom.com/index.html?UPLab.html). It is quite possible that tonal curve is applied to the L channel there (Iliah can confirm/deny this as I am not entirely sure).
if you are talking about the curve that you select in RPP's UI = "Curve Type : Film-like, L*, Gamma, Colorimetric Gamma" , then it is applied to raw channels post per channel multiplication for WB and before demosaicking... hence before a color transform guided by a "camera profile"
-
if you are talking about the curve that you select in RPP's UI = "Curve Type : Film-like, L*, Gamma, Colorimetric Gamma" , then it is applied to raw channels post per channel multiplication for WB and before demosaicking... hence before a color transform guided by a "camera profile"
You absolutely sure it applies before camera profile?
-
You absolutely sure it applies before camera profile?
well, I collected a number of quotes from the authors from raw-rpp.l...j......l.com, plus you can select an output to a .tiff w/o applying any color transform and this control is still enabled - so it works before that.
-
I've worked with reproduction curves now for a while and I think I'm closing in on something that works.
Using faces is unbeatable when it comes to subjectively evaluating color, much easier to see subtle differences there than in other color ranges.
My goal is to make a tone reproduction curve where the contrast itself is a subjective/artistic matter, but hue and saturation is kept perceptually stable, ie the same as with a linear curve. That is you get to subjectively choose a contrast curve, and then color is automatically adapted to match.
The current prototype uses RGB in linear prophoto to calculate luminance change, this is to make the curve "compatible" with RGB tone curves, ie brightness/contrast becomes the same. The actual working space is CIECAM02 JCh though. Hue is kept at the original value, J (lightness) adjusted according to the curve, and C (chroma) is increased as a function of midtone contrast. Using multiplication rather than addition seems to work best for chroma increase, that is more saturated colors get more increase than less saturated colors. Some fine-tuning left to do though, and then implement in DCamProf.
It's a bit scary to do this as one have to trust one's eyes, there's nothing to verify against if the result is "correct" :)
RawTherapee already has this type of curve, "weighted standard" it's called (try it out if you like), it often produces good results but is a bit low on saturation easily noted on skin tones. In the RT team we're looking at adding a new curve with better properties. Not sure my curve will do then as it's a bit slow, at least in my current implementation. We'll see...
-
well, I collected a number of quotes from the authors from raw-rpp.l...j......l.com, plus you can select an output to a .tiff w/o applying any color transform and this control is still enabled - so it works before that.
I thought that at least for matrix profiles the input needs to be linear.
-
I thought that at least for matrix profiles the input needs to be linear.
still, AT = http://raw-rpp.livejournal.com/6180.html?thread=105252#t105252
-
still, AT = http://raw-rpp.livejournal.com/6180.html?thread=105252#t105252
He does not say anything about profile - only demosaic though I can see how that can be implied.
-
He does not say anything about profile - only demosaic though I can see how that can be implied.
you are not saying that color transform is applied in RPP to vectors like (r, 0, 0, 0) or (0, g1, 0, 0) or (0, 0, b, 0) or (0, 0, 0, g2), are you ? so that is how it is implied and I think I have some more quotes along those lines...
-
you are not saying that color transform is applied in RPP to vectors like (r, 0, 0, 0) or (0, g1, 0, 0) or (0, 0, b, 0) or (0, 0, 0, g2), are you ? so that is how it is implied and I think I have some more quotes along those lines...
I am saying I don't know and that you are probably right. I'll have a look at it under Hopper a bit later
-
> I thought that at least for matrix profiles the input needs to be linear.
Depends on CMM.
Yes, user-selectable curves are applied before assigning profiles in RPP; and yet another curve is applied after the profile is assigned to compensate for certain ColorSync weakness - except if the selected curve is "colorimetric gamma". Simulation (film) profiles are a different story.
-
> I thought that at least for matrix profiles the input needs to be linear.
Depends on CMM.
Yes, user-selectable curves are applied before assigning profiles in RPP; and yet another curve is applied after the profile is assigned to compensate for certain ColorSync weakness - except if the selected curve is "colorimetric gamma".
Thanks for confirmation Iliah. I was looking (quite some time ago) at a binary with Hopper but have not dug deep to understand differences bewteen curves applied at various stages. I was not referring to simulation profiles at all (those are quite different beasts even between generation 1 and 2 from what I could see).
-
yet another curve is applied after the profile is assigned to compensate for certain ColorSync weakness
isn't that increasing some math errors, curve after curve ...
-
Thanks for confirmatrion Iliah.
may be they, knowing what is going to happen when ColorSync CMM will apply the trc(s) from some camera profile - which is known at that moment from what user selects, do some pre-compensation in advance to negate Colorsync and then only apply the real curve 8) ...
-
> isn't that increasing some math errors, curve after curve
We know what we are doing ;)
-
Okay now I'm hating Adobe for their DNG profile format :)
Their DNG Profile processor requires that all 3D LUT entries for saturation=0 or value=0 must be 0,1,1 that is no scaling of saturation or value can happen for neutral values. That saturation (and thus hue) cannot be changed is sort of logical, as saturation for neutral gray is zero and since it's scaled with multiplication it can't change from zero.
However, Adobe has taken this a step further meaning that you can't change value of grays either. That is you cannot make a middle gray a little brighter or darker, it stays at 1.0. This is less logical, as it is possible to scale those values with multiplication, and indeed RT's DCP code does it, but not Lightroom and I just checked not the DNG reference code either.
I should have checked this earlier. I've just completed a tone reproduction operator implemented in a LookTable (and as the table applies a curve it must scale neutrals), but it won't work in Lightroom... ***arrgghh***. It should be possible to work around though by combining the use of a curve and a table, but it got a bit messier...
-
I have now released version 0.8.0.
The big thing in this release is the neutral tone reproduction operator, that is it's now possible to make a profile that has a tone-curve but still retains neutral color. I will probably fine-tune the tone reproduction operator in coming releases, but it won't change that much. The current version has some issues with blue channel clipping that in some cases can cause some transition issues. Thanks to the blue handling blue skies perform much better than typical profiles so I can't just compress the blue channel without doing something smart. We'll see.
Currently you can only apply the neutral tone reproduction operator on DNG profiles, but I intend to add it for ICC profiles eventually. Although the ICC LUT is simpler to work with than the DNG profile LUTs, it's not well-defined which color-space the curve will be applied in or if it's applied in the ICC or separately, so it's quite some effort to get it in there. It's summer here now so programming will be slower, unless the weather is bad.
If you have no idea what I'm talking about or just want to get some further info I suggest reading the new section on tone curves in the manual, it also contain example images so you get an idea of how it looks and what difference it makes: http://www.ludd.ltu.se/~torger/dcamprof.html#tone_curves
Other smaller features I know some will appreciate is that you can set more DCP tags directly from the command line, like baseline exposure offset and copyright string. Go to the home page to get a full list of changes.
-
0.8.0 build for Windows (mingw = dcamprof.exe + libgomp_64-1.dll + PDF manual / = copy of Torger's web page converted to PDF /) : https://app.box.com/s/97elm0weiiyackz0dopbwk9e6pw04eap
-
those who love smooth graphs shall like this one = B53 Blue 3 - it is a black line on the graph
(http://s4.postimg.org/guhyfng31/chart.png)
I decided to try to play with that
(http://s1.postimg.org/dnrldx14f/leedf.jpg)
-
It looks like the instrument needs cleaning, after all.
so _4_ spectrophotometers later (colormunki, i1Pro2, 2 x Spectrolino - I got one more just in case) here goes XRite
We've been informed that ColorChecker SG charts (January 2014 production date only!) contain a production failure, especially in the black patches (Lightness and Hue shift). These charts can be replaced under warranty (even, if they are out of warranty) via RMA.
Please send us a digital picture of the reverse of your ColorChecker SG target that will show us the production date of your target. The only production date that is affected is January 2014.
no, they are still not willing to share the numbers, but it seems they are willing to exchange the target...
-
so _4_ spectrophotometers later (colormunki, i1Pro2, 2 x Spectrolino - I got one more just in case) here goes XRite
no, they are still not willing to share the numbers, but it seems they are willing to exchange the target...
Jan 2014 production only? Or maybe not...
-
Jan 2014 production only? Or maybe not...
speaking about the proverbial patch (tuft) of wool from a sheep...
-
speaking about the proverbial patch (tuft) of wool from a sheep...
Is yours from Jan 2014?
-
Is yours from Jan 2014?
yes and "they" (CSR at least - it is the same name since the beginning) know that - they asked about the edition even during the first round - this is 3rd round of exchange about the matter already, as I asked somebody else couple of weeks ago to help me to get to the right people in X-Rite (and may be that /person/ helped actually /just in case - thank you, Joshua/)
-
yes and "they" (CSR at least - it is the same name since the beginning) know that - they asked about the edition even during the first round - this is 3rd round of exchange about the matter already, as I asked somebody else couple of weeks ago to help me to get to the right people in X-Rite (and may be that /person/ helped actually /just in case - thank you, Joshua/)
So, they knew it is Jan 2014 beforehand. Does not add much credibility to their "only".
-
So, they knew it is Jan 2014 beforehand. Does not add much credibility to their "only".
at least it seems I have a chance to get a "proper" target, will see how it measures then... albeit if they state that the formulation was changed it still leaves the question about some /data/ basis vs which to compare your own measurements to see if something is out of the ordinary (unless they have a stock of legacy targets from the old days somewhere), like with this one from Jan 2014 - I am curious to see how all the devices I can use will measure the black patches if I /hopefully/ receive a replacement...
-
Released v0.8.1, fixed a critical bug in the tone reproduction operator and improved its extreme highlight rendering
-
0.8.1 build for Windows (mingw = dcamprof.exe + libgomp_64-1.dll + HTML & PDF manual / = copy of Torger's web page and the same converted to PDF /) : https://app.box.com/s/sx1f8asu3ze7zxmnwa96cixjcwceqsec
-
Released v0.8.2 where I added the neutral tone reproduction support also for ICC profiles.
-
Released v0.8.2 where I added the neutral tone reproduction support also for ICC profiles.
JFYI for few users my windows binaries, I will not be able to build this till this evening (EST)...
-
JFYI for few users my windows binaries, I will not be able to build this till this evening (EST)...
Even with the delay, much appreciated. And thank you Anders, for the ICC support.
Cheers,
Bart
-
Even with the delay, much appreciated. And thank you Anders, for the ICC support.
I haven't been able to test it yet in Capture One as my license is on a computer in a different location, and there was some mess with trial version on this box so I gave it up. It should work though, otherwise I'll fix it when I get the opportunity.
If you want an ICC profile designed with the neutral tone reproduction operator it will currently always be applied directly in the LUT, which for C1 means that you must always use the profile with the linear curve as the curve is applied in the LUT itself. I've tested it in RT and it works well there, and I think it should work well also for C1 but I'm not 100% sure as I've not been able to test yet. I may in the future make it possible to not include the curve in the LUT, but in that case one have to know which working space C1 is using for applying the curve and that seems to vary between cameras in some way I don't fully know how it works.
When mentioning RT (RawTherapee) where I'm also a contributor I've today pushed a patch that makes looktables be applied after exposure as they should. Before RT could not handle looktables properly. This means that if you get a build made July 6 or later you will get excellent control of DCamProf DNG profiles, in one and the same profile you can have a matrix-only profile, refined with a HusSatMap to a more accurate colorimetric result, and then a curve and looktable on top for the neutral tone operator, each element can be toggled directly inside RT depending on how you want to use the profile.
-
0.8.2 build for Windows (mingw = dcamprof.exe + libgomp_64-1.dll + HTML & PDF manual / = copy of Torger's web page and the same converted to PDF /) : https://app.box.com/s/t0b7hd7ufvccsvgntr3vt6e3vtouxmsj
-
That's super-cool, I'd love to see some photos when you have it all working!
I assembled my monochromator setup at last with all bits (it took long time to complete and test). Yesterday made a test run to calibrate and took spectral sensitivities of the Kodak ProBack645 (lots of data to process and not sure that it went ok yet - I think I need some adjustments to my light measurement device on Arduino), the transmittance spectra of all filters (IR, AA etc) and transmittance spectra of almost all lenses. I now need to process it all and build a library and experiment with simulation for the camera readings of a target.
That's how the setup looks (sorry for the pictures - took them with mobile).
Monochromator with the integrating sphere and with a lightmeter on Arduino:
(http://www.aletan.com/img/s1/v22/p1305760524-5.jpg) (http://www.aletan.com/img/s1/v22/p1305760524-6.jpg)
(http://www.aletan.com/img/s7/v154/p1305757851-5.jpg) (http://www.aletan.com/img/s7/v154/p1305757851-6.jpg)
And switched on (light is very dim - it was too dark without any light for a photo and nothing visible with full light on):
(http://www.aletan.com/img/s8/v0/p1305757819-5.jpg) (http://www.aletan.com/img/s8/v0/p1305757819-6.jpg)
(http://www.aletan.com/img/s3/v7/p1305757943-5.jpg) (http://www.aletan.com/img/s3/v7/p1305757943-6.jpg)
(http://www.aletan.com/img/s5/v127/p1305757849-5.jpg) (http://www.aletan.com/img/s5/v127/p1305757849-6.jpg)
The monochromator itself (http://www.optometrics.com/monochromators) used in my setup is expensive if bought from new but they could be found as a part of the industrial assembly of Luxtron/Lumasense 1108 (http://www.ebay.co.uk/itm/170625275474) (they can be found on ebay quite cheap every now and then). The integrating sphere was part of the assembly from Photonis XP3112/PB (http://www.ebay.co.uk/itm/271166072389). Arduino device I built and coded myself and as for the light source - any with goose neck fiber optic cable will do (plenty cheap on eBay). The whole setup generally cost a lot less than a new ColorChecker SG.
The calibration procedure is simple. I pre-heat the light source (halogen 150W Philips) to stabilize, turn on my instrument, and then walk through the spectrum (400-730nm) at 5nm intervals measuring spectra with i1Pro in high res mode in Argyll (all in a dark room with insulation of i1Pro and integrating sphere) and record the readings off the Arduino meter (just upper ones - it is just a reading of the full visible range photodiode). All this is then placed in a calibration table and used with readings of the light meter when taking the shots with camera to adjust the light source drifts (and there are definitely drifts there as I discovered).
Knowing the transmission spectra of lenses and filters, it should be possible to make the simulation of a target shot by camera in a certain light (will need to do that experiment with real target).
-
I assembled my monochromator setup at last
beautiful !
-
Wow Alexey, great to see!
-
Since you already employed Arduino, I would be adding a rotary encoder (like those Sparkfun are carrying) to log the wavelengths. Maybe a stepper motor too.
Saying "record the readings off the Arduino meter (just upper ones - it is just a reading of the full visible range photodiode)" - how do you mean "upper" - those "Ful 28" on the second image?
-
Since you already employed Arduino, I would be adding a rotary encoder (like those Sparkfun are carrying) to log the wavelengths. Maybe a stepper motor too.
Saying "record the readings off the Arduino meter (just upper ones - it is just a reading of the full visible range photodiode)" - how do you mean "upper" - those "Ful 28" on the second image?
Perhaps one day - I'd like to get the feeling of this whole thing first :). For now I employ a very "sophisticated" method of using pocket torch between the measurements to adjust the dial ;)
Re photodiode - I used Adafruit lux sensor (same as sparkfun uxing tao chip) that has 2 diodes in a package (full visible and ir). I used calculations from their library to calculate 4 values: full range(raw reading), ir,fill-ir and calculated lux (just for fun). I'm using fuĺl range reading - is that wrong?
-
> Perhaps one day - I'd like to get the feeling of this whole thing first
I agree. Just saying how I do those things (though I use Linux boards, not Arduino, unless I need real time - and that case I use small Atmel or PIC boards or chips as slaves to some Linux board; humidity sensors in my film drying cabinet would be one of the examples, power supply control to stabilize light is another one).
> I used Adafruit lux sensor (same as sparkfun uxing tao chip) that has 2 diodes in a package (full visible and ir). I used calculations from their library to calculate 4 values: full range(raw reading), ir,fill-ir and calculated lux (just for fun). I'm using fuĺl range reading - is that wrong?
Not at all, it is perfectly OK. I was just asking because your reference to "upper" did not came clear to me. What you did with a photodiode is a classic setup, same one I use - though I use a different diode. Never occurred to me to try a TAOS chip in this application.
-
> I agree. Just saying how I do those things (though I use Linux boards, not Arduino, unless I need real time - and that case I use small Atmel or PIC boards
> or chips as slaves to some Linux board; humidity sensors in my film drying cabinet would be one of the examples, power supply control to stabilize light is another one).
I am not that experienced with hardware so Arduino seemed a no brainer for me ;).
> power supply control to stabilize light is another one).
I am quite interested in that (assuming you are talking about halogen sources) - would you point me in a right direction?
> Not at all, it is perfectly OK. I was just asking because your reference to "upper" did not came clear to me. What you did with a photodiode is a classic setup,
> same one I use - though I use a different diode. Never occurred to me to try a TAOS chip in this application.
I had the chip assembly left from experiments with SLR/n nonlinearities in the last 1/6 stop (we discussed it a while ago if you remember) so it was an easy choice.
I had extracted my ProBack spectras tonight and they are rubbish. The first attempt went astray so I will need to redo it. I'd need to correct a few things as well:
- I used Kodak firmware capacity to subtract black on long exposures but it still generates a lot of noise so will need to do black subtraction myself
- have not closed viewfinder (I know its stupid but I forgot) for the 400-450 range so blue is captured quite badly
- I need to invent something to connect camera to integrating sphere more tightly to prevent even slightest leaks
- And add baffle to either entry from monochromator or photodiode (the latter seem to be influenced a bit by entry light positioning and sits opposite exit from monochromator albeit offset a bit).
-
Thought I'd post an example. I haven't actually made that many finished profiles as I've always been busy fixing something or adding some new feature.
Attached is an example dual-illuminant DNG profile for Canon 5D mark II, made using DCamProf v0.8.2 with the workflow outlined in the code block. It uses a CC24 for base colors combined with a homemade semi-glossy inkjet target for extreme colors. Light sources are halogen with voltage control, a normal lamp for StdA and a Solux on overdrive for D55. The uneven light (only one point light) has been corrected with DCamProf's flatfield correction function.
A curve was designed "by eye" in RawTherapee to reasonably match the camera's JPEG brightness/contrast, so we get predictable result when using the camera's auto exposure. The Canon does quite some black subtraction to blacken shadows in its JPEGs, I've chosen to have lower contrast shadows and keep detail. Then this curve is applied when generating the profile using DCamProf's neutral tone reproduction operator.
CAT02 was used for StdA apperance rendering, while Solux "D55" was deemed to be close enough to D50 so no CAT was used there (-C).
The DCP contains two pairs of matrices (StdA and D55) which aim to make best match for the normal range of colors (represented by the CC24), the glossy target has been excluded from matrix correction. Then there are two 2.5D HueSatMaps (StdA/D55) for further colorimetric correction, and which also stretches the borders to decently match the glossy colors. Weighting and relaxation has been used to (my) taste to make a tradeoff between smoothness and accuracy.
Then we have the Looktable+Tone Curve which implements the subjective contrast increase and with maintained neutral reproduction. The Looktable has its value dimension gamma-encoded for better perceptual resolution. The looktable is what makes the profile fairly big, as it must be a 3D LUT. Finally there's a baseline exposure offset of -0.3, which is supposed to be added on top of Adobe DNG Converter's baseline exposure of 0.4 stops it adds for 5Dmk2 files. If you don't use Adobe-generated DNGs (ie have 0 baseline exposure) that should be changed to +0.1 stops.
If you have "toggable" DCP handling like a RawTherapee built after July 6, you can turn off the Looktable+Curve to get a colormetric profile, and then turn off the HueSatMap to get a pure matrix profile. The elements are sort of layered, first matrix, then huesatmap on top and then looktable and then curve. I like that aspect of DNG profiles, but unfortunately this aspect is rarely used in profile design, and I think only RT supports toggling the elements in it.
# 1. Make a custom target to top of the CC24 with some super-saturated colors
dcamprof make-testchart -l 15 -d 14.46,12.26 -O -p 210 custom-target.ti1
printtarg -v -S -iCM -r -h -T300 -p A4 custom-target
# 2. Print custom target to a semi-glossy OBA-free paper
# 3. Scan the target with a spectrometer
chartread -v -H -T0.4 custom-target
# 4. Create reference file
spec2cie -v -i D50 custom-target.ti3 glossy.cie
# 5. Setup light and measure spectrum, save to light.sp for later use
spotread -a -H -x
# 6. Shoot CC24 target, glossy target, and white card.
# 7. Crop export and convert to cc24.tif, glossy.tif and ff.tif
# 8. Apply flatfield
dcamprof testchart-ff cc24.tif ff.tif cc24-ff.tif
dcamprof testchart-ff glossy.tif ff.tif glossy-ff.tif
# 9. Scan values
scanin -v -dipn cc24-ff.tif ColorChecker.cht cc24.cie
scanin -v -dipn glossy-ff.tif custom-target.cht glossy.cie
# 10. Merge targets into one, letting CC24 have priority
dcamprof make-target -p cc24-ff.ti3 -a cc24 -p glossy-ff.ti3 -a glossy -d 0.03 combo.ti3
# 11. Make a preliminary profile, dumping plots. Exclude glossy
# from the matrix optimizer to get as good base match as
# possible for the important normal colors represented by cc24
# Add -C (no CAT) to all make-profile below if your light is
# close to D50.
dcamprof make-profile -r dump1 -w cc24 0,1 -w glossy 0,0 -i light.sp combo.ti3 preliminary.json
# 12. Start gnuplot (cd dump1; gnuplot -background gray) and plot
# target and LUT, plus LUT stretch vectors and DE vectors.
# Use 'set view equal xyz' and 'set view equal xy' to turn
# on/off scaling of lightness axis, must be turned off if
# error vectors are viewed in 3D.
gnuplot> splot 'nve-lut.dat' w l lc "beige", 'gmt-locus.dat' w l lw 4 lc rgb var, \
'gmt-adobergb.dat' w l lc "red", 'gmt-pointer.dat' w l lw 2 lc rgb var, \
'target-nve-lutve2.dat' w vec lc "black", 'target-nve-lutvm.dat' w vec lw 2 lc "olive", \
'targetd50-xyz.dat' pt 4 lc rgb var, 'targetd50-xyz.dat' using 1:2:3:5 w labels offset 3
# 13. Look in the plot for patches that pull in opposite directions and cause a
# bad bend in the LUT. Add those (typically one or two) to an
# exclude.txt and render
dcamprof make-profile -r dump1 -x exclude.txt -w cc24 0,1 -w glossy 0,0 -i light.sp combo.ti3 preliminary.json
# 14. Make matrix-only and full correction profiles for sanity
# check comparisons later
dcamprof make-dcp -n "Canon EOS 5D Mark II" preliminary.json no-relax.dcp
dcamprof make-dcp -n "Canon EOS 5D Mark II" -L preliminary.json matrix.dcp
# 15. Relax the LUT (primarily for the glossy class) to improve
# smoothness. Re-render and replot for each change.
# a) It may be worthwhile to lock the matrix before changing DE k
# weights: save profile to separate file matrix.json and
# provide -m and -f parameters.
# a) Try relaxing DE weight even for the important CC24, setting
# it to at least 1 leads often to some relax without much loss
# in accuracy.
# b) Try changing CIEDE2000 k weights, 4,1,1 good start (less
# weight on lightness)
# Example result after iterating:
dcamprof make-profile -r dump1 -x exclude.txt -f matrix.json -m matrix.json \
-w cc24 1,1,4,1,1 -w glossy 2,0,4,4,1 -i light.sp combo.ti3 final-1.json
# 16. Make final DCP, sanity check it by comparing it with
# no-relax.dcp and matrix.dcp. It should have better high
# saturation correction than matrix.dcp, and not lose too much
# accuracy compared to no-relax.dcp
dcamprof make-dcp -n "Canon EOS 5D Mark II" -d "Final 1" final-1.json final-1.dcp
# 17. Repeat steps 5 - 16 for the second illuminant
# 18. Decide which tone curve to use, which baseline exposure
# offset if any, and if black subtraction should be
# automatically applied or not. Often the default "acr" curve
# with no baseline exposure offset provides what you want,
# unless you more closely want to match camera's JPEGs. If you
# prefer to make visual accuracy comparisons with tone curve
# applied, you can apply the curve earlier in the process.
# 19. Merge to a dual-illuminant profile and apply your curve with
# DCamProf's neutral tone reproduction operator (enabled per
# default).
dcamprof make-dcp -n "Canon EOS 5D Mark II" -d "My Profile" -i StdA -I D50 -b 0.1 -t curve.rtc final-1.json final-2.json final-dual.dcp
-
>> power supply control to stabilize light is another one).
> I am quite interested in that (assuming you are talking about halogen sources) - would you point me in a right direction?
Much depends on the power supply you are using currently. If it is not a stabilizing one, you can use a dimmer with a sine wave output in front of it and control the dimmer using the feedback from the diode. If it is stabilizing, you may want to replace it with a regulated power supply, or introduce a dimmer right after it.
-
> I am quite interested in that (assuming you are talking about halogen sources) - would you point me in a right direction?
Much depends on the power supply you are using currently. If it is not a stabilizing one, you can use a dimmer with a sine wave output in front of it and control the dimmer using the feedback from the diode. If it is stabilizing, you may want to replace it with a regulated power supply, or introduce a dimmer right after it.
No idea really, the one that is in the light source and it seems a rather primitive one: transformer with small board and regulator which provides 0..+17.6V on lamp connector.
-
No idea really, the one that is in the light source and it seems a rather primitive one: transformer with small board and regulator which provides 0..+17.6V on lamp connector.
Than you may want to see if you can control the regulator, if it is a variable resistor you may be able to replace it with either analog output of Arduino, or with a digital potentiometer like https://www.sparkfun.com/products/10613
-
Than you may want to see if you can control the regulator, if it is a variable resistor you may be able to replace it with either analog output of Arduino, or with a digital potentiometer like https://www.sparkfun.com/products/10613
Yep its a variable resistor (old and large). At first attempt I was trying not to change the light intensity - but set it to max output at 17.6V (I have Philips halogen bulb - this one (http://www.lighting.philips.com/main/prof/lamps/special-lamps/optical-medical-equipment/halogen/halogen-reflector/923915819104_EU/product)), measure all SPDs and photodiode output with 5nm steps, then take a series of raws and correct them following the measured SPDs (to max SPD and adjusting by photodiode measurement deviation). The light stayed the same throughout the experiment.
If I understand you correctly you are saying that I need to build a calibration table and then attempt to control the light output with it using photodiode as a feedback adjusting to the same level (presumably blue one as it is the weakest)? If this is correct then I'd need that rotary encoder to know what wavelength it is at since photodiode output although linear is different in different parts of the spectrum.
-
First, I would inspect the resistor (may take de-soldering), read the markings, see how it is used (potentiometer, rheostat with two pins connected together, or one pin free). It is also important to measure the current through the resistor. Next, I would figure out how to replace it with a processor-controllable element.
The goal is to maintain the light flux. This is usually achieved through a simple PID regulator (many implementations are available for microprocessor, it is bread-and-butter code - an example http://playground.arduino.cc/Code/PIDLibrary ). The input to the regulator is the measurement of the light (diode reading), the output is the "variable resistor" control signal.
-
Thanks Iliah will do that
The goal is to maintain the light flux.
Do you mean luminous flux or radiant flux? The photodiode does not allow the radiance hence the tables and SPD or am missing something and the simple alignement of light output to have the same photodiode readings will do?
-
> the simple alignement of light output to have the same photodiode readings will do
Yes, that will do, works from the times of early "colour stabilizers" in enlargers. If one wants to go overkill, TCS3471x / TCS3472x and its variations are just what the doctor ordered :)
-
> the simple alignement of light output to have the same photodiode readings will do
Yes, that will do, works from the times of early "colour stabilizers" in enlargers. If one wants to go overkill, TCS3471x / TCS3472x and its variations are just what the doctor ordered :)
So with this then there is no need to scale raw values to adjust for intensity difference (provided all exposures were the same)? If so great - that is easier for me than taking 66 i2pro measurements and processing them all. Thanks Iliah for the help
-
> So with this then there is no need to scale raw values to adjust for intensity difference (provided all exposures were the same)?
Generally, yes; subject to periodic testing, taking SPD, and calculating CCT to log the aging of the lamp (I do it after each 10 hours of use approximately).
-
at least it seems I have a chance to get a "proper" target, will see how it measures then... albeit if they state that the formulation was changed it still leaves the question about some /data/ basis vs which to compare your own measurements to see if something is out of the ordinary (unless they have a stock of legacy targets from the old days somewhere), like with this one from Jan 2014 - I am curious to see how all the devices I can use will measure the black patches if I /hopefully/ receive a replacement...
so after some ping-pong with X-Rite's customer service about RMA & proof of purchase I received today a replacement for my ColorChecker SG target... the one that was sent by X-Rite is "November 2014" edition (as printed on the back of the target) now, so this evening I will try to measure it and see how the patches are there (or rather how my spectrophotometers will be metering them).
-
So, they knew it is Jan 2014 beforehand. Does not add much credibility to their "only".
I did one measurement (did not have time to do several to average yet) with a newer out of 2 spectrolinos that I have = attached
Results looks "better", at least black patches are darker in November 2014 edition of CC SG.
-
I've just released version 0.8.3: http://www.ludd.ltu.se/~torger/dcamprof.html
The main thing is that I've reworked the tone reproduction operator as a part of the work of providing a realtime version for RawTherapee (which I just committed too). It works along the same principles and the result of it is almost the same as before, but there are some slight improvements especially concerning highlight rolloff. I've had some extra expert eyes to help me fine-tune weights. It has a bit better skin tone performance now.
-
0.8.3 build for Windows (mingw = dcamprof.exe + libgomp_64-1.dll + HTML & PDF manual / = copy of Torger's web page and the same converted to PDF /) : https://app.box.com/s/6dbmzwrvucjz6tprdhvit2cnjpdn3mo1
-
Seems like tone reproduction can keep one busy for a while.
With DCamProf the intention has from the start to make neutral profiles, but as soon as contrast should be applied it's impossible not to bring some taste to the table.
To keep color appearence with increased contrast I've noted that non-linear 3D tables are required. So far I keep hue stable, but saturation is changed depending on contrast, and also depending on lightness (more saturation increase of dark colors) and source saturation. This is not to create a "look", but to keep color appearance as constant as possible.
With the latest release I've made it possible for the user to tune all these weighting parameters with a JSON configuration file.
Another big thing when it comes to tone reproduction is highlight rolloff. It's often possible to retain the correct color close to clipping, but I've noted that there will then be problems with the look. We can't change that the brightest spot on the media is white, so making a nice transition to the whitepoint is an important aspect, even if it hurts color precision.
As said DCamProf doesn't attempt to do any subjective looks so far. I might add features for that at some point, but it will be difficult to use without a GUI, I would have to provide pre-made recipes that the user can adjust to taste. A typical base recipe is to warm up highlights/cool down shadows, limit saturation of high saturation colors, increase saturation of mid-saturated colors (being careful with skintones). I think it's better to control the look in the raw converter, but unfortunately few raw converters have tools that allow these types of fine adjustments.
The attached image shows a comparison between a DCamProf profile as used in RT and Hasselblad's native Phocus rendering and illustrates a neutral vs subjective look, the camera is a H4D-50. At first glance the images are very similar, and one reason for that is that I have matched the contrast curve quite well (I like my profiles to provide the same contrast as the native camera images so default rendering becomes predictable). Another reason is that Hasselblad's color is quite sane, not crazy pop as you can see in some consumer cameras, and is thus fairly close to the neutral result.
However looking closer we see some differences, the main one that it seems like the Hassy image has a warmer white balance. This is not the case though, the white balance multipliers are exactly the same, and if we look closer at neutrals and shadows the white balance seems the same, what the Hassy look does is to warm up some of the saturated tones. The Hassy look has a little bit more saturation overall, but the difference is certainly not huge. Warming up of tones does cause some notable hue errors, here seen in the red pants that look more yellow-red rather than the actual wine-red.
The example image is not good for comparing skin tones, but anyway I've looked at other more suitable pictures for that and quite big differences can seen there, much more than in the attached example. Hassy's subjective look make caucasian skin quite a lot more yellow/golden, while a neutral look has more red as there is in reality. Golden skin looks smoother though so I'd expect many prefer that look, and that's why Hassy have hard-coded it into their profile.
Since I've published DCamProf I've got quite many requests on making profiles for camera XYZ as the manufacturer provided profile "is no good". I've been a bit reluctant to help as I know there is a big chunk of subjectivity involved. Making a neutral realistic profile is now quite easy with the tone operator in place, but I'm not so sure that is what most people want especially when it comes to skin tones.
-
I got a request to be able to disable the lightness correction all-together, and with the new 0.8.4 release this is now possible, just specify "-l -1,0" to make-profile and the LUT will only work with chromaticity.
As a sidenote this is how Adobe's DNG Profile Editor does, the LUT only corrects hue and saturation and leave lightness (actually RGB-HSV value) unchanged, or rather leave it at the result provided by the matrix. There is some robustness advantages to this, measurement errors in lightness is larger (I think) than in chroma and hue and there seems to be some robustness advantages in handling of clipped highlights.
-
0.8.4 build for Windows (mingw = dcamprof.exe + libgomp_64-1.dll + HTML & PDF manual / = copy of Torger's web page and the same converted to PDF /) : https://app.box.com/s/if132xhpl94u32jb4kg9h43oaz197xc9
-
Thanks guys, for the ongoing improvements, and Windows binaries.
Cheers,
Bart
-
Here's a guess game for you;
I'm working with subjective tuning features in DCamProf so you can make the profile render "flattering" skin tones rather than accurate (and any other subjective tunings you may want to do). It's not exactly easy tuning without a GUI, but it can be done. You write a specification of the subjective adjustments in a JSON config file.
The attached image shows three images, one is a neutral DCamProf profile with a tone curve using the neutral tone reproduction operator, one is my work in progress concerning hand-tuned skin tones using DCamProf, and one is a Hasselblad Phocus original rendering. The camera is a H4D-50.
So which one is which? And which skin tone rendering do you prefer?
-
Can't tell you which one is which but I prefer 'A''s skin tone.
Based on 40 years of observation I can say all skin has some yellow in it at any given time of day, shade or sunshine unless the subject is an albino.
'B' & 'C' are a bit too pinky where I see the tuft of hair just above the forehead has a magenta tinge. Being able to get rid of any Rosacea or other skin conditions that show pinkish hues in a profile is a big plus. I have to make hue adjustments constantly using ACR/LR's HSL.
Good work, torger. Thanks for posting samples. It helps me make sense of this thread.
-
Agree with Tim, skin needs to contain a little yellow, even for Northern European children. Hemoglobin + Melanin + skin thickness does that. "A" for me, too.
-
Hi,
I guess I like 'A' best followed by 'B'.
I agree with Tim and Iliah on what has been said. But, I have little experience with skin.
Best regards
Erik
Can't tell you which one is which but I prefer 'A''s skin tone.
Based on 40 years of observation I can say all skin has some yellow in it at any given time of day, shade or sunshine unless the subject is an albino.
'B' & 'C' are a bit too pinky where I see the tuft of hair just above the forehead has a magenta tinge. Being able to get rid of any Rosacea or other skin conditions that show pinkish hues in a profile is a big plus. I have to make hue adjustments constantly using ACR/LR's HSL.
Good work, torger. Thanks for posting samples. It helps me make sense of this thread.
-
My preference would be "B" as the one I perceive is most accurate.
"A" looks nicer to me, but the lighting looks pretty cool and the skin tone looks warmer (almost unnaturally so). "A" looks more like what I would expect with the sun out (more yellow, less blue).
Andy
-
Here's the answer:
A: DCamProf with the neutral tone reproduction operator, that is the most "accurate" colors, no manual tunings (except for the smoothness/accuracy tradeoff during colorimetric design)
B: DCamProf using the upcoming feature set that allows hand-tuning colors, here with my own skin-tone tunings
C: Hasselblad Phocus original rendering
Personally I think Hasselblad puts too much pink/magenta into skin tones, but it will depend on the light how they appear. Fixing things in a profile is a crude way to do things as it will be static adjustments which will work better in some images and worse in others so I'm not really sure it's a good idea at all. It's how every commercial raw converter does it so I thought it would be interesting to provide the possibility to do the same with DCamProf.
The adjustments I've made: compressed the skin tone hue range so that reds and yellows are brought closer together, adjusted the overall tone, added a tad more saturation on low saturation skins (to avoid grayish look on pale people). The range that fall on lips (and often cheeks) has got a tiny magenta and lightness boost (less so than Hassy's rendering) which is supposed to separate lips more and add a "freshness" to the skin.
The midtones and highlights has got a warming up, but that's for landscape (one can see it on the tone of the wall), I'm not really happy with that adjustment yet so I may revert/change that part and adjust skin tone range accordingly so it stays about the same as now. I'm quite pleased with the current skin tone result, but I'm not sure I'd use it myself over the original neutral rendering. I just want to test and see that it can be done and have an example file before I release this functionality.
The white balance is same for all (daylight) but since it's in the shade the look is a little bit on the cool side, maybe should have had a shade white balance here but it was a bit too yellow I thought. I've used several other shots in other light conditions as well when doing this tuning. I took this one as an example to show as the differences where a bit easier to see here. In bright sunlight the results are a bit closer. The wall is in reality not really neutral gray but has a bit of blue in it which may amplify the cool look.
It's interesting that most of you prefer the A rendering. Maybe the secret to skin tones may just be to render them as accurately as possible :-)
-
I was going to say I think A is hasselblad and C neutral. But clearly that was wrong. B has that tinge to the hair that made me think it is a work in progress.
-
I was going to say I think A is hasselblad and C neutral. But clearly that was wrong. B has that tinge to the hair that made me think it is a work in progress.
Well spotted, I'd stared so much on the skin I did not notice that. I thought it was a moire effect at first, but it's most likely the lip/cheek magenta boost that taints the hair. It's the generic problem of subjective profiles, no local adjustments are possible so any colors in the same range will be modified even if it's not skin.
Another issue is that subjective profile adjustments are also very sensitive to white balance setting as it moves around the colors, that is the skin tone hue range varies a bit depending on white balance.
-
> Maybe the secret to skin tones may just be to render them as accurately as possible :-)
It is a cultural thing. My Far East customers always prefer less saturated skin tones on their portraits.
-
> Maybe the secret to skin tones may just be to render them as accurately as possible :-)
It is a cultural thing. My Far East customers always prefer less saturated skin tones on their portraits.
did they ask you to photoshop their eyes too :) ?
-
I've just released v0.9.0, the big new things is of course the "look operators" feature so you can design a look. It's really not intended to make crazy "filters", but really just very subtle adjustments from the default neutral look.
Probably it is like only myself that will ever use it :) , because it's really tedious to work with without a GUI, but if you're the patient type of person you can do it.
I've included one example look, which is an updated version of the one posted in an example a few messages back. It should be used as a template example only, as adjustments are really fine it will probably vary if it will look well or not on a profile made for a different camera, and also depending on the chosen contrast curve.
It's also a time-wasting experience to tune these looks, you can get stuck like forever on making an almost invisible adjustment, so I just said "enough is enough" and made the release instead :). For the specific test images I've had I think my look became more pleasing than Hasselblad's own, but it's of course a matter of taste :).
-
did they ask you to photoshop their eyes too :) ?
Don't know about their eyes, but I saw a report that they do go to the beach at night, to avoid a suntan...
And to get back on topic, I feel an urge rising within myself for some sort of GUI, but I do not have the time right now :(
Maybe there are other takers?
Cheers,
Bart
-
0.9.0 build for Windows (mingw = dcamprof.exe + libgomp_64-1.dll + HTML & PDF manual / = copy of Torger's web page and the same converted to PDF /) : https://app.box.com/s/z5zu2zsw84tbiof65esgn66d8uz6cz2u
-
And to get back on topic, I feel an urge rising within myself for some sort of GUI, but I do not have the time right now :(
Maybe there are other takers?
too few users I think... I can see that for example the 0.8.4 build for Windows was downloaded 12 times... OK, some are on Mac (or even Linux/whatever) and some do build themselves... but I 'd guess not more than 50 users at all in a best scenario... fewer are actually using it so regularly...
-
When it comes to GUIs I'd prefer to make it myself in my company and in that case it would be a commercial product, but of course anyone's free to make a GUI for the current program following the GPL terms.
Looking at hits on the web page I'd say the interest is quite broad, but few users have even seen a command line so those that actually proceed to download are quite few. If I'd really make a GUI for the complete workflow, which is quite a big job, I think it would see quite many users.
But it's a speciality tool you wouldn't use often, and still requires quite some skill and patience to use even with a GUI so it wouldn't be any big volume software for sure.
The command-line only Argyll has a stable following though, so we'll see in a few years. DCamProf is still new software.
The only part that is truly tedious without a GUI is the subjective design, all other aspects would not really be that much simpler with a GUI. Concerning the design aspect I do it in RawTherapee where you easily can load new profiles. I render with monochrome selections to fine-tune which gamut area an operator should apply on, and once I've done that I start tuning the actual operator values.
-
Just a note that there might be more people who watch the thread but don't download/participate just yet. Like myself.
It seems like a terrific tool and I'd like to compile it for Mac in the future.
-
Just a note that there might be more people who watch the thread but don't download/participate just yet. Like myself.
I'm in this category. I've been watching this thread like a hawk, but haven't done anything with DCamProf yet because I'm still working on getting a handle on Argyll recipes (http://forum.luminous-landscape.com/index.php?topic=97262.0).
Yeah, once you have the recipes worked out, there is no real need for a GUI. (At least for Argyll.)
Wayne
-
I am working on a tutorial article which is supposed to provide a step-by-step guide of how to design a high end profile using DCamProf, but it's still a fairly long way to go. Hopefully within a few months. There's so much to investigate while doing it, sometimes ending up in having to develop yet another feature in the software, that is what makes the time fly...
-
I've compiled the sources on a Mac with OpenMP. Feel free to use it and link it directly.
http://projectsbin.com/resources/dcamprof (http://projectsbin.com/resources/dcamprof)
-
Working on a new release with some new features, but now just ran into a pretty severe limitation of the DNG profile format, or actually the DNG pipeline which applies it.
The DCP LUT is HSV-based. Hue is changed by adding an offset. To interpolate between table entries you calculate an average of the neighbors.
Say if entry A says hue -30 degrees, and entry B says +40 degrees and we want to interpolate exactly inbetween, then we get (-30 +40) / 2 = 5 degrees, just as we would expect. But say entry A is -170 degrees and entry B is +160 degrees, then we get (-170 + 160) / 2 = 5 degrees, which indeed numerically the average but as hue goes around 360 degrees the average should be (360-170 + 160) / 2 = 175 degrees in his case.
These cases could easily be handled when the LUT is applied, the problem is that the DNG reference code doesn't do that, and I assume Adobe Lightroom doesn't either (haven't tested Lightroom yet). This means in practice that hue rotation is limited to the range -90 to +90, which indeed should be enough for making colorimetric corrections, but is an issue when designing looks, even with very mild looks there is a significant risk to run into this discontinuity. The reason for this is that for lower saturation colors hue rotation can be quite huge in the RGB-HSV space despite that there is a small adjustment in visible space.
It's actually not a limitation of the LUT format itself, but how the LUT is applied. Forgetting about hue angle discontinuity is a classic bug when working with a coordinate system like HSV, JCh or other that has hue defined as an angle; been there, done that. It's a bit unfortunate that Adobe has limited their DNG profile format by having this bug in the DNG pipeline, and I'm quite sure that they'd call it a "feature" themselves as you can optimize LUT application if you don't need to care about discontinuity.
I have not yet figured out how to deal with this. If you don't use the look operators it's quite unlikely that you run into it though.
EDIT: you're not limited to +/-90 degrees (although that will guarantee that you are safe), what you need to avoid is discontinuity, that is a jump from say +179 to -179 between table entry neighbors.
-
Just released version 0.9.1
The hue discontinuity problem can't really be worked around, but it's a quite small problem. By adding an RGB blend mode and HSV axis in the look operators it's quite easy to make subjective looks that doesn't trigger hue discontinuity.
DCamProf will now detect hue discontinuity in the make-dcp command and abort if found.
I've added TIFF file versions of the patch matching reports so you can see those diagonally split square patches like various patch tools show.
The test-profile command can now be run without a target if you just want to render plot files, and a new generated gradient file which can be used for visual inspection of smoothness. It's very useful when designing a look using look operators.
-
0.9.1 build for Windows (mingw = dcamprof.exe + libgomp_64-1.dll + HTML & PDF manual / = copy of Torger's web page and the same converted to PDF /) : https://app.box.com/s/lr054ihziq8iv248a780q67fveo7boqx
-
Done some experiments with Capture One again (it was a while ago). Anyway I've discovered an issue, well I kind of new about it but had forgot;
When using DCamProf profiles in C1 high saturation blues sometimes clips to black.
This doesn't happen when the same profile is used in RawTherapee.
I think I know what the reason is. The ICC profile defines colors with a LUT from Raw RGB to CIELab. Then C1 converts Lab (which is virtually boundless) to some sort of RGB for output. If the Lab color is outside the target RGB gamut it will clip, and for blue this seems meaning to go from 100% blue down to 0%. Why this happens in C1 and not RT (except if you artificially push it) is because C1 is using a small RGB space and RT is using ProPhoto.
This means that for Capture One the ICC profile must do some sort of gamut mapping to avoid clipping the output space.
The problem is that I don't know what Capture One's output RGB space is (so I don't know what to gamut map against), and if it's the same for all camera models. Anyone that knows?
-
The problem is that I don't know what Capture One's output RGB space is (so I don't know what to gamut map against), and if it's the same for all camera models. Anyone that knows?
do you mean that C1 does the final output in 2 steps : -> some unknown intermediate RGB space with smallish gamut -> RGB space designated by you, yourself, for output in C1's UI ?
-
Done some experiments with Capture One again (it was a while ago). Anyway I've discovered an issue, well I kind of new about it but had forgot;
When using DCamProf profiles in C1 high saturation blues sometimes clips to black.
This doesn't happen when the same profile is used in RawTherapee.
I think I know what the reason is. The ICC profile defines colors with a LUT from Raw RGB to CIELab. Then C1 converts Lab (which is virtually boundless) to some sort of RGB for output. If the Lab color is outside the target RGB gamut it will clip, and for blue this seems meaning to go from 100% blue down to 0%. Why this happens in C1 and not RT (except if you artificially push it) is because C1 is using a small RGB space and RT is using ProPhoto.
This means that for Capture One the ICC profile must do some sort of gamut mapping to avoid clipping the output space.
The problem is that I don't know what Capture One's output RGB space is (so I don't know what to gamut map against), and if it's the same for all camera models. Anyone that knows?
I'm pretty sure that C1's native color space is ProPhoto, if that is what you're looking for.
-
I'm pretty sure that C1's native color space is ProPhoto, if that is what you're looking for.
what is the basis of being "pretty sure" though ?
for example http://help.phaseone.com/en/co6/output/learn-more-about-file-formats/colors-in-capture-one.aspx?p=1
Capture One works in a very large color space, similar to that captured by camera sensors. A large color space ensures that little clipping of the color data can occur. Clipping is the loss of image information in a region of an image. Clipping appears when one or more color values are larger than the histogram (color space of the output file).
At the end of the workflow, the RAW data has to be processed to pixel based image files, in defined color spaces. These spaces are smaller than the internal color space used by Capture One.
you can output to ProPhoto RGB for example, but it is claimed to be smaller ;) than whatever C1 is using...
so do we have any P1 people on record or something of the same quality ?
-
I'll back off a little from the claim that it's a RGB space clipping problem. The profile itself does clip to black in the blue area as it goes outside the valid XYZ range, for strange input like raw R = 0, G = 0, B = 100. I suspect that if I smooth away the "black hole" in the LUT it will work fine in C1 regardless of their output space. Still would be nice to know what it is.
Smoothing/interpolating away the black hole is easier said than done though... I'll experiment some more.
-
what is the basis of being "pretty sure" though ?
for example http://help.phaseone.com/en/co6/output/learn-more-about-file-formats/colors-in-capture-one.aspx?p=1
you can output to ProPhoto RGB for example, but it is claimed to be smaller ;) than whatever C1 is using...
Not directly though I can see how that conclusion can be made. I would have thought that they talk about ERIMM (linearised ProPhoto eseentially proposed by Kodak). But if the above reading is correct then I guess they might just use LAB in some shape (normal LAB, UpLab etc) to have a "very large color space, similar to that captured by camera sensors".
-
Just released 0.9.2, which has a fix for the ICC blue-goes-black problem.
When fixing the "black hole" the problem in C1 indeed disappeared, the easiest way was to interpolate from defined neighbors. Thus my initial assumption was incorrect, I don't need to know the Capture One output color space is, fortunately.
I have now tested a complete workflow in Capture One 8 (as it was a long time ago I tested) making both linear profiles and with a tone curve using DCamProf's neutral tone reproduction operator. It all seems to work as intended.
I've updated the basic workflow docs both for the ICC and DNG workflow to show typical weighting parameters and the use of tone curves, that is making the basic workflows a bit more informative so the casual user can get more out of DCamProf.
-
Just released 0.9.2, which has a fix for the ICC blue-goes-black problem.
The downloads section still references 0.9.1
-
The downloads section still references 0.9.1
Ooops! Fixed the link. Thanks for reporting.
-
0.9.2 build for Windows (mingw = dcamprof.exe + libgomp_64-1.dll + HTML & PDF manual / = copy of Torger's web page and the same converted to PDF /) : https://app.box.com/s/xkiv79m5shimsrw7alq8cxdvvczjfh7x
-
what is the basis of being "pretty sure" though ?
for example http://help.phaseone.com/en/co6/output/learn-more-about-file-formats/colors-in-capture-one.aspx?p=1
you can output to ProPhoto RGB for example, but it is claimed to be smaller ;) than whatever C1 is using...
so do we have any P1 people on record or something of the same quality ?
I'm sorry. If I could remember I would have provided that information. If memory serves, and there is a real good chance it isn't in this case, I heard that from a C1 guy on a forum. I would have thought this one, but if no one else is sure, that probably isn't true.
-
Played some more with Capture One tonight.
I wanted to make sure that their film curve really is a pure RGB curve, and indeed you can quite easily "copy" the curve by replicating it in the RGB curve tool. As we know RGB curves has some color shift issues. However they avoid the worst shift (and over-saturation) issues by not going below y=x in the shadow range of the S, the curve is above the linear curve throughout the range. It also seems overall a bit low on saturation (at least the P45+ which I used in testing), which does keep the colors more stable when applying an RGB curve.
It's still quite clear that the profile is designed for the standard curve, which causes some undersaturation and shifts when using the linear curve. Yellow-orange-red tones are good for spotting issues.
Today DCamProf will apply the curve to the ICC LUT, that is you must always use the "linear response" curve in Capture One (don't use linear scientific as it disables highlight reconstruction). I've done some tests and I haven't so far seen any disadvantages of the approach. I made some brief experiments to reverse the curve so you could have profiles designed like C1, that is no curve in the LUT but the right colors only appear when the intended film curve is applied. I didn't get it to work though, and I'm not sure it ever will, but I may try a bit more.
The thing is that C1, just like Hassy's Phocus, seems to have embraced the "RGB curve look" to some extent. If you like DCamProf have a different tone curve model then it means the RGB curve works very much against you, meaning that you need to "pre-distort" colors to quite much to make the intended look after an RGB curve has been applied. This may cause the profile look outright bad with a linear curve. The C1 bundled profiles still look fine with a linear curve as they haven't tried to work too much against the RGB curve quirks.
That is even if I succeed making DCamProf ICCs without tonecurve embedded for C1 it will likely still be a bad idea as it's not really an RGB style profile and is thus not as flexible as the bundled ones concerning changing curve. I will most likely not spend time on making an "RGB-friendly" tone reproduction operator as I'm not into the RGB look myself.
-
Here's an example to show the differences between different type of tone reproduction. I've borrowed one of Erik's P45+ samples as it had a perfect strong but plain blue gradient sky for showing the differences more clearly.
The reference is a low contrast linear rendering, which shows the original color as is. I made it low contrast and brighter to make it more easy to compare with the S-curve renderings. Then we see how the RGB curve modulates the color, you get a bit of yellow into the sky. The reason is that blue is compressed more than the other channels that is the color gets less blue which means more yellow.
Adobe identified this problem and made a special variant of the RGB curve for Lightroom/ACR/DNG. It makes sure that RGB-HSV hue is kept constant. This way you don't get the color shift towards yellow. However if you make a gradient along RGB-HSV/HSL from blue to white it will not really look hue constant (see attached example), as it's not really a perceptually uniform space. This leads to that you typically get a little magenta feel to the sky.
DCamProf's tone curve (tone reproduction operator) uses CIECAM02 with some extra special tricks. Pure CIECAM02 will give you extreme highlight rendering issues. The camera clips so we must eventually blend into white which will be too abrupt with a too strong perceptual uniform model. With DCamProf I've tried to find a balance in the rolloff by mixing CIECAM02 with RGB-HSV. I'm thinking about adjusting the rolloff and actually make it non-uniform over tones. The skin-tone range seem to gain from a longer rolloff with more desaturation, while the sky blue range seems to gain from a shorter rolloff. For the blend-into-whitepoint-problem there's not much in color science models to find (as far as I know), so you have to come up with your own perceptual transition.
The examples here are rendered in RT but the RGB yellowish sky is also seen in Capture One's native rendering - that is Phase One uses the RGB color shift as a look feature and don't compensate for it that much, which makes their profiles work less bad with linear response than they otherwise would. Obviously if I make a DCamProf profile that "pre-distorts" the colors to end up right after applied RGB curve the colors will look bad with a linear response. Thus applying the curve directly to the ICC LUT is the right thing to do for every profiling software that doesn't embrace the RGB curve look.
-
Here's an example to show the differences between different type of tone reproduction. I've borrowed one of Erik's P45+ samples as it had a perfect strong but plain blue gradient sky for showing the differences more clearly.
The reference is a low contrast linear rendering, which shows the original color as is. I made it low contrast and brighter to make it more easy to compare with the S-curve renderings. Then we see how the RGB curve modulates the color, you get a bit of yellow into the sky. The reason is that blue is compressed more than the other channels that is the color gets less blue which means more yellow.
Hi Anders,
I'm sorry that I do not have the time to analyze in detail, so what I'm going to say may be off the mark (if so just ignore).
All RGB manipulations of brightness/contrast, should be done in linear gamma space as a minimum safeguard against unbalanced shadow/highlight changes. So not linear tone curve, but linear gamma, which means that any tone curve must first be removed/reversed then brightness/contrast adjusted for all channels, then tone-curve reapplied.
That of course still does not decouple Chrominance and Luminance, which would require conversion from RGB to e.g. HSL or HSV or XYZ.
As long as calculations are performed in floating point precision, the round-off errors from multiple colorspace conversions should not become an issue.
Cheers,
Bart
-
Not sure I follow completely, but in any case all the images above are rendered in floating point linear gamma space. I started with color stuff quite late after most things had already become linear floating point so I am quite unfamiliar with working in gamma spaces.
-
Hello,
First, thank you Torger, it is a very good job with Dcamprof, thank you also for the publication of AlterEgo with versions under Windows.
A question about to make a dcp profile, how do you do to make a “tiff“ of the target for Lr? (The ideal is to generate a profile from a DNG file created with Lightroom, no?)
-
A question about to make a dcp profile, how do you do to make a “tiff“ of the target for Lr? (The ideal is to generate a profile from a DNG file created with Lightroom, no?)
what do you mean ? Torger suggests using dcraw binary or rawtherapee to make .tif for argyll utilities (you can as well use rpp or libraw utilities too) ... open http://www.ludd.ltu.se/~torger/dcamprof.html and search for example for text beginning with "Basic workflow for making a DNG profile using a test target"
I am using rawdigger (profile edition) as it is more convenient (for me).
-
what do you mean ? Torger suggests using dcraw binary or rawtherapee to make .tif for argyll utilities (you can as well use rpp or libraw utilities too) ... open http://www.ludd.ltu.se/~torger/dcamprof.html and search for example for text beginning with "Basic workflow for making a DNG profile using a test target"
I am using rawdigger (profile edition) as it is more convenient (for me).
Yes for Dcraw or Rawtherapee, but it is better to make profil with the software (Lr for instance) that performs the "tiff" (or DNG), the color corrections are done better.
(I use Rawdigger to check clipping RGB values on the raw)
-
Yes for Dcraw or Rawtherapee, but it is better to make profil with the software (Lr for instance) that performs the "tiff" (or DNG), the color corrections are done better.
(I use Rawdigger to check clipping RGB values on the raw)
If you're asking how to make a tiff file that Argyll scanin can read, then you can't use Lightroom as it cannot export a linear tiff without white balancing.
What can be desirable though if you intend to use the finished profile in Lightroom is that you convert to DNG first, and then use RawTherapee or DCRaw to make the tiff for Argyll's scanin. Usually this is not necessary, but some raw formats contain calibration data which may be applied differently by Adobe's DNG converter and DCRaw/RawTherapee
-
I have now released v0.9.3.
I've made the neutral tone operator more flexible on the rolloff parameter so it can be specified per hue. I've seen advantages in subjective profiles to have a shorter rolloff for the skies range, and a longer rolloff for the skintones range. This maintains better sky color in landscapes, and render high key portraits with a more pleasing and nautral-looking transition to the whitepoint.
Then I've digged deeper down the Capture One converter and added some features required for making general-purpose profiles that work in the same way as their bundled.
C1's profiles are a bit messy, it seems like they apply the tone curve separately, which they do, but the also have a residual curve in the LUT. This means that "Linear Response" still means that you have some residual S-curve left. The purpose of this as far as I can tell is to minimize the RGB curve color shift. The separate curve is never a true "S" so it doesn't distort color as much, and then the LUT curve is probably added with some other technique, perhaps a Lab Lightness curve. This split approach makes the profiles work better than they otherwise would with different curves. It explains the mystery howcome C1 can produce good results despite using RGB curves as tone curves.
You can now design this type of profiles for Capture One using DCamProf, by following the workflow described in the docs. It will then get its look optimized for the default curve, but will look ok with the others, just as native profiles. Perfectionists should still design one profile per curve, as always.
What still may be left in terms of C1 support is gamut mapping. DCamProf currently doesn't do any, meaning that high saturation colors indeed get high saturation and may clip. There are ways in C1 to get them in gamut again but it seems like their approach is to have gamut compression in the profile itself. Context sensitive dynamic gamut mapping is a zillion times better of course, but the 1990s way to do it was in the profile, and it seems C1 is still there :-/.
-
I have now released v0.9.3.
it seems that you didn't update the webpage @ http://www.ludd.ltu.se/~torger/dcamprof.html with the text matching to what is in the dcamprof.html from the distribution
-
0.9.3 build for Windows (mingw = dcamprof.exe + libgomp_64-1.dll + HTML manual, no PDF this time) : https://app.box.com/s/74jijebztt5nupgypmvqzqq2i5p4i4yo
-
it seems that you didn't update the webpage @ http://www.ludd.ltu.se/~torger/dcamprof.html with the text matching to what is in the dcamprof.html from the distribution
Just looked, it matches here. Could be that the there was some cached stuff showing?
-
Just looked, it matches here. Could be that the there was some cached stuff showing?
may be - now it shows.
-
If you're asking how to make a tiff file that Argyll scanin can read, then you can't use Lightroom as it cannot export a linear tiff without white balancing.
What can be desirable though if you intend to use the finished profile in Lightroom is that you convert to DNG first, and then use RawTherapee or DCRaw to make the tiff for Argyll's scanin. Usually this is not necessary, but some raw formats contain calibration data which may be applied differently by Adobe's DNG converter and DCRaw/RawTherapee
Ok, thank you for the answer, I thought to do as you indicate.
-
What still may be left in terms of C1 support is gamut mapping.
Hmm... it's probably not gamut mapping I need, I just need to shape up the handling of those "undefined" areas of the RGB->Lab LUT, the current implementation doesn't do it good enough. It's still a difficult problem, but I have some new ideas to try. Until then the ICC profiles may have some issues with super-saturated colors.
-
I had extracted my ProBack spectras tonight and they are rubbish. The first attempt went astray so I will need to redo it. I'd need to correct a few things as well:
- I need to invent something to connect camera to integrating sphere more tightly to prevent even slightest leaks
...
Finally managed to get all the parts and get a more secure connection camera-to-integrating sphere. Did another run of taking readings for the cameras I have at 5nm steps. Results are below.
I used Kodak ProBack 645 and Kodak SLR/n. ProBack was used without IR filter (transmittance of several IR filters measured separately so usage of them can be simulated). Both cameras were used in the mode where they are taking dark shot aftre exposure ans subtract it to reduce the noise (SLR/n uses temperature weighted subtraction).
The SLR/n spectral responses are closely repeating the shape published in the sensor documentation but are more smooth. I managed to build very decent matrix profile with them in dcamprof using only CC24.
The ProBack spectral responses however did came out quite jagged at first but sampling different areas of raw file produced smoother spectra.
All the spectral data in the form of CSV files and JSON files (for dcamprof usage) could be downloaded here (https://drive.google.com/file/d/0Bw2ZohnbXtyAbEJWNEttYVBiNEk/view?usp=sharing).
Spectral responses of Kodak SLR/n:
(http://www.aletan.com/img/s1/v21/p1410111954.jpg) (http://www.aletan.com/img/s1/v21/p1410111954.jpg)
Spectral responses of Kodak ProBack 645 + Kodak IR filter:
(http://www.aletan.com/img/s3/v41/p1411755938.jpg) (http://www.aletan.com/img/s3/v41/p1411755938.jpg)
Spectral responses of Kodak ProBack 645 + KnightOptics IR filter (it allows a bit more red but within Kodak IR filter specs):
(http://www.aletan.com/img/s2/v61/p1411755885.jpg) (http://www.aletan.com/img/s2/v61/p1411755885.jpg)
-
Hi
After our exchanges, I tried to make a profile for Lr from a raw demosaiced with RT (Rawtherapee) by making a linear tif.
The dcp (or icc) profiles give very good results with RT (very low delta E 2000)
With LR, dcp profile doesn't work well, there is a shift in the temperature color display and hue for the white balance, I tested dcp linear profile and with the TRC "ACR" without better results.
The command lines used:
scanin -v - G 1.0 -dipn target.tif target.ti3
dcamprof make-profile –i D55 (1) –I D50 (2) -b P11 (3) target.ti3 profil.json
dcamprof make-dcp -n "Nikon D700" -d "Profile name" –t acr profil.json profil.dcp
(1) The pattern is taken with flash
(2) The data of the test pattern are D50
(3) P11 is the most neutral patch (a and b values of L * a * b *) of the target
If you have any comments on the process ... thank you
Someone made profiles for Lr? (With what procedure)
-
Nikon D700
dear, dear... you have SSF/CMF for the camera ( credit = https://www.nikonschool.it/experience/infrarosso-dslr4.php ) , why bother with targets ? this graph says that with IR-Cut filter (hence not removed)
(https://www.nikonschool.it/images/infrarosso-dslr/big/sensibilita-D700-con-IRcut.jpg)
and this will help you to digitize the graph http://arohatgi.info/WebPlotDigitizer/download.html
voila...
PS: or our friend Trantor created some already if you want just to use some = http://wowcamera.info/forum/viewtopic.php?t=967&start=40
-
I tried with http://www.cis.rit.edu/jwgu/research/camspec/db.php (http://www.cis.rit.edu/jwgu/research/camspec/db.php) (the link is broken today), there were the SSF data for D700.
The profile is lower than that achieved with a pattern, why? I'm not sure if it was the SSF data or the way I do, I have kept the SSF data and I would have time I will make a new test
For "Trantor's profil" thank you, I look at
-
The profile is lower than that achieved with a pattern, why?
what "lower" means ? low contrast ? low saturation ?
PS: if you have a loss of contrast when shooting a target to get a raw file to use in profile creation then it is natural for a profiling software to compensate (based on target measurements, where it sees that contrast has to be higher)
-
I tried with http://www.cis.rit.edu/jwgu/research/camspec/db.php (http://www.cis.rit.edu/jwgu/research/camspec/db.php) (the link is broken today)
I think they simply removed the stuff, I had it saved
RED:
0.00177749581440642
0.00274445751769572
0.0487655203252541
0.0556496138020373
0.0398169097399976
0.0313482706319957
0.0266321977506102
0.0281100251101365
0.0324368880182042
0.0298626577750321
0.0316728595832234
0.0383465440299556
0.0621504452454818
0.0753729257442601
0.0394324854435398
0.0219264580025719
0.0252601293128276
0.0953911590772382
0.494608494836299
0.743715011217136
0.711720027647541
0.61752326479373
0.512359892198394
0.411283156067318
0.319804683055095
0.242359166236931
0.165468210996257
0.0821160418097345
0.0262800791741362
0.00739375747881804
0.00285676663650002
0.00111688551607914
0.000471281415204309
GREEN :
0.00143354304586119
0.00187398911552759
0.0230803142653007
0.0433564348087553
0.0599136084381471
0.0823639826379627
0.133560455257778
0.244984139558533
0.359608596552274
0.409368145681599
0.576987109214872
0.766192716804414
0.900613338014063
1
0.954947098200156
0.887281275289021
0.747135409589881
0.624904015610591
0.45115151851569
0.286898667071245
0.144885824038473
0.0650367429493248
0.0319470263207801
0.0194491179982937
0.012242204930572
0.00814265442363087
0.00593576724862311
0.00422657598707906
0.00205652351596989
0.000881204570773891
0.000480143028493641
0.000237596621203383
0.000117563261068653
BLUE :
0.00591568345365685
0.0143952300672906
0.374546467413338
0.652706391660757
0.753092472546832
0.90218044480169
0.911669627000415
0.867874807032218
0.815767336811676
0.644026296824005
0.460406091187772
0.277858930228507
0.130722325247247
0.0640848940275101
0.0317483362505199
0.0150751490571673
0.00743144412683003
0.00536619979974889
0.00411640917428182
0.00265646535288544
0.00126895149591723
0.000756851904319394
0.000506425955964308
0.000332196038026167
0.000305303155847557
0.0003917628185241
0.000235394162743942
0.000188476692220806
0.0000804704363975458
0.0000457342402698008
0.0000345036366345881
0.0000279619189396018
0.0000246591433308175
-
what "lower" means ? low contrast ? low saturation ?
PS: if you have a loss of contrast when shooting a target to get a raw file to use in profile creation then it is natural for a profiling software to compensate (based on target measurements, where it sees that contrast has to be higher)
Lower* : less good, less accurate profile.
*, sorry for my english...
-
I think they simply removed the stuff, I had it saved
Me too (thank a lot)
-
I tried with http://www.cis.rit.edu/jwgu/research/camspec/db.php (http://www.cis.rit.edu/jwgu/research/camspec/db.php) (the link is broken today), there were the SSF data for D700.
The profile is lower than that achieved with a pattern, why? I'm not sure if it was the SSF data or the way I do, I have kept the SSF data and I would have time I will make a new test
For "Trantor's profil" thank you, I look at
dcp Strantor's profiles do not work well with Lr, Lr has the same behavior with the profiles of Strantor and mine.
To make a good profile, it is essential to be able to generate the linear tiff (or dng) with the soft for which the profile is made.
-
Lower* : less good, less accurate profile.
and how do you test for "accuracy" ?
-
To make a good profile, it is essential to be able to generate the linear tiff (or dng) with the soft for which the profile is made.
that again begs the question - how do you test exactly ?
here is what I get when SSF/CMF approximated (for example, not the best approximation that I made) using the matlab script from RIT (so less precise than a quality measurement with monochromator setup) and dcp profile built based on that for ACR used for a shot (raw) of an individually measurement target (passport) vs the measurements :
(http://s13.postimg.org/ih0qhg4fr/de125_1.jpg)
is it precise enough to your eye ? the test is with babelcolor patchtool :
(http://s9.postimg.org/nr68fpihb/de125_2.jpg)
no mess with "essential to be able to generate the linear tiff (or dng) with the soft for which the profile is made" in ACR case for Sony A7 (mark I).
now when I am sure that I can get that close with the SSF/CMF I approximate with ACR settings specific for such comparison (process 2010, etc) then I can, if I want, go into some creative mode with extra features that DCamProf provides (but I don't - leaving all that for postprocessing in photoshop)
-
I tried with http://www.cis.rit.edu/jwgu/research/camspec/db.php (http://www.cis.rit.edu/jwgu/research/camspec/db.php) (the link is broken today)
https://web.archive.org/web/20140107051347/http://www.cis.rit.edu/jwgu/research/camspec/camspec_database.txt
The original file is still alive in the previous location too, http://www.cis.rit.edu/~dxl5849/projects/camspec/database.txt
All the projects by Dengyu Liu, including camspec database - scroll to the bottom - http://www.cis.rit.edu/~dxl5849/projects/camspec/
-
Like you, with Patch tool and look at the gamuts and curves profiles Copra3.
The shape and the absence of breakage is also important that the volume.
I plan to go to Color Think.
-
[...]
is it precise enough to your eye ?
[...]
Yes, it is really very good result.
I made a few icc profiles with SSF data D700, with a simulation of the color digital SG , but I have a doubt about the values of this target (in memory, I took the values given on the website of Babelcolor). I kept all the files and I'll try on dcp profiles (I still work).
Thank you for your help.
-
https://web.archive.org/web/20140107051347/http://www.cis.rit.edu/jwgu/research/camspec/camspec_database.txt
The original file is still alive in the previous location too, http://www.cis.rit.edu/~dxl5849/projects/camspec/database.txt
All the projects by Dengyu Liu, including camspec database - scroll to the bottom - http://www.cis.rit.edu/~dxl5849/projects/camspec/
Thank a lot for links
-
Hi
After our exchanges, I tried to make a profile for Lr from a raw demosaiced with RT (Rawtherapee) by making a linear tif.
The dcp (or icc) profiles give very good results with RT (very low delta E 2000)
With LR, dcp profile doesn't work well, there is a shift in the temperature color display and hue for the white balance, I tested dcp linear profile and with the TRC "ACR" without better results.
The command lines used:
scanin -v - G 1.0 -dipn target.tif target.ti3
dcamprof make-profile –i D55 (1) –I D50 (2) -b P11 (3) target.ti3 profil.json
dcamprof make-dcp -n "Nikon D700" -d "Profile name" –t acr profil.json profil.dcp
(1) The pattern is taken with flash
(2) The data of the test pattern are D50
(3) P11 is the most neutral patch (a and b values of L * a * b *) of the target
If you have any comments on the process ... thank you
Someone made profiles for Lr? (With what procedure)
When you use "-t acr" you will embed a tone curve which will activate the neutral tone reproduction operator which tries to make a "perceptual" color appearance match, it's then no longer a colorimetric profile whose accuracy can be measured with standard methods.
Without tone curve it should be colorimetric though, and it sounds strange that you've got different results from RT compared to Lr. The results should be the same. In earlier versions of DCamProf no linear curve was embedded per default which lead to that Lr added a default curve, destroying the colorimetric property. However, later version should add a linear curve per default, unless you disable it (or I have broken it, it was a while since I tested that aspect), and then you should get the same result as in say RT.
You could check using dcp2json command if there is a linear curve in there, or simply look at Lr if it renders with a contrast curve or not when applying the profile.
-
https://web.archive.org/web/20140107051347/http://www.cis.rit.edu/jwgu/research/camspec/camspec_database.txt
The original file is still alive in the previous location too, http://www.cis.rit.edu/~dxl5849/projects/camspec/database.txt
All the projects by Dengyu Liu, including camspec database - scroll to the bottom - http://www.cis.rit.edu/~dxl5849/projects/camspec/
Thanks, will update the link on the web page to the next release.
-
When you use "-t acr" you will embed a tone curve which will activate the neutral tone reproduction operator which tries to make a "perceptual" color appearance match, it's then no longer a colorimetric profile whose accuracy can be measured with standard methods.
Without tone curve it should be colorimetric though, and it sounds strange that you've got different results from RT compared to Lr. The results should be the same. In earlier versions of DCamProf no linear curve was embedded per default which lead to that Lr added a default curve, destroying the colorimetric property. However, later version should add a linear curve per default, unless you disable it (or I have broken it, it was a while since I tested that aspect), and then you should get the same result as in say RT.
I tested with "t acr" and without in RT, although I found the addition of the tone curve (in fact it is possible to disable it with RT, and to see the difference).
With Lr, it is difficult to tell by looking whether there is or is not the curve, although I see difference, but it is not possible to tell the origin. With Lr, I see a shif of white balance, 5500k Normally, and Lr displays 9900K and a hue to + 150, when I make a white balance with a pipette (I do not know how to say ... :D :the measuring probe WB on the patch), the value turns to 7500K and always on the hue +150.
The profile is good with RT, not usable with LR.
I will try to redo a profile with shooting with white RGB values slightly lower. I found that LR has a management of the high lights different that RT.
I'll let you know
You could check using dcp2json command if there is a linear curve in there, or simply look at Lr if it renders with a contrast curve or not when applying the profile.
I will also try. Thank a lot
-
For over-exposed images Lr will crunch the image to look more like over-exposed film, meaning applying some desaturation overall to make a smoother transition into the blown areas. RT doesn't do that automatically.
However for normally exposed images it should be the same. RT is much smoother to work with when it comes to DCP testing so I quite rarely fire up Lr and test there (I work almost exclusively on Linux boxes too which make using Lr a bit messy), which means that I may have introduced some bug that has passed undetected. I'm a bit busy with other stuff now so I don't have the opportunity to do much Lr testing.
Last time I checked I got the same result though.
However note that the white balance in RT is not using the DCP, it's always using the built-in matrices, and is having a different WB model than the one specified in DNG. This means that you cannot compare the temp/tint numbers in RT with what is shown in Lr. If you want to set the same WB in both either use the "as shot" balance as is, or use a WB color picker and click at some well-defined white/gray area.
That you get 7500k +150 tint in Lr does sound suspicious though, I'm starting to suspect some bug in DCamProf, unless there are some problem in your workflow.
-
Thank you for the information (difference between white balance with Lr and RT)
That you get 7500k +150 tint in Lr does sound suspicious though, I'm starting to suspect some bug in DCamProf, unless there are some problem in your workflow.
Bug? May be dcamprof, but it is possible that it is my workflow?
Here is the beginning of dcp profile file decoded with dcp2json:
==>
I find that the coefficients of the color matrix are odd (?)
Is that the forward matrix (D50) is correct?
Color matrix
{
"UniqueCameraModel": "Nikon D700",
"ProfileName": "Df-5500k_dt7-6-031",
"ProfileCopyright": "Copyright, the creator of this profile",
"ProfileEmbedPolicy": "No restrictions",
"CalibrationIlluminant1": "D55",
"ColorMatrix1": [
[ 1.430500, -0.307400, -0.122100 ],
[ -0.598500, 1.364700, 0.204700 ],
[ -0.100300, 0.190100, 1.008600 ]
],
"ForwardMatrix1": [
[ 0.748500, 0.180100, 0.035600 ],
[ 0.324700, 0.810600, -0.135300 ],
[ 0.027200, -0.129500, 0.927400 ]
],
"DefaultBlackRender": "None",
"ProfileHueSatMapDims": [ 90, 30, 1 ],
"ProfileHueSatMap1": [
{ "HueDiv": 0, "SatDiv": 0, "ValDiv": 0, "HueShift": 0.000000, "SatScale": 1.000000, "ValScale": 1.000000 },
{ "HueDiv": 0, "SatDiv": 1, "ValDiv": 0, "HueShift": -3.063314, "SatScale": 0.971598, "ValScale": 0.997364 },
{ "HueDiv": 0, "SatDiv": 2, "ValDiv": 0, "HueShift": -2.713044, "SatScale": 0.983901, "ValScale": 0.996794 },
[...]
-
Bug? May be dcamprof, but it is possible that it is my workflow?
may be you can also post a sequence of commands (dcamprof command line paramereters for each call) how do you build a profile too...
-
The command lines used for a dcp profile (mono illuminant):
scanin -v - G 1.0 -dipn target.tif target.ti3
dcamprof make-profile –i D55 (1) –I D50 (2) -b P11 (3) target.ti3 profil.json
dcamprof make-dcp -n "Nikon D700" -d "Profile name" profil.json profil.dcp
(1) The target is taken with flash
(2) The data of the test pattern are in D50
(3) P11 is the most neutral patch (a and b values of L * a * b *) of the target
For an icc profile,the last line is:
dcamprof make-icc -n "Nikon D700" -p lablut (or xyzlut) profil.json profil.icc
-
With this command line :
dcamprof make-dcp -n "Nikon D700" -d "Profile name" -t linear -h 90,35,15 profil.json profil.dcp
The profile is linear, does not seem linear by "default" (I looked in the file "profile".json), but there are always a matter the white balance in LR et ACR (I verified for ACR), why, I dn't undestand (?)
I had made a dcp profile in June (dcampprof V7 or V8), I tested it and it has the same behavior.
A question that has no link: Why there are the -i and -I options with the command dcamprof make dcp?
-
oops... the linear curve should be default, that's a bug, will fix that to next release. Anyway for now you can enable it with -t linear for now. Lr will apply a default curve to the profile if the profile doesn't have any, and RT doesn't do that. So to get a well-defined result one have to embed the curve.
The make-dcp has -i and -I options to make it possible to specify the EXIF illuminants for illuminant 1 and 2. Say if you have profiled with a custom light and recorded its spectrum, there is no EXIF name for that so the native format will say "Other" on the exif tag. When you make a dual-illuminant DCP you need to have defined light sources, so then you manually pick something close from the available EXIF illuminants.
I still can't figure out why you have a white balance difference between RT and Lightroom though. Since I got your report I haven't yet been able to run a Lr test so I haven't been able to verify myself. Maybe someone else in this thread can confirm if they've had it working in Lr recently?
-
oops... the linear curve should be default, that's a bug, will fix that to next release. Anyway for now you can enable it with -t linear for now. Lr will apply a default curve to the profile if the profile doesn't have any, and RT doesn't do that. So to get a well-defined result one have to embed the curve.
The make-dcp has -i and -I options to make it possible to specify the EXIF illuminants for illuminant 1 and 2. Say if you have profiled with a custom light and recorded its spectrum, there is no EXIF name for that so the native format will say "Other" on the exif tag. When you make a dual-illuminant DCP you need to have defined light sources, so then you manually pick something close from the available EXIF illuminants.
I still can't figure out why you have a white balance difference between RT and Lightroom though. Since I got your report I haven't yet been able to run a Lr test so I haven't been able to verify myself. Maybe someone else in this thread can confirm if they've had it working in Lr recently?
Hi Torger,
Thank you for the answers.
I have the idea that the dcp profile (made with dcamprof) does not run with lr (white balance problem) because Lr (or Acr) wants to have a bi-illumimant profile, this would cause the calculation error by Lr of white balance.
I opened (with dcp2json) dcp profiles mono illuminant made with DNG profile editor, and, inside, they have this structure: 2 illuminants (StdA and D65), 2 colors matrix, 2 forwards matrix and 2 ProfileHueSatMap.
I tried to manually edit a dcp profile with a similar structure, but it does not work properly yet with Lr, I continue.
-
FYI I just ran through a workflow making a lightroom profile and it works for me, single illuminant. I get the same result in RT as in Lightroom. The actual temperature number shown in the white balance widget is different though, but that's normal as explained a few posts back.
-
I continue.
I created a simple single illuminant / linear curve dcp profile from available D700 SSF data, it works OK... beats me why you are refusing to use SSF data in D700 case to create profiles... why do you think that using a target makes it better - you really think that it, for example, will help you to account for a specific lens mounted on your camera ?
-
FYI I just ran through a workflow making a lightroom profile and it works for me, single illuminant. I get the same result in RT as in Lightroom. The actual temperature number shown in the white balance widget is different though, but that's normal as explained a few posts back.
Thank you. You refer to this:
However note that the white balance in RT is not using the DCP, it's always using the built-in matrices, and is having a different WB model than the one specified in DNG. This means that you cannot compare the temp/tint numbers in RT with what is shown in Lr. If you want to set the same WB in both either use the "as shot" balance as is, or use a WB color picker and click at some well-defined white/gray area.
That you get 7500k +150 tint in Lr does sound suspicious though, I'm starting to suspect some bug in DCamProf, unless there are some problem in your workflow.
I agree about the difference calculation between RT and LR, but but it doesn't seem normal (for me) to have a white balance to 7000k and 150 of hue (witjh color picker on neutral patch, white balance is about 7300k and always 150 of hue).
One more, with profil made with DNG adobe editor, this shift is not present.
I continued my tests this afternoon, the white balance shift is not a bi-illuminant profile issue, I found that this shift came from the color matrix, why, I don't know?
-
I created a simple single illuminant / linear curve dcp profile from available D700 SSF data, it works OK... beats me why you are refusing to use SSF data in D700 case to create profiles... why do you think that using a target makes it better - you really think that it, for example, will help you to account for a specific lens mounted on your camera ?
I created a simple single illuminant / linear curve dcp profile from available D700 SSF data, it works OK... beats me why you are refusing to use SSF data in D700 case to create profiles...
This will be the next step (profile with SSF), when I found a matter, I try to understand (and solve if possible).
why do you think that using a target makes it better -
No, with target, it is difficult to find the right target with the good "inks" (reflectance matter), without to speak about the shot...
- you really think that it, for example, will help you to account for a specific lens mounted on your camera ?
Neither, I am not at this level of detail ;)
-
I did find a suspiciously large deviation in white balance temp/tint calculation (which comes from the color matrix, the actual colors comes from the forward matrix and LUTs), the Adobe bundled profiles said ~4850, my profile said ~5500, so maybe there's some problem with the color matrix. I'll investigate that further when I get some time. It's a bit slower period now though so it may take a while.
-
I did find a suspiciously large deviation in white balance temp/tint calculation (which comes from the color matrix, the actual colors comes from the forward matrix and LUTs), the Adobe bundled profiles said ~4850, my profile said ~5500, so maybe there's some problem with the color matrix. I'll investigate that further when I get some time. It's a bit slower period now though so it may take a while.
Ok thank you.
A slight shift of the white balance is normal with a process DNG, from about 200 to 500K, because Adobe made another calculation of raw demosaicing.
What is amazing is that this does not affect RT. I think Adobe to do a special treatment, I am trying to read back the DNG specification to try to understand, but there are not all the information, including limit values for ColorMatrix and ForwardMatrix.
I continue my trials and if I found something, I tell you.
-
beats me why you are refusing to use SSF data in D700 case to create profiles...
What kind of virtual target (CC24, CC digital SG* or other) you use? (when you do profile with the SSF data from the camera)
Do you take generic or measured spectral datas to simulate virtual target?
For CC Digital SG, I dn't find the spectral datas. (It seems that Xrite did not given them)
-
Ok thank you.
A slight shift of the white balance is normal with a process DNG, from about 200 to 500K, because Adobe made another calculation of raw demosaicing.
What is amazing is that this does not affect RT. I think Adobe to do a special treatment, I am trying to read back the DNG specification to try to understand, but there are not all the information, including limit values for ColorMatrix and ForwardMatrix.
I continue my trials and if I found something, I tell you.
For historical reasons RT always use its hard-coded color matrices for the white balance calculations, never the DCP color matrix (the RT white balance model needs some re-work, but noone's had time to do it yet). Adobe uses the DCP of course. Still if you use "as shot" you should see the same result in RT as in Lr, even if the stated color/temp is different, and this is the result I get. Have you tried "As Shot" white balance? Do you get the same result in RT as in Lr, as I do? If not you may have some other issue.
Only when you have set a manual temp/tint Lr will use the color matrix "in reverse" to figure out the multipliers and then the color matrix will matter, that is a different color matrix will result in a different tint of the image.
By using json2dcp/dcp2json and a text editor you can experiment with extracting the color matrix from Adobe's bundled profile and put that into the DCamProf profile. Then you should get the exact same white balance as the Adobe profile, but the color rendition is still completely DCamProf, as that is only affected by forward matrix and LUTs.
Really old DCPs had only a color matrix (no forward matrix), and in that case the color matrix also affected the color rendition, but as soon as you have a forward matrix, the color matrix is only used for calculating white balance multpliers from temp/tint or calculating temp/tint from the raw-embedded as-shot multipliers.
-
I am new to DCAMPROF.
I just follow the instructions below, X100.DNG is from DNG converter.
dcraw -v -r 1 1 1 1 -o 0 -H 0 -T -W -g 1 1 X100.DNG
scanin -v -G 1.0 -dipn X100.tif ColorChecker.cht cc24_ref.cie
dcamprof make-profile -w all 1.5,1,8,2,1 -l 0.1,0.1 X100.ti3 profile.json
dcamprof make-dcp -n "Fujifilm Finepix X100" -d "DCAMPROF" -t linear profile.json DCAMPROF_X100.dcp
Finally,I import DCAMPROF_X100.dcp to LR, and export to tif.
I make crop to show CC24 area. Is this ok for the calibration results? White balance looks fine here.
But it looked somewhat low contrast, dark and flat. Anything I am doing wrong here??
Thanks for help.
(http://i.imgur.com/uSaejXQ.jpg)
-
I am new to DCAMPROF.
I just follow the instructions below, X100.DNG is from DNG converter.
dcraw -v -r 1 1 1 1 -o 0 -H 0 -T -W -g 1 1 X100.DNG
scanin -v -G 1.0 -dipn X100.tif ColorChecker.cht cc24_ref.cie
dcamprof make-profile -w all 1.5,1,8,2,1 -l 0.1,0.1 X100.ti3 profile.json
dcamprof make-dcp -n "Fujifilm Finepix X100" -d "DCAMPROF" -t linear profile.json DCAMPROF_X100.dcp
Finally,I import DCAMPROF_X100.dcp to LR, and export to tif.
I make crop to show CC24 area. Is this ok for the calibration results? White balance looks fine here.
But it looked somewhat low contrast, dark and flat. Anything I am doing wrong here??
Thanks for help.
(http://i.imgur.com/uSaejXQ.jpg)
By my eye it seems correct. However as you do "-t linear" you get a colorimetric profile without a curve, which indeed looks low contrast, dark and flat. A flat colorimetric profile is good when you do reproduction work (copy artwork etc), but not so nice for general purpose photography.
To get a default curve embedded change your make-dcp command to:
dcamprof make-dcp -n "Fujifilm Finepix X100" -d "DCAMPROF" -t acr profile.json DCAMPROF_X100.dcp
If you want to fine-tune to match the contrast and brightness of the bundled profile you can "steal" the curve and baseline exposure from the bundled profile, or you can design your own.
-
Here's a more detailed description of matching the curve / baseline exposure. It's from a tutorial I'm working on but haven't published yet:
Choose a tone curve and exposure offsets
When we reproduce a scene on screen or on paper the luminance is typically lower which leads to that the eye experience the image as lower contrast, a bit dull even, this even if the (mid-tone) contrast is 100% accurate. This is a normal perceptual phenomenon and the solution is as old as photography: we apply an S-shaped contrast
curve. In digital photography there's more compression of highlights than shadows as that suits the linear sensor behavior better, but the principle is the same. By compressing highlights and shadows we get increased midtone contrast so the whole image seems to have higher contrast, and we can get a better perceptual match with the original scene.
A less known side effect of applying contrast is that the appearance of colors change. In human vision contrast and color appearance is tightly connected which means that if we want to retain the original color appearance when changing global contrast we must make some adjustments to the colors. Broadly speaking higher contrast requires higher saturation, and increased saturation is a natural side-effect of a basic RGB curve which is what's been traditionally used in digital photography. However an RGB curve will overdo saturation and it will also distort color so it's far from perfect, in fact there is still today no broadly used standard curve that is reasonably
perceptually accurate. (It's actually not possible to make it 100% accurate, as that would require image-dependent local adjustments which a camera profile can't do.)
DCamProf provides its own custom curve, a "neutral tone reproduction operator" to make it possible to embed a curve without distorting color appearance, and I think this is one of the more important features. Without that it doesn't matter how accurate we make our profile, as soon as we apply a curve in the raw converter the perceptual accuracy goes down the drain. And no, applying contrast in the Lab lightness channel or HSV value channel will not cut it (although it may work well for smaller adjustments or for creative effect). As I also contribute to the RawTherapee project, there's now actually a "perceptual" curve there which is based on DCamProf's
neutral tone reproduction operator, so for RawTherapee you can provide a profile without a curve and instead apply it using the builtin curve adjustment, but as far as I know RawTherapee is unique in this aspect. For convenience you may still want to embed it in the profile though.
For a longer more detailed discussion on how tone curves affect color, see http://www.ludd.ltu.se/~torger/dcamprof.html#tone_curves tone curve section in DCamProf's reference manual.
Anyway, in order to make our profile fit for all-around photography we need to have an S-shaped tone curve to apply some contrast. So which shape to choose? With Adobe Camera Raw and the DNG reference code comes a standard curve which is used in many of Adobe's own profiles and it's also available as a built-in choice in DCamProf. It will provide a good result and if you don't have some special need I suggest using that (<tt>-t acr</tt> provided to make-dcp).
The typical reason to want some other curve is to better match the look of in-camera JPEGs. This can indeed be important. With Adobe's curve the look may become significantly brighter or darker than the in-camera JPEG, and while it's okay if you will "expose to the right" (ETTR) manually anyway (common for planned photography shot from a tripod), the camera's auto exposure is of course tuned for the in-camera JPEG curve. That is to get good results without adjustments you need the curve to match what the camera expects.
For DNG profiles there are actually three parameters for this, the curve itself, a baseline exposure offset and a black render tag. The DNG file itself can also contain a fixed baseline exposure which is added to the offset in the profile, at least if you're using Adobe's products. The exposure offsets and black render tag may or may not be
supported by your favorite raw converter, so they may not be usable. The curve is generally supported though if the converter has a decent DNG profile support.
The baseline exposure offset is sort of redundant as a curve can also include an exposure offset (it's just the average slope of it), but if the offset is large the curve gets an extreme shape which is hard to design so in that case it can be easier to separate them. The same can be said about the black render tag. That tag does not quantify an offset, it just tells the raw converter if it should make an automatic black subtraction (that is clip away darkest shadows if there is no detail there) or leave as is.
Matching the curve used by the camera
How to get the curve from your camera? Obviously we could shoot a backlit step wedge and make a precise measurement-based match that way, but there's no real need to make an exact match, we won't have the exact camera appearance anyway concerning color. We just need to make a curve that has about the same brightness and contrast as the camera so when we use the camera's auto exposure we get a good result.
To do this I think visual matching works well, and you can use RawTherapee for that.
* If possible set the camera to write JPEGs together with the raw.
* If possible set color space to sRGB, unless you know what
you're doing and can display the JPEG properly with AdobeRGB (or
whatever the other color space is).
* Shoot basic sunny outdoor scenes with both highlights and shadows.
* It should be contrasty, but avoid back light.
* Avoid extremely saurated colors as they will exaggerate
differences between profiles and make it harder to match
curves.
* Preferably shoot more than one so you have a few to test.
* If you're using a DNG workflow, convert the raw to DNG using
your chosen DNG converter (it may add an baseline exposure offset
tag).
* If you use DNG files with non-zero baseline exposure and you
intend to honor that (as Adobe Camera Raw does), make sure
you apply corresponding exposure in RawTherapee (at the time
of writing RawTherapee ignores any baseline exposure in the
DNG file).
* If you couldn't get JPEGs directly, use for example exiftool to
extract the embedded preview image from the raw. If the color space
is not sRGB you may need to attach a profile to it (usually AdobeRGB)
which can also be done with exiftool.
* Make sure you get the camera's preview, not a re-rendered
preview made by your DNG converter if you're using that.
* exiftool -b -PreviewImage -w _preview.jpg _MG_0715.CR2
* exiftool "-icc_profile<=AdobeRGB.icc" _MG_0715_preview.jpg
* Open the raw file in RawTherapee and bring up the JPEG side by
side on screen, either by launching another RawTherapee instance and
open the JPEG there or bring it up in an image viewer.
* Apply (Neutral) processing profile so you get a clean start.
* Fold out "Tone curve 1" by choosing "Custom" (which is a spline curve).
* Curve type can be any, some are more saturated than others,
try one that matches the look of the camera JPEG the best (makes
it easier to match curve, optionally use the saturation slider
to make a better match.
* You can make the panel wider to make the curve larger and
easier to fine-tune.
* Add three control points to the curve, and make an S-shape
similar to the one shown in the screen shot, it's a good starting
point.
* Look at the darkest shadows in the JPEG and see if it's likely
some black subtraction has been made. If so match as well as
you can with the "Black" slider.
* Is the image considerably darker? Adjust "Exposure compensation"
to make a better match.
* Small offsets like 0.1-0.2 stops can typcially be solved by
brightening using the curve, while above that then it's better
to have an exposure compensation.
* Fine-tune the curve to match by moving and adding handles. Try
to keep down the number of handles, the more you have the harder it
becomes to make it smooth.
* Save the curve to an .rtc file.
* The .rtc file can be used directly by DCamProf.
* Write down if you had to adjust the black slider and how much
exposure adjustment you needed if any.
It is indeed hard to make an exact match, but it's not important. This is just about getting predictable results from the camera's auto exposure. It's more important that you like the shape of the curve, that it has suitable contrast and shadow compression. I often find it desirable to keep a little more shadow detail than camera JPEGs do for example.
If you had to adjust black subtraction and/or exposure you have some choices to make. Either you make a processing profile for your favorite raw converter that contains these presets, or you include the settings in your DNG profile. What is best depends on the feature set of the raw converter (if it supports the offset DNG profile tags or
not, and how presets are managed) and what you prefer.
You can also try to reshape the curve to compensate for any black subtraction and exposure offset.
If we start with black subtraction this can be 100% mirrored with a curve, but it will then unrecoverably cut shadows. It's better to just add an extra handle for the darkest shadows and compress a bit more there. Your will undoubltly get a less contrasty look in the shadow range this way, but also more shadow detail which you may
prefer anyway.
The exposure offset can also be mirrored with the curve but again not without unrecoverable clipping, so you could instead just increase highlight compression to increase brightness overall to match.
In general I think embedding an exposure offset in the profile makes more sense than black subtraction, especially since black subtraction cannot be set as a specific number but the result will instead vary between raw converters. My recommendation is thus combine the curve with an exposure offset if it makes sense, but try to avoid black subtraction.
The total baseline exposure should rather not be negative. That is the file's baseline exposure plus the profile's baseline exposure offset should be zero or positive. If you make it negative you will force the raw converter into showing potentially clipped highlights which is not a good default.
Notes on baseline exposure
The DNG specification has two baseline exposures, one that is stored in the DNG file itself, "baseline exposure", and one in the profile "baseline exposure offset". The latter was introduced in version 1.4 of the standard, prior to that baseline exposure could only be embedded in the DNG.
This is an unfortunate situation, if you ask me it's a poor design choice made by Adobe. The baseline exposure is 100% related to the profile, as it will depend on the curve shape which number you want. Naturally Adobe's DNG converter will embed a baseline exposure tag with its DNG files that is adapted to work with Abobe's proprietary profiles, and it may not really suit your profile.
It would be much better if the DNG profile itself specified all the baseline exposure, which it can with the new baseline exposure offset tag which is stored in the profile. Unfortunately the specification says that it shouldn't override the DNG file tag, but just adds an offset to it and that is what happens in Adobe's products. This means that you still need to know what value your DNG converter will put there, and if your raw converter cares about the value. Many raw converters can do both DNG and native raw. The native raw file has no baseline exposure offset, meaning that a DNG profile may need a different baseline exposure offset when used with native raws than when used with Adobe's DNGs.
I recommend to test your favorite raw converter to see how it reacts to baseline exposure, it's not certain that it will care about the value in the DNG file. It's not unlikely that you may need one offset for Adobe's products and a different offset for others.
-
than when used with Adobe's DNGs.
shall be added "or with Adobe converters"... they (ACR/LR) have that component also hardcoded in their code.
-
Usual way to get the curve is to take a series of of bracketed shots of some more or less uniformly lit smooth surface (a computer screen out of focus is a good target), and the curve can be constructed easily from that.
-
For historical reasons RT always use its hard-coded color matrices for the white balance calculations, never the DCP color matrix (the RT white balance model needs some re-work, but noone's had time to do it yet).
For RT, I think, since version 3.xx, white balance is calculated on a range from 4000k to 25000k, § White balance http://50.87.144.65/~rt/w/index.php?title=Color_Management_addon (http://50.87.144.65/~rt/w/index.php?title=Color_Management_addon)
Adobe uses the DCP of course. Still if you use "as shot" you should see the same result in RT as in Lr, even if the stated color/temp is different, and this is the result I get. Have you tried "As Shot" white balance? Do you get the same result in RT as in Lr, as I do? If not you may have some other issue.
Yes, with "as chot" the display of white balance with RT(any dcp (or icc) profil ) and Lr (profil Adobe Std) is the same (delta 30k, it is negligible) , with the profil made with Dcamprof the value of white balance change (~7000k et hue 150).
Only when you have set a manual temp/tint Lr will use the color matrix "in reverse" to figure out the multipliers and then the color matrix will matter, that is a different color matrix will result in a different tint of the image.
I undestand the same thing as you, except that :
-With a "normal ColorMatrix" dcp profil with Lr, it is ok
-with profil made dcamprof, the white balance is done, but hue stay saturated.
By using json2dcp/dcp2json and a text editor you can experiment with extracting the color matrix from Adobe's bundled profile and put that into the DCamProf profile. Then you should get the exact same white balance as the Adobe profile, but the color rendition is still completely DCamProf, as that is only affected by forward matrix and LUTs.
Really old DCPs had only a color matrix (no forward matrix), and in that case the color matrix also affected the color rendition, but as soon as you have a forward matrix, the color matrix is only used for calculating white balance multpliers from temp/tint or calculating temp/tint from the raw-embedded as-shot multipliers.
yes I am agree. I do a profil dcp with DNG profil editor with a cc24 shot with the same light (flash). I swap the ColorMatrix in the profil made with Dcamprof, now it is ok, white balance and hue run normally.
Below you will find the matrix.
ColorMatrix Dcamprof that dn't run
"ColorMatrix 1": [
[ 1.455000, -0.315800, -0.121900 ],
[ -0.607600, 1.379900, 0.208200 ],
[ -0.135800, 0.218600, 1.022600 ]
ColorMatrix DNG adobe editor (D65)
"ColorMatrix1": [
[ 0.813900, -0.217100, -0.066400 ],
[ -0.874800, 1.654100, 0.229600 ],
[ -0.192400, 0.200800, 0.809300 ]
I dn't know, if this matrices can help you to understand the matter (?). I searched without finding, may be, there are an overflow with the matrix calculated with Dcamprof?
-
Usual way to get the curve is to take a series of of bracketed shots of some more or less uniformly lit smooth surface (a computer screen out of focus is a good target), and the curve can be constructed easily from that.
Interesting! I'll try that technique.
-
Thanks for testing Bip, I'll do some debugging of the color matrix calculator in DCamProf when I get time and opportunity, hopefully soon.
As the color matrix is not used at all when rendering ICC profiles, or when using (single-illuminant) DCPs in RawTherapee and I'm not a Lightroom/ACR user I haven't given it much test so far.
-
Thanks for testing Bip, I'll do some debugging of the color matrix calculator in DCamProf when I get time and opportunity, hopefully soon.
As the color matrix is not used at all when rendering ICC profiles, or when using (single-illuminant) DCPs in RawTherapee and I'm not a Lightroom/ACR user I haven't given it much test so far.
May be, it is not a bug of Dcamprof, but an incompatibility with Lr / Acr.
Thank you for your help to find a solution.
-
May be, it is not a bug of Dcamprof, but an incompatibility with Lr / Acr.
Thank you for your help to find a solution.
I've done a few experiments, not running on Lr yet though as I have that on a different box in a different house. Anyway I haven't so far discovered any error.
In any case even if there is an error and I fix it, the color matrix result can differ quite much. Although the color matrix is white-point preserving (that is no compromise is made to match the white-point, it's the only color that is matched exactly), the raw RGB values for white will differ depending on the exact shape of the illuminant spectrum. (The actual numbers between two matrices can differ quite a lot too, even if they would map white to the exact same number, as the matrices map other colors too and can do that very much different.)
In any case as we cannot get the exact setup Adobe has used, DCamProf will come to a different conclusion, and thus estimate temp/tint a bit differently. This means that in Lr/ACR for all other white balances than "as shot" you will get a white balance shift. If you don't want that to happen there's an easy fix though, just copy Adobe's color matrix and put it into DCamProf's json profile, and re-run make-dcp.
What surprised me and made me suspect a bug is that here is quite large differences in temperature estimation, like in my example 4800K for Adobe and 5500K for DCamProf. I would have guessed the temperature difference should be smaller, so it's that point I'm investigating. But as said, you can't get a 100% match, if there is a bug I would still expect that at least +/-200K would differ in the normal case. So if you don't want your new custom profile to affect the white balance of your old adjusted photos when you change profile, you need to do the copy trick.
-
An example for a 5D mark II;
For a daylight shot:
As shot temperature, according to ACR with bundled profile: 4803K (tint 1)
As shot temperature, according to DCamProf profile generated with D50 and SSF: 5762K (tint 20)
As shot temperature, according to DCamProf profile generated with Solux ~5500K and test targets: 5123K (tint 6)
Canon themselves call the "as shot" temperature "5200K", and the white balance multipliers are 2.14,1.00,1.68
Looking at DxOmark measurements, the D50 white balance is stated as 2.57,1.00,1.44
DCamProf D50 white balance based on SSFs is 1.93,1.00,1.78
DCamProf D50 Solux/targets: 2.12,1.00,1.72
Adobe bundled profile D50: 2.22,1,1.68
The values are all over the map. DxOMark measurement indicates that the as shot daylight settings is a lower temperature than D50. The SSFs I've got leads me to much different result (the quality of the SSF is unclear though), and my DCamProf profile made with traditional methods is in most agreement with Canon's own labeled temperature.
So far I have not found any error in the calculation process, and if it where I would suspect gross color errors as the forward matrix is calculated the same way as the color matrix.
Could it be that all those differences are related to different calibration illuminants?
-
The numbers of CCT and Tint strongly depend on the light spectrum, and the way the matrices are calculated; _never_ I saw a CCT in a converter in good agreement with direct measurements, or good inter-converter agreement for that matter. CCT and Tint are convolutions, and as it is with the ordinary maps, they may be relatively accurate (direct measurements of distances/angles) for a close proximity, but not so for a larger vicinity.
WB coefficients can be easily close for very different light, but at least they are directly transportable between converters (that is, if a converter allows such input). That is why we don not use CCT in RPP, in spite of all user demands. Why chase something something that does not make any sense?
-
The numbers of CCT and Tint strongly depend on the light spectrum, and the way the matrices are calculated; _never_ I saw a CCT in a converter in good agreement with direct measurements, or good inter-converter agreement for that matter. CCT and Tint are convolutions, and as it is with the ordinary maps, they may be relatively accurate (direct measurements of distances/angles) for a close proximity, but not so for a larger vicinity.
WB coefficients can be easily close for very different light, but at least they are directly transportable between converters (that is, if a converter allows such input). That is why we don not use CCT in RPP, in spite of all user demands. Why chase something something that does not make any sense?
Sounds reasonable. I've noticed this issue with temperatures before, but I haven't really known exactly how unreliable it is.
The problem with Lr/ACR is that if you have edited a file with a custom white balance, they store the temp and tint number, not the white balance multipliers. So when you change profile, it probes the profile with the stored temp/tint to find out which white balance multipliers that lead up to that temp/tint for that profile, which as shown before lead to entirely different multipliers and you get a white balance shift.
Seems to me that this is a design problem with DCP/ACR which Lightroom/ACR users just have to live with. I think I'll document the work-around (copy color matrix from existing profiles) in the documentation.
-
> The problem with Lr/ACR is that if you have edited a file with a custom white balance, they store the temp and tint number, not the white balance multipliers. So when you change profile, it probes the profile with the stored temp/tint to find out which white balance multipliers that lead up to that temp/tint for that profile, which as shown before lead to entirely different multipliers and you get a white balance shift.
Yes, that's how it is.
A plugin to Lr that adds WB coeffs to XMP files and reads those coeffs back is possible.
-
I think I'll document the work-around (copy color matrix from existing profiles) in the documentation.
\
may be add a command line parameter to dcamprof itself to automate that
-
I've just released 0.9.4
I did not find any bug in the color matrix calculation so I assume that the fairly large differences we see depends on other natural factors. So no changes regarding that.
As suggested by AlterEgo I did add a new parameter, -m, to make-dcp so you can automatically copy color matrix/matrices from an old profile if you want to keep the exact same color temp estimation.
The big work is on ICC profile LUT generation which has been reworked to avoid high saturation color artifacts mainly in Capture One. Although the algorithm is really slow and not so elegant in terms of implementation, it does seem to work just fine. So to the best of my knowledge DCamProf is now mature enough to make really high end Capture One ICC profiles that work in the same way as the native profiles.
Get it at the usual place.
-
0.9.4 build for Windows (mingw = dcamprof.exe + libgomp_64-1.dll + HTML & PDF manual / = copy of Torger's web page and the same converted to PDF /) : https://app.box.com/s/hxy4q0rzi59jhxdv4aqrefl5gd1p8xu1
-
I've just released 0.9.4
I did not find any bug in the color matrix calculation so I assume that the fairly large differences we see depends on other natural factors. So no changes regarding that.
As suggested by AlterEgo I did add a new parameter, -m, to make-dcp so you can automatically copy color matrix/matrices from an old profile if you want to keep the exact same color temp estimation.
The big work is on ICC profile LUT generation which has been reworked to avoid high saturation color artifacts mainly in Capture One. Although the algorithm is really slow and not so elegant in terms of implementation, it does seem to work just fine. So to the best of my knowledge DCamProf is now mature enough to make really high end Capture One ICC profiles that work in the same way as the native profiles.
Get it at the usual place.
Thank you, I downloaded the 9.4
-
@ AlterEgo
Can you say me :
What kind of virtual target (CC24, CC digital SG* or other) you use? when you do profile with the SSF data from the camera.
Do you take generic or measured spectral datas to simulate virtual target?
For CC Digital SG, I dn't find the spectral datas. (It seems that Xrite did not given them)
-
For CC Digital SG, I dn't find the spectral datas. (It seems that Xrite did not given them)
GM data from PM attached
-
What kind of virtual target (CC24, CC digital SG* or other) you use?
for a start you can use CC24 data embedded into DCamProf itself... I test on that small virtual target initially.
-
GM data from PM attached
Thank you.
for a start you can use CC24 data embedded into DCamProf itself... I test on that small virtual target initially.
yes, to begin, it is alright.
-
yes, to begin, it is alright.
but then it is totally up to you... I might be happy even with adobergb-grid target for my purposes.
-
but then it is totally up to you... I might be happy even with adobergb-grid target for my purposes.
yes, a profile with adobe RGB is already well. I will return my ideas when I have a few profiles with SSF method.
-
Do you know why the gamut of icc profiles (for APN) defined in the XYZ space are larger ones defined in space CIE Lab?
I found that with several profilers without understanding even if the equations give the possibility to move from one space to another.
The difference between the 2 spaces being that CIE Lab is perceptual intent rendering.
-
Do you know why the gamut of icc profiles (for APN) defined in the XYZ space are larger ones defined in space CIE Lab?
I found that with several profilers without understanding even if the equations give the possibility to move from one space to another.
The difference between the 2 spaces being that CIE Lab is perceptual intent rendering.
The ICC gamut for a camera does not really say much about what real colors the camera actually can capture. A camera is an input device, and you can't see the gamut the same way as for an output device such as a printer or a display.
A ICC Lab LUT will take raw RGB inputs and convert to Lab output, that is 3 values in 3 values out -- a 3D LUT. The LUT needs to cover all combinations of input values (with a limited resolution). As a camera have overlapping color filters it's impossible to get an input signal with say 100% green an 0% red and blue, but that still needs to have some output value in the LUT.
Actually this has been a very hard problem for when designing DCamProf, say 30% of all LUT values are invalid colors and since the calibration makes a model based on real colors and interpolate the rest, the impossible input value combinations get thrown out to crazy positions way outside the human locus, negative lightness and such things. First I just clipped those values, but in practice you then get an "unstable" ICC profile which can lead to strange color effects when you apply an extreme white balance (which may make some of those "impossible" input combinations arise). So I had to fake the impossible values by a special type of interpolation that makes sure that you get sane color output also for insane inputs. It seems like most other profile designers does the same thing in one way or another.
Try pulling the white balance slider to the lowest temperature possible (that will make the whole image blue) and see how the ICC profile behaves. If you have a profile that haven't cared to interpolate outputs the "impossible" inputs, you will probably see colors get clipped to black and possibly get thrown into totally different colors like green, magenta or yellow rather than staying blue.
A LUT can also have XYZ output rather than Lab, but it's not as common so I assume when you refer to XYZ you think about matrix profiles. In this case there's just a simple matrix multiplication that makes the colors, and the matrix has been optimized to produce sane colors in the range it was profiled for, typically a CC24 or similar. It means that for extreme input values you will get crazy output values. The corners of the "gamut triangle" you see in an ICC viewer are 100% green, 100% red and 100% blue, and you can't really reach those points with real inputs, and even if you could the matrix will map those to crazy positions for sure as it has been optimized to make normal colors correct, and that will cause highly saturated colors to get to bad positions.
So in other words, the gamut you see in a camera profile doesn't say anything about the camera's ability to capture colors.
With DCamProf I made a color separation analysis function that can give you some insights into how well the camera can separate colors: http://www.ludd.ltu.se/~torger/dcamprof.html#ssf_csep
It's common to say that "a camera doesn't have a gamut", and sure it does not have a gamut in the same well-defined way as a printer+paper, but it does have limitations concerning color separation. There is no standardized way to define those limits though, and unfortunately you can't get any of that information from the ICC profile.
Of course the ICC profile won't output values outside it's gamut, so you know what values you can't get, but you cannot see which values that represent those "impossible" combinations and thus are meaningless parts of the gamut.
-
The ICC gamut for a camera does not really say much about what real colors the camera actually can capture. A camera is an input device, and you can't see the gamut the same way as for an output device such as a printer or a display.
[...]
Torger, thank you for this detailed answer. I did not think asking this question the answer would lead you to give as much precision on the design profiler, but I understand the ideas you give. It is very interesting and there is not these details in books.
When I spoke gamut size (L * a * b / XYZ), I thought of the -p option of make-icc and selects Lablut, XYZlut or matrix. With Copra3 (similar to Colorthink, 3D / 2D viewer), when I look at the size of gamut, I noticed that the volume (3D) or surface (2D) colors (gamut) is more important with a XYZlut profile than Lablut profile.
I also find that the matrix profiles are a little less accurate for correcting delta E 2000, but gives made more smooth (more natural colors) on the pictures.
-
Torger, thank you for this detailed answer. I did not think asking this question the answer would lead you to give as much precision on the design profiler, but I understand the ideas you give. It is very interesting and there is not these details in books.
When I spoke gamut size (L * a * b / XYZ), I thought of the -p option of make-icc and selects Lablut, XYZlut or matrix. With Copra3 (similar to Colorthink, 3D / 2D viewer), when I look at the size of gamut, I noticed that the volume (3D) or surface (2D) colors (gamut) is more important with a XYZlut profile than Lablut profile.
I also find that the matrix profiles are a little less accurate for correcting delta E 2000, but gives made more smooth (more natural colors) on the pictures.
When the ICC LUT for XYZ and Lab is generated in the same way in DCamProf, but I think the XYZ has a bit larger range before clipping which may be the reason the gamut looks larger. It's the Lab LUT I've concentrated my testing on and testing that it handles strange white balance settings etc, while the XYZ LUT I have just briefly tested that it renders the correct colors in normal conditions. I would not be surprised if the Lab LUT has better behavior for extreme white balance settings than the XYZ LUT, I haven't tested that though.
A matrix-only profile can often be quite accurate for a CC24, but will not be able to match high saturation colors so well. Concerning LUTs DCamProf will per default try to match colors as good as possible (with the natural limits of a 2.5D LUT), and this often leads to a less than smooth profile. To get a smooth LUT profile you generally need to relax it a bit, I've added a few recipes in the basic workflows in the docs.
If you look at a consumer profiler like Adobe's DNG profile editor the LUT it generates doesn't correct lightness at all (-l -1,0 parameter with DCamProf). I think they do it because it's quite difficult to shoot a target without glare and uneven light issues, so correcting lightness for a typical CC24 target shot introduces sharp bends in the LUT without adding much accuracy. It seems like many bundled commercial profiles don't correct lightness either. I'm not 100% sure but I think Hasselblad has only chromaticity correction (hue+saturation) on their bundled profiles, and when looking at the bundled P45+ profile in Capture One it seems to be the same there. Adobe's bundled profiles has some lightness corrections though.
I have not myself really decided if skipping lightness correction in full is a good idea or not for "high end" profiles, but the less sure you are that your CC24 shot is of high quality the more you should consider smoothing lightness or even disabling it all-together.
I suspect that lightness corrections are the most likely to cause problems with gradients, while hue/saturation corrections are less likely to cause problems but I haven't made any deeper research on that yet.
When I make my own profiles I do correct lightness, but I work quite thoroughly with smoothing, looking at plots etc to fix bends where I find them.
-
yes, a profile with adobe RGB is already well. I will return my ideas when I have a few profiles with SSF method.
Depends, here are some screenshots of a plot of Adobe RGB vs. colours used in household, measured with a spectroradiometer
-
just in case it might be of interest/use for somebody or simply for collection ...
I measured my copy (one copy, not multiple samples, sorry) of DataColor SpyderCheckr 24 with i1Pro2 (one device used) using BabelColor, both in XRGA and non XRGA mode, 5 spectral measurements each time... the attached .zip has 2 x 5 and averaged measurements.
this target =
(http://spyder.datacolor.com/wp-content/gallery/spyder-checkr-24/140626_spydercheckr24_chart_front.jpg)
-
The ICC gamut for a camera does not really say much about what real colors the camera actually can capture. A camera is an input device, and you can't see the gamut the same way as for an output device such as a printer or a display.
A ICC Lab LUT will take raw RGB inputs and convert to Lab output, that is 3 values in 3 values out -- a 3D LUT. The LUT needs to cover all combinations of input values (with a limited resolution). As a camera have overlapping color filters it's impossible to get an input signal with say 100% green an 0% red and blue, but that still needs to have some output value in the LUT.
Actually this has been a very hard problem for when designing DCamProf, say 30% of all LUT values are invalid colors and since the calibration makes a model based on real colors and interpolate the rest, the impossible input value combinations get thrown out to crazy positions way outside the human locus, negative lightness and such things. First I just clipped those values, but in practice you then get an "unstable" ICC profile which can lead to strange color effects when you apply an extreme white balance (which may make some of those "impossible" input combinations arise). So I had to fake the impossible values by a special type of interpolation that makes sure that you get sane color output also for insane inputs. It seems like most other profile designers does the same thing in one way or another.
Try pulling the white balance slider to the lowest temperature possible (that will make the whole image blue) and see how the ICC profile behaves. If you have a profile that haven't cared to interpolate outputs the "impossible" inputs, you will probably see colors get clipped to black and possibly get thrown into totally different colors like green, magenta or yellow rather than staying blue.
A LUT can also have XYZ output rather than Lab, but it's not as common so I assume when you refer to XYZ you think about matrix profiles. In this case there's just a simple matrix multiplication that makes the colors, and the matrix has been optimized to produce sane colors in the range it was profiled for, typically a CC24 or similar. It means that for extreme input values you will get crazy output values. The corners of the "gamut triangle" you see in an ICC viewer are 100% green, 100% red and 100% blue, and you can't really reach those points with real inputs, and even if you could the matrix will map those to crazy positions for sure as it has been optimized to make normal colors correct, and that will cause highly saturated colors to get to bad positions.
So in other words, the gamut you see in a camera profile doesn't say anything about the camera's ability to capture colors.
With DCamProf I made a color separation analysis function that can give you some insights into how well the camera can separate colors: http://www.ludd.ltu.se/~torger/dcamprof.html#ssf_csep
It's common to say that "a camera doesn't have a gamut", and sure it does not have a gamut in the same well-defined way as a printer+paper, but it does have limitations concerning color separation. There is no standardized way to define those limits though, and unfortunately you can't get any of that information from the ICC profile.
Of course the ICC profile won't output values outside it's gamut, so you know what values you can't get, but you cannot see which values that represent those "impossible" combinations and thus are meaningless parts of the gamut.
Thanks a lot for these informations.
I made 2 dcp and icc profiles from the ssf D700 and spectral datas of CC24.
With the ICC profile, the gamut is good, in 2D, the behavior seems correct profile for the high and low lights, the 3D shape is regular.
With Lr, I watched the behavior of these profiles with pictures, visually they are as good as the profile made last week with a semi gloss target with 570 patches without the flare and glare with the shooting. All this remains to be confirmed with measurements (Delta E). (The white balance is ok)
I will continue with CCDigital SG.
-
Depends, here are some screenshots of a plot of Adobe RGB vs. colours used in household, measured with a spectroradiometer
I see some colors outside the gamut Adobe RGB (mesh), is that right must be understood?
-
just in case it might be of interest/use for somebody or simply for collection ...
I measured my copy (one copy, not multiple samples, sorry) of DataColor SpyderCheckr 24 with i1Pro2 (one device used) using BabelColor, both in XRGA and non XRGA mode, 5 spectral measurements each time... the attached .zip has 2 x 5 and averaged measurements.
this target =
(http://spyder.datacolor.com/wp-content/gallery/spyder-checkr-24/140626_spydercheckr24_chart_front.jpg)
Is there a difference between the target CC24 Xrite and Datacolor?
Is what he has one of the best?
(I have not had time to look at the spectral data)
==> I confirm that made a profile with ssf method is easier (if they have the ssf of camera)
-
Is there a difference between the target CC24 Xrite and Datacolor?
some comparison was done time ago = http://www.rmimaging.com/information/SpyderCheckr_Technical_Report.pdf
-
I see some colors outside the gamut Adobe RGB (mesh), is that right must be understood?
If I understood your question correctly, yes, some colours fall off the AdobeRGB gamut.
-
Hey guys, I am fascinated by this discussion (and pls forgive the crosspost) but I am over my head in trying to get this to compile correctly on OSX. At AlterEgo's suggestion, I'm adding a reply here with my request from this post:
http://forum.luminous-landscape.com/index.php?topic=103561
Would any of you who are smarter than me using Argyll and DCamProf on a Mac be willing to help me out with it, or point me in the right direction? I'm not at all scared of command-line utilities but I feel like I have no idea how to even get these to compile correctly. I was able to get Argyll to run the scanin command based on the instructions I found here (and after moving the chart reference files to the bin folder):
http://www.trumpetpower.com/photos/Exposure#Normalizing_exposure
But that's about it. DCamProf's instructions say "It should also be relatively easy to build on Mac OS X". I have no idea what to do with that information :/ All I want to do is generate a profile that's roughly as accurate as what the X-rite software can create for use in ACR but one that I can use in Capture One instead (so, ICC right?). I don't need to build a custom target or anything crazy (yet) just the most basic of camera calibrations.
Help? Please?
Again, any help would be HUGELY appreciated, this looks like an awesome and sorely needed tool but I'm not savvy enough to get it running just yet. I suspect I'm not the only one who want's to get their hands on this but can't figure it out.
-
OS X executable:
http://s3.amazonaws.com/IliahBorg/dcamprof.0.9.4.zip
BTW, in profio.c / dto_u1fixed15 line 1144
if (frac > 0x7FFF) frac = 0x7FFF
This may cause an error.
Maybe to compare with zero?
-
I've just released 0.9.5, very minor updates.
Included the fix of the suspicious overflow comparison noted by Iliah. The encoding/decoding of ICC values can be written much more elegant than my quick hack code, may replace it with LCMS2 at some point as was debated early in this thread :).
Anyway the bigger news is that I've completed a tutorial for making a camera profile using DCamProf, it's a separate document found at:
http://www.ludd.ltu.se/~torger/photography/camera-profiling.html
I plan to maintain that like my other photography articles and have it separate from the DCamProf distributition itself.
DCamProf is closing version 1.0. I don't have any pressing features left to implement. The goal was to make a software that would enable me and others to make profiles for general-purpose photography that can work in the same way as bundled profiles (that is work with embedded curves, have some subjective adjustments) in the large well-known raw converters, and I think I've reached that goal now.
-
0.9.5 build for Windows (mingw = dcamprof.exe + libgomp_64-1.dll + both manual AND tutorial / = copies of Torger's web pages and the same converted to PDF /) : https://app.box.com/s/l1ci6j8ybh9t438fnkhlbqwdr9ooxnaq (https://app.box.com/s/l1ci6j8ybh9t438fnkhlbqwdr9ooxnaq)
-
OS X executable
http://s3.amazonaws.com/IliahBorg/dcamprof.0.9.5.zip
-
some comparison was done time ago = http://www.rmimaging.com/information/SpyderCheckr_Technical_Report.pdf
I looked at, but it is difficult to do an opinion. I downloaded the spectral datas and, a day, I will try to do a profile.
-
If I understood your question correctly [...],
Yes, thank you
-
I looked at, but it is difficult to do an opinion.
I am for sure way less qualified than Robyn Myers, I just shared my measurements for posterity...
-
Versions follow and I don't follow ... :D
Thank you for everything, and for this new guide "Making a camera profile with DCamProf". I have a good read.
I did the dcp profile with the CC digital SG from the ssf datas of the D700, it works great on images, I have the Delta E measures to do when I have the time.
-
Anders,
How would you design the "make close-to-gray more gray" look ? Something like ScaleChroma < 1 applied to [0,10] chroma range ?
In your experience, is the W Faust C1 IT8.7 saturated enough to use alongside a CC24, or a target printed on glossy paper would be needed ?
Thanks.
-
How would you design the "make close-to-gray more gray" look ? Something like ScaleChroma < 1 applied to [0,10] chroma range ?
In your experience, is the W Faust C1 IT8.7 saturated enough to use alongside a CC24, or a target printed on glossy paper would be needed?
I think the C1 IT8 is glossy enough if I remember correctly, so it's probably a good complement. You can think about that later though, the difference between a CC24 profile only and one strengthened with glossy colors is not big.
Your recipe for making close-to-gray more gray seems fine to me. It takes some trial-and-error to nail down the range and value of course. You'd want to fade out the effect towards higher saturation, using a roundedstep curve is good for that.
(As reported in another thread there's currently a bug in the ICC LUT generator that makes darkest shadows too light and posterized. I'm working on a fix for that.)
-
In this older message:
http://forum.luminous-landscape.com/index.php?topic=100015.msg820035#msg820035
you can see a u'v' chromaticity diagram with IT8 patches plotted compared to the pointer's gamut and adobeRGB. As seen the colors are pretty saturated.
-
Just released v0.9.6, fixed a critical bug in ICC LUT generation that caused bad shadows. It doesn't affect DNG profiles, but ICC profiles which uses the neutral tone reproduction operator.
-
Great, thanks.
We'll get the IT8.7 this week and shoot all the targets at the same time with the Aptus II-7, IQ260 and A7rII.
-
May I suggest changing line 266 in colmath.c to a simple "else"
OS X executable:
http://s3.amazonaws.com/IliahBorg/dcamprof.0.9.6.zip
-
0.9.6 build for Windows ( mingw = dcamprof.exe + libgomp_64-1.dll + both manual & tutorial / = copies of Torger's web pages / in 3 formats : IE archive .mht, Mozilla archive .maff and regular .pdf ) : https://app.box.com/s/nhqbo5znht2htv0fuwcrh4dxgjdtziby
-
Yet another release, 0.9.7, yet another fix for the ICC LUT. Hopefully I got it right this time, but I can't promise it. This is one of those test for 30 seconds then release... not sure how much time I have next week to code so I just wanted to get something out before I go to bed.
http://www.ludd.ltu.se/~torger/files/dcamprof-0.9.7.tar.bz2
-
0.9.7 build for Windows ( mingw = dcamprof.exe + libgomp_64-1.dll + both manual & tutorial / = copies of Torger's web pages / in 3 formats : IE archive .mht, Mozilla archive .maff and regular .pdf ) : https://app.box.com/s/1yjsgml85xr1na3m7olkeqsncgsyb2h5 (https://app.box.com/s/1yjsgml85xr1na3m7olkeqsncgsyb2h5)
-
version 0.9.7 statically compiled for MacOSX
https://app.box.com/s/mpf11cs0clolsyb87hjuzzzrjx0e5qv1
-
I'll probably code a bit less on DCamProf for a period.
The next area to look into I think may be the rendering of very high saturation colors, found in for example flowers. I've noted that it's quite typical to get issues with clipping, the blue range often seems to be the most hurt. In theory the raw converter's built-in gamut mapping would handle this, but few raw converters have any gamut mapping at all, and if they have it it's much to simplistic to handle this type of issues well. Commercial bundled profiles generally have gamut mapping built into the profile itself.
Rendering say a deep purple flower with many shades is difficult. If we want the chromaticity to be as accurate as possible we need to pull it towards gamut clipping, and then we lose tonality. An alternative is to desaturate and lighten to "artificially" add tones which otherwise would just clip.
I personally don't shoot these type of subjects much so I need to get to the florist and make some shots so I can investigate this area more.
-
I'll probably code a bit less on DCamProf for a period.
The next area to look into I think may be the rendering of very high saturation colors, found in for example flowers. I've noted that it's quite typical to get issues with clipping, the blue range often seems to be the most hurt.
Hi Anders,
Yes, although the clipping will often occur after White Balancing, while the Raw data is not clipped. Blue can result in underexposure if some flower Reds and Yellows are very pure, and those Reds and can be pushed into clipping depending on the illuminant's color temperature. RawTherapee allows to scale the yet undemosaiced linear gamma Raw data to prevent such issues, but if instead the profile is supposed to adjust for that, then things will get difficult pretty fast.
In theory the raw converter's built-in gamut mapping would handle this, but few raw converters have any gamut mapping at all, and if they have it it's much to simplistic to handle this type of issues well. Commercial bundled profiles generally have gamut mapping built into the profile itself.
Not optimal, but such is life ...
I personally don't shoot these type of subjects much so I need to get to the florist and make some shots so I can investigate this area more.
I have a nice Yellow/Red Raw example for you, which is just 1/3rd stop below saturation clipping of the Raw data and with extremely deep blue channel data bordering on underexposure clipping. I also have some others, all very saturated flowers. I'll PM you a link, if you can use it exclusively for testing, just let me know. I do understand that you'd want to visually compare the original with its rendered version, but I can't arrange that.
Cheers,
Bart
-
Hi Anders,
Yes, although the clipping will often occur after White Balancing, while the Raw data is not clipped. Blue can result in underexposure if some flower Reds and Yellows are very pure, and those Reds and can be pushed into clipping depending on the illuminant's color temperature. RawTherapee allows to scale the yet undemosaiced linear gamma Raw data to prevent such issues, but if instead the profile is supposed to adjust for that, then things will get difficult pretty fast.
Not optimal, but such is life ...
I have a nice Yellow/Red Raw example for you, which is just 1/3rd stop below saturation clipping of the Raw data and with extremely deep blue channel data bordering on underexposure clipping. I also have some others, all very saturated flowers. I'll PM you a link, if you can use it exclusively for testing, just let me know. I do understand that you'd want to visually compare the original with its rendered version, but I can't arrange that.
You mean you won't send me flowers? ;D
I don't think I need to compare to the originals in the first stage, I just need to get a sense on how the profile behaves compared to others. After that investigation I'll decide if I need to do more work in the area or not. So please do send me a PM with the link.
-
You mean you won't send me flowers? ;D
does it make any sense to get a petal, make some (many) readings with spectrophotometer (I'd assume putting it on a black non reflective background underneath) and then make a shot to produce a raw ? that will eliminate he says/she says about the color.
-
does it make any sense to get a petal, make some (many) readings with spectrophotometer (I'd assume putting it on a black non reflective background underneath) and then make a shot to produce a raw ? that will eliminate he says/she says about the color.
There's FReD, Floral Reflectance Database too: http://www.reflectance.co.uk/ where you can get spectra for many flowers. In this case I will be more interested in the subjective qualities then the objective match though. I'd like to investigate if there is value in implementing some sort of gamut mapping feature or not.
-
There's FReD, Floral Reflectance Database too: http://www.reflectance.co.uk/ where you can get spectra for many flowers.
but not the associated raw files with the actual shots of the full petal which is /IMHO/ might better for subjective (with your eyes and converter with profile being tested) evaluation than using that database + spectral data for the camera sensor & illumination...
-
but not the associated raw files with the actual shots of the full petal which is /IMHO/ might better for subjective (with your eyes and converter with profile being tested) evaluation than using that database + spectral data for the camera sensor & illumination...
I've just sent a couple of file links to Anders, one of which is a WhiBal shot to at least have 'some idea' (although natural light changes between shots) of what neutral is supposed to look like. The angle of the target (as do leafs) will pick up more, or less, sky and ambient reflections. Flower petals, especially in nature are hard to get 'correct' due to light being filtered by overhead foliage, reflections from the soil and other leafy material, some transparency, and semi-specular reflection of the sky that varies by surface angle of the petals. So a large part will be about subjective rendering anyway. Besides, the colors can also not be displayed without display/print gamut limitations anyway, they can be too pure/saturated at certain angles.
I think Anders is looking at the effect that gamut limitations that are designed into the profile, will have on the overall look.
Cheers,
Bart
-
Thanks for the files. I've just made some brief observations. The camera is a 1DsIII, I'm assuming it's almost exactly the same as 5DmarkII so I used profiles for the latter as I have DCamProf profiles for it.
DCamProf makes no gamut mapping, Adobe Standard makes some small gamut mapping, but not very successful I think, just a quick compression towards the end which reduces the number of tones, then C1 gamut map quite heavily which is likely causing a bit of hue shift, red roses become more orange. The C1 rendering gives the impression of having most tones though.
I think what happens is that the red roses are so red that they glow, and in that red range the eye don't differ tones that well, so it looks less detailed. With the C1 gamut mapping the rose is less saturated and more orange and thus looks more detailed. C1's result looks quite similar to Trantor's SSF-based sRGB gamut-mapped profiles published in this forum a number of months ago.
I don't feel any urgent need to do anything with DCamProf in this area. A small gamut compression towards the edge like Adobe does seems to be a bad idea, and while the C1 more heavy gamut compression (Hasselblad's Phocus does similar things) can make better prints per default, it does distort color and it should be quite easy to do the same with post-processing techniques. But at some point I'll probably include gamut mapping features for those that want it.
-
Thanks for the files. I've just made some brief observations. The camera is a 1DsIII, I'm assuming it's almost exactly the same as 5DmarkII so I used profiles for the latter as I have DCamProf profiles for it.
Yes, quite likely they are rather close.
DCamProf makes no gamut mapping, Adobe Standard makes some small gamut mapping, but not very successful I think, just a quick compression towards the end which reduces the number of tones, then C1 gamut map quite heavily which is likely causing a bit of hue shift, red roses become more orange. The C1 rendering gives the impression of having most tones though.
Some of the tulips are indeed extremely red (I also have some backlit ones that really glow as if lit from the inside), and C1 does have a tendency towards orange if the conversion is pushed in brightness in postprocessing.
I think what happens is that the red roses are so red that they glow, and in that red range the eye don't differ tones that well, so it looks less detailed. With the C1 gamut mapping the rose is less saturated and more orange and thus looks more detailed.
Possible, but the details can also be achieved by boosting the higher spatial frequency (luminance) amplitude (I use "Topaz Labs Detail" for that), but that's a different beast than achieving it through a profile look. These are samples of very saturated tulips, not the 'regular' colorful ones one will commonly encounter.
I don't feel any urgent need to do anything with DCamProf in this area. A small gamut compression towards the edge like Adobe does seems to be a bad idea, and while the C1 more heavy gamut compression (Hasselblad's Phocus does similar things) can make better prints per default, it does distort color and it should be quite easy to do the same with post-processing techniques. But at some point I'll probably include gamut mapping features for those that want it.
Yes, I prefer the control that postprocessing offers, but if a profile adjustment can get us further in the right direction, then that might require less additional work and maybe less noise amplification. At least you have some more images to test your ideas on.
Cheers,
Bart
-
Ok, here the first results using a CC24 only. I'll work with the other charts later.
Except for the over-saturation and slightly reddish tint it's pretty close to what my eyes saw, much closer than the default P1 daylight profile.
Now I've to figure out what's going on with the reds, looking at the curve in Colorsync there's clearly an issue.
-
Is the oversaturated look only in the reds? I'm on the mobile phone now (bad screen), but I get the sense that the DCamProf result is globally oversaturated. Could be some kind of problem in the patch reading process rather than make-profile make-icc step. If you PM me all data (including the cc24 raw shot) I can test on my side.
If it's a Leaf back there is a risk I need to make something different.
-
It's globally oversaturated, needs -35 saturation or so in C1, regarding reds it's only some slight global cast.
The back is the IQ260, the phenomenon is the same with the aptus though. I've tried to use your CC24-ref.cie instead of my CC24 averaged spectro readings, and the red cast is a bit lower but still present.
I'm uploading data for you to have a look, will PM you the link.
-
I will look when I get a gap. I have some social stuff I need to attend, in order to maintain my popularity among my nearest.
-
Please do, nothing urgent on my side ;)
I will play with the IT8 and DT targets this afternoon
-
Got the files in PM, and I think I found the problem, the data does not get linearized as the transfer function is lost along the way (you got detailed solution in PM). DCamProf is not user-friendly so it's not a strange mistake to do. To the next release I'll make sure DCamProf aborts instead of just prints a cryptic warning in this situation.
-
Good catch, that was it. I admit I didn't notice the warning.
So far the profile behaves quite well, thanks to it I've managed to get rid of that weird yellowish mess I had in a lot of pics.
Time to play with looks a bit, before some work with the IT8 chart.
Thanks for your work !
-
Don't know if it's due to the use of the CC24 target but I notice its familiar gradual roll off of shadow detail down to black in the foreground tree trunk on the right and the background trees as I get on DNG profiles with my camera using that chart. I do much prefer DCamProf's rendering mainly for its overall uniformity of color saturation and definition over the P1 Daylight version. And I can see the overly yellow cast in the brick wall on the left and in the tree foliage but it's a nasty dirty yellow.
But I don't know how that profile was constructed.
I'm impressed with the results. Good work, Torger.
-
The dcamprof default neutral profile has been corrected since, here it is.
It clearly fixes that nasty yellow in the shadows on the left, and brings back the chroma of the rose building.
So yes, great job indeed :)
-
If the difference is very large between the bundled profile and the DCamProf result then one should suspect some error in the profile making process. As seen in the corrected example above the result is different but not huge, as Capture One generally makes decently neutral color, more neutral than Adobe does for sure.
As usual I'm not at my best screen (I'm moving this autumn so my work places are more chaotic than usual), but I think I can see that the main difference between P1 and DCamProf's result is that P1 is a fair bit warmer, ie more yellow. I've noted this in many bundled profiles, also Phocus for my Hassy warms up the colors a bit, but less global. I think a slight warmup in the green to yellow range can be an advantage for many landscape scenes (sunlit foliage) so I have that in the "look" example I provide in the tutorial, but I avoid applying it to blues, reds and shadows.
-
Here's an high saturation flower example. Thanks to Bart for the test file.
With high saturation colors the differences between different profiles start to become quite large. The attached JPEG has a Prophoto ICC embedded and you most likely need a wide gamut AdobeRGB-capable calibrated screen to get good display result. The embedded thumbnail has no ICC, so you need to click to enlarge to get the correct image.
I'm focusing on the red roses here.
I noted that I got quite large difference between an older version of DCamProf 0.8.2, and the current one. This is due to some difference in the neutral tone reproduction operator. I'm going to look into this, because I think the older result is better. The older does clip quite a lot, but lowering the exposure (final image) the tones are nicely rendered. I think this is better than the sharper rolloff of the current version.
In terms of tone separation I think ACR and current DCamProf 0.9.7 is quite similar, but ACR is a bit less saturated and a more orange hue. C1 render has good tone separation, but is quite desaturated and very orange hue. DcamProf 0.8.2 dark has same tone separation as C1 but I assume more correct hue.
-
I agree with your analysis. Dcamprof 8.2 is more natural, Version 9.7 gives the idea of artificial color (Tulip) with too much saturation.
After, it is difficult to say which interpretation is faithful,
-
Heh oh yes, red tulips of course, not red roses... ;D
Anyway, concerning what is most faithful I don't know the original, but both DCamProf versions are in agreement of what the colorimetric color is, the magic happens when the curve and neutral tone reproduction operator is applied, it works a little bit different between 0.8.2 and 0.9.7.
When the color is so saturated that we're dealing with clipping it's much a matter taste what is most "faithful", do we want to compromise saturation (and possibly hue) to gain better tone separation or not? It's a bit like comparing "perceptual rendering intent" vs "relative colorimetric" when making prints. My guess is that DCamProf 0.9.7 is more similar to "relative colorimetric" (be as correct as possible as far as possible and then just clip, with a short rolloff to avoid the ugliest artifacts), while DCamProf 0.8.2 is more "perceptual intent" with more desaturation to improve tonality. There's no intentional "gamut mapping" in 0.8.2 though, so there's something else going on.
At the moment I don't know what causes the difference, but I'm about to dive into it.
-
I now know what the difference between 0.8.2 and 0.9.7 is that causes the difference in look of the red tulips. Back in 0.8.2 the curve was applied in the luminance channel, but this was later changed to instead be "ghost-applied" in RGB-HSV and luminance taken from that (to get a more predictable/comparable result with a standard curve). Perhaps that was not a good idea, it does cause luminance separation to be reduced in this context.
I need to re-run tests on a bunch of images before I know if I can just revert it.
-
Maybe we could have make-icc to automatically populate the ICC creator tag with dcamprof version number ?
-
Just released 0.9.8.
Tested through my images and adjusted the tone reproduction operator accordingly so it renders saturated colors with better tone separation, like older version did. The look is not 100% the same as the old version, as there has been some other improvements on the way too.
I've attached a comparison, here all darkened to about the same level to make it easier to compare. Still a prophoto jpeg so calibrated wide gamut screen is recommended.
-
Maybe we could have make-icc to automatically populate the ICC creator tag with dcamprof version number ?
It didn't get into 0.9.8... I'll look into it. I'm not sure it's okay by the standard, the creator signature should be registered I think. There's the copyright string and (set with -c "my copyright string") and description string (-n "my description") which you can set if you want to tag them yourself.
-
Just to test whether a wide gamut screen is necessary to view your posted sample I've uploaded a screengrab of the first attempt to confirm whether the red tulips of the DCamProf versions are suppose to have less definition and be on the slightly magenta side of red.
I'm viewing this on a Colormunki Display calibrated LG IPS 27" LED display with its profile embedded in the screengrab. I find the C1 the more realistic rendering and seeing its color issues described by torger.
Just a suggestion but I'ld find a test image having a wide range of saturated objects but exposed to look as real as the scene and not have over cranked color. I don't find this image as looking anything close to reality when it comes to flower rendering.
-
It didn't get into 0.9.8...
will it be a nice idea to have a command line parameter (for relevant dcamprof commands), that if used explicitly will embed some comments into JSONs and pass also then to icc/dcp in appropriate tags as to what were command line parameters of dcamprof used to build JSONs and dcps/iccs (so the final dcp or icc output can have somewhat complete history how it was built if the you so desire, for debugging purposes for example)... $0.02
-
0.9.8 build for Windows ( mingw = dcamprof.exe + libgomp_64-1.dll + both manual & tutorial / = copies of Torger's web pages / in 3 formats : IE archive .mht, Mozilla archive .maff and regular .pdf ) : https://app.box.com/s/aj53xxbw2jy77uw4vdilk6ubathikb8p (https://app.box.com/s/aj53xxbw2jy77uw4vdilk6ubathikb8p)
-
Just to test whether a wide gamut screen is necessary to view your posted sample I've uploaded a screengrab of the first attempt to confirm whether the red tulips of the DCamProf versions are suppose to have less definition and be on the slightly magenta side of red.
I'm viewing this on a Colormunki Display calibrated LG IPS 27" LED display with its profile embedded in the screengrab. I find the C1 the more realistic rendering and seeing its color issues described by torger.
Just a suggestion but I'ld find a test image having a wide range of saturated objects but exposed to look as real as the scene and not have over cranked color. I don't find this image as looking anything close to reality when it comes to flower rendering.
It does not seem like your LG is a wide gamut monitor because the attached image is severely clipped. It's probably more close to an sRGB monitor? The image won't work on that, that's why I pasted a warning over the whole image. C1 rendering will work as the profile has built-in gamut mapping. I will probably introduce a gamut mapping option to DCamProf at some point, but currently you need to manually adjust post-processing to fit into your gamut.
Concerning if color is over-cranked or not it may be that, but it's not intentionally so. If it is then it's due to precision errors in the profiling, which could be the case as it's only a inkjet semi-glossy target that has been used to match high saturation reds and I'm not sure how well it works on the tulips. At some point I may try base it on SSF and real tulip data from the FReD database.
In any case C1 is under-cranked for sure (gamut-mapped to a smaller gamut), and the tulips should be more saturated than sRGB can handle, so if displayed in a small space the results will not be great. The intention with using the tulip image is to test behavior in the range that is outside sRGB, so the base colorimetric accuracy is not really that important for the test itself. I just needed ultra-high saturation colors with fine tonal gradients, and that image is excellent for that purpose.
-
will it be a nice idea to have a command line parameter (for relevant dcamprof commands), that if used explicitly will embed some comments into JSONs and pass also then to icc/dcp in appropriate tags as to what were command line parameters of dcamprof used to build JSONs and dcps/iccs (so the final dcp or icc output can have somewhat complete history how it was built if the you so desire, for debugging purposes for example)... $0.02
Argyll can do that, embed lots of debug data in the ICC profile. Unfortunately DNG profiles does not have the same possibility to embed debug data (AFAIK), and although I think the idea is good I'm not prioritizing features that can only be implemented in one of the formats.
-
I might adjust the curve handling in the neutral tone reproduction operator further, I'll do some testing. I'm not 100% sure the 0.9.8 change was 100% good.
A reason I was using a RGB-HSV curve as luminance guide earlier was that the S-dip in the shadow range becomes a bit darker and duller with the pure luminance curve, which had some negative effect on shadowy skin tones. The difference is very small, but it's there. As the curves are typically designed using some sort of RGB curve, it may be wise to have a luminance that better matches RGB. I guess that if you do design the curve with luminance curve in mind you would simple not make as big S-dip in the shadow range and then you'd get rid of the dullness, so it's possibly not the fault of the luminance curve as such.
The RGB(-HSV) curve treats R G and B equal, that is over-estimates the luminance of red and blue. This means that dark colors with low green gets lighter. It also makes specifically saturated reds lighter and tonally tighter spaced, which is why the ACR rendering has worse tone separation in the tulips. The same can to some extent be observed in the blue range, but the perceptual effect is much larger in the red range (as saturated blues are dark anyway).
A pure RGB curve like C1 is using for its film-curve has the same problem with tonal compression of reds, but their ICC profile is compensated for that. Overall ACR seems to "suffer" more of the look side-effects the curve has as their profiles are not as compensated (I think they originally intended that the RGB-HSV curve would be used as-is with no compensations), while C1 has much more designed color.
There's no surprise C1 has better reputation than ACR when it comes to color. It's not an issue of the DNG profile format though, with designed profiles you can get the color you want, DCamProf profiles look the same with ICC as with DCP.
-
A bit OT, sorry...
W. Faust sent me spectral data for its C1 IT8 chart, and those are in either .cxf or .cgt format. Does someone have a way to convert one of them into a CGATS style file ? It seems cxf is used by X-rite but I i1profiler refuses to load this one.
Thanks.
-
A bit OT, sorry...
W. Faust sent me spectral data for its C1 IT8 chart, and those are in either .cxf or .cgt format. Does someone have a way to convert one of them into a CGATS style file ? It seems cxf is used by X-rite but I i1profiler refuses to load this one.
Thanks.
They are text files and .cgt is in CGATS format
-
Indeed, I should have said Argyll CTI2 or CTI3 format instead. As is the .cgt file is not recognized ?
cxf is an xml file.
-
Indeed, I should have said Argyll CTI2 or CTI3 format instead. As is the .cgt file is not recognized ?
Edit the header to make it CGATS then you can use it in dcamprof (if that's what you are trying to do)
-
Problem is to get the spectra, it has columns for both nm and amplitude, I haven't seen that before. Usually the nm is encoded into the column name.
You can manually edit the file to remove the nm columns and add nm to tho column name, but it's quite tedious. I'll look into making a parser for this column layout. Argyll has some conversion tools, txt2ti3, kodak2ti3 etc but I don't think it has anyone for this format.
-
Here you go. I will add the feature to parse these IT8.7/2 CGATS files directly with make-target to the next release.
-
Great, thanks again !
Back to dcamprof, it seems I can't get a true black point in C1 no matter how I try to mimic its built-in curves & profiles. Is it related to the curve handling you talked about earlier ?
This can be fixed easily once in photoshop, so no big deal, just thought I'd ask.
-
Great, thanks again !
Back to dcamprof, it seems I can't get a true black point in C1 no matter how I try to mimic its built-in curves & profiles. Is it related to the curve handling you talked about earlier ?
This can be fixed easily once in photoshop, so no big deal, just thought I'd ask.
It's sounds more likely related to ICC LUT generation, the code that I have rewritten, patched and adjusted like a million times :)
I assume that with the bundled profile you get pure black RGB = 0,0,0 and when you switch to your DCamProf profile it's no longer black? Rather than a bug it could possibly also be the case that the bundled profile makes some sort of black subtraction, which DCamProf doesn't do. It would be relatively easy to add though.
If you can describe a bit more in detail what behavior you expect/want (perhaps with an example) I can look closer at it.
-
I'll send you some examples tonight.
With dcamprof the very dark areas can be a bit muddy, while with C1 profiles they have a crisper/contrastier look (maybe thanks to a steeper curve in the very low end that is hard to recreate with C1 curve editor).
-
A bit OT, sorry...
W. Faust sent me spectral data for its C1 IT8 chart, and those are in either .cxf or .cgt format. Does someone have a way to convert one of them into a CGATS style file ? It seems cxf is used by X-rite but I i1profiler refuses to load this one.
Thanks.
BabelColor PatchTool will convert for you, for example see attached
PS: I do not think that W. Faust does individual measurements, just for batches... so you might want to experience the pain of doing several passes over IT8 with your own spectrophotometer... or not.
-
BabelColor PatchTool will convert for you, for example see attached
PS: I do not think that W. Faust does individual measurements, just for batches... so you might want to experience the pain of doing several passes over IT8 with your own spectrophotometer... or not.
Correct, Wolf Faust produces small batches that get measured (see footnote (http://www.targets.coloraid.de/) under quality). Depending on the material it's produced on (reflective/transmissive and film/paper brand), they can be more or less recent productions (exposures) with his own equipment. Some targets have a larger gamut than IT8s from other sources, which can help with profiling. The quality is excellent, and the price is usually much better than of other makes. Custom targets can also be produced, and Wolf Faust understands the difference between using white backing behind the target or not.
Cheers,
Bart
-
I wanted to do my own measurements but according to Mr Faust my i1pro - early rev A model - could experience important errors with high densities. So far the profiles built with his batch spectral data are in tune with the CC24 based ones.
-
a question for Torger :
what are the minimum set fields that shall be present in .ti3 target file ("[target.ti3]") for dcamprof test-profile command ?
-
Release of the day: 0.9.9
Added the IT8.7/2 parsing and refined the change I made yesterday in 0.9.8.
Now the tone reproduction uses both RGB-HSV based luminance and pure luminance to get "best of both worlds". The pure luminance is used for high chroma colors, such as the red tulips, while the luminance based on a RGB-HSV curve is used for low and normal saturation colors.
The pure luminance curve is more "correct", but as a curve is as a perceptual model is not "correct" anyway it's not really valid argument. What I've seen earlier (together with the RT team) is that the RGB-HSV-based luminance lighten shadows in a way that simply looks better. The difference is very small, but after some further testing I decided that it was worth bringing it back. With a smooth transition into a pure luminance curve for high saturation colors the good tonal separation properties are kept.
-
what are the minimum set fields that shall be present in .ti3 target file ("[target.ti3]") for dcamprof test-profile command ?
it depends. You can regenerate XYZ values from spectra if spectra is available in the file, otherwise it needs XYZ. Likewise it can regenerate RGB values from SSF if SSF is provided, otherwise it needs RGB values.
With no spectra, the minimum should be:
SAMPLE_ID RGB_R RGB_G RGB_B XYZ_X XYZ_Y XYZ_Z
(it currently does not parse Lab values, it wants XYZ)
-
it depends. You can regenerate XYZ values from spectra if spectra is available in the file, otherwise it needs XYZ. Likewise it can regenerate RGB values from SSF if SSF is provided, otherwise it needs RGB values.
With no spectra, the minimum should be:
SAMPLE_ID RGB_R RGB_G RGB_B XYZ_X XYZ_Y XYZ_Z
(it currently does not parse Lab values, it wants XYZ)
so I can surely construct the file myself like this with 2 parts, 2 set of fields :
1) RGB_R RGB_G RGB_B fields I can extract (rawdigger) from the actual raw shot, and those will be used by dcamprof along with the JSON profile data to come to what conversion from the actual raw shot shall be
+
2) spectral data (or XYZ) so that dcamprof can compare the values from the above (received from actual RGB from the raw + applied profile) with what target values are under the specific illumination
-
Globally, it looks a lot more saturated than 0.97 and 0.98
Release of the day: 0.9.9
Added the IT8.7/2 parsing and refined the change I made yesterday in 0.9.8.
Now the tone reproduction uses both RGB-HSV based luminance and pure luminance to get "best of both worlds". The pure luminance is used for high chroma colors, such as the red tulips, while the luminance based on a RGB-HSV curve is used for low and normal saturation colors.
The pure luminance curve is more "correct", but as a curve is as a perceptual model is not "correct" anyway it's not really valid argument. What I've seen earlier (together with the RT team) is that the RGB-HSV-based luminance lighten shadows in a way that simply looks better. The difference is very small, but after some further testing I decided that it was worth bringing it back. With a smooth transition into a pure luminance curve for high saturation colors the good tonal separation properties are kept.
-
Globally, it looks a lot more saturated than 0.97 and 0.98
There should be a very small difference. Are you sure that there's not something else in your process that has changed? I'll double-check on my side.
EDIT: I see your example now, I could possible have missed that, I made a last minute change that I did not fully verify... I'll look into it.
-
I've checked on a couple of pics with profiles based on CC24 and IT8, strangely blues seem to be more affected than other hues ?
-
Error confirmed... I'll take down the 0.9.9 and put it up again when fixed, I think it's a last minute bug....
-
Now a new 0.9.9 is out, let's pretend that never happened :). I had made a last minute debug check by turning off mid/high chroma rolloff, and forgot to turn it back on before releasing.
I just uploaded a fixed archive.
-
Fixed indeed. I'll check more thoroughly tomorrow, for now I see some changes in very high chroma values and can't say what looks "best" yet.
-
Fixed indeed. I'll check more thoroughly tomorrow, for now I see some changes in very high chroma values and can't say what looks "best" yet.
In high chroma it could be just "different" rather than better, the change did not really intend to make changes there. What I looked at is the shadowy parts of a face, a dark shadow can look something like 1-2 deltaE brighter. It's a tiny difference but I remember the lengthy work in July when I and the RT team (mainly Michael Ezra) tuned the curve stuff for skin tones, and that little difference was needed to get it right.
In saturated tones the difference between the RGB-HSV and luminance only is large, so in the transition area (25 to 65 chroma, ie wide transition) one should detect differences if doing A/B swapping, this is not intended as an improvement or anything like that, just a transition.
The intention with 0.9.9 is that the "old" curve (0.9.7) is used on normal saturation colors and the luminance curve (0.8.2, 0.9.8 ) is used on high saturation colors. And the two reasons the old curve is used on normal saturation colors is that we had better results on shadowy skin tones, and that it makes it easier to import curves designed in other contexts as most curves are designed in some RGB-like situation.
-
0.9.9 build for Windows ( mingw = dcamprof.exe + libgomp_64-1.dll + both manual & tutorial / = copies of Torger's web pages / in 3 formats : IE archive .mht, Mozilla archive .maff and regular .pdf ) : https://app.box.com/s/aj53xxbw2jy77uw4vdilk6ubathikb8p (https://app.box.com/s/aj53xxbw2jy77uw4vdilk6ubathikb8p)
-
In high chroma it could be just "different" rather than better, the change did not really intend to make changes there. What I looked at is the shadowy parts of a face, a dark shadow can look something like 1-2 deltaE brighter. It's a tiny difference but I remember the lengthy work in July when I and the RT team (mainly Michael Ezra) tuned the curve stuff for skin tones, and that little difference was needed to get it right.
In saturated tones the difference between the RGB-HSV and luminance only is large, so in the transition area (25 to 65 chroma, ie wide transition) one should detect differences if doing A/B swapping, this is not intended as an improvement or anything like that, just a transition.
The intention with 0.9.9 is that the "old" curve (0.9.7) is used on normal saturation colors and the luminance curve (0.8.2, 0.9.8 ) is used on high saturation colors. And the two reasons the old curve is used on normal saturation colors is that we had better results on shadowy skin tones, and that it makes it easier to import curves designed in other contexts as most curves are designed in some RGB-like situation.
Below is what can be observed in high chroma areas in 0.9.9 and 0.9.7 (I would have said it was a 0.9.8 profile but according to your explanations it must be 0.9.7).
There's nothing obvious going on in the shadows.
-
Below is what can be observed in high chroma areas in 0.9.9 and 0.9.7 (I would have said it was a 0.9.8 profile but according to your explanations it must be 0.9.7).
There's nothing obvious going on in the shadows.
Thanks for the example. 0.9.7 and 0.9.9 is using the same curve for low to normal saturation curves, so they should be exactly the same there (except for possibly some microscopic differences due to a simplification of mixing code), but 0.9.9 transitions into a pure luminance curve for high saturation colors. This means more focus on tonal separation rather than maintaining chroma. The largest difference is observed in orange/reds.
Very bright saturated orange/reds becomes more desaturated in 0.9.9 which is observed in this example. In this particular example I'd say the 0.9.7 rendering is better as it keeps the chroma of the lights and as the lights are flat and featureless anyway there's no gain of tonal separation. The 0.9.9 also suffers from that the orange is so bright it's pushed into the heavily desaturated range. But that's the way it is, it's not possible to make one profile that is best for all situations. The 0.9.9 is more all-around which can be demonstrated on the tulip pictures, so that's what I'm going to prefer as the default.
However, with 0.9.9 it's possible to configure the curve transition of the neutral tone reproduction operator, if you want the 0.9.7 behavior you can enable that by setting:
"Curve": { "KeepFactor": 1.0 ... } (that is keep factor to 1.0 then the base RGB-HSV curve is never faded out)
in an ntro_conf.json and provide that with the -o option. See data-examples/ntro_conf.json for a full example and documentation.
-
Considering what I shoot most of the time, the former curve is preferred. Nice to have the option for high chroma cases though.
There are some stretches in my LUT I want to fix. I wanted to give gnuplot a try but some data are missing in my dump. All I have is the icc-lut.dat, so how can I generate the others ?
splot \
'icc-lut.dat' w d lc "beige", \
'gmt-locus.dat' w l lw 4 lc rgb var, \
'gmt-adobergb.dat' w l lc "red", \
'gmt-pointer.dat' w l lw 2 lc rgb var, \
'target-icc-lutve.dat' w vec lw 2 lc "black", \
'targetd50-xyz.dat' pt 5 ps 2 lc rgb var
-
As usual I'm not at my best screen (I'm moving this autumn so my work places are more chaotic than usual), but I think I can see that the main difference between P1 and DCamProf's result is that P1 is a fair bit warmer, ie more yellow. I've noted this in many bundled profiles, also Phocus for my Hassy warms up the colors a bit, but less global. I think a slight warmup in the green to yellow range can be an advantage for many landscape scenes (sunlit foliage) so I have that in the "look" example I provide in the tutorial, but I avoid applying it to blues, reds and shadows.
Phase One should have a look at the Leaf Product curve and profiles and build them for their IQ backs. Not everyone is shooting sunny landscapes only, so a more neutral and less yellowy Daylight or Flash profile would be welcomed I guess...
Given how "easy" it is to create a rather neutral profile with Dcamprof and a simple CC24, it baffles me they haven't released one. Most IQ owners I personally know are doing fashion/portrait and their import presets all have quite strong corrections applied through C1 color editor.
/rant over
-
There are some stretches in my LUT I want to fix. I wanted to give gnuplot a try but some data are missing in my dump. All I have is the icc-lut.dat, so how can I generate the others ?
I haven't documented it very well (ie not at all) but different dcamprof commands produce different data. If you only have the icc-lut.dat you probably have only dumped data for the make-icc command. To get all those files you should dump data for the make-profile command.
-
Considering what I shoot most of the time, the former curve is preferred. Nice to have the option for high chroma cases though.
If you want less desaturation of bright colors the "Rolloff" section has an even more powerful impact. Again, the default is adapted for "all-around photography" which means that there's quite long transition with much desaturation, this makes things like high key portraits look better (otherwise the look a bit tonally compressed and dull in the highlights). However it will also make things like blue skies more desaturated than they need to be. Changing the KeepFactorHueCurve output to 0.40 instead of the default 0.20 can be a good starting point for experimenting with that. An example picture with a bright blue sky is good for testing the effect. With the example custom look provided the KeepFactor is varied over the hue range, with a lower value in skin tone range and a higher value in cyan-blue-magenta (skies).
Default:
// How much of the correct color to keep above high limit. Can be adjusted per RGB-HSV Hue
"KeepFactorHueCurve": {
"CurveType": "Linear",
"CurveHandles": [ [ 0,0.20 ], [ 360,0.20 ] ]
},
Suggested alternative to try:
// How much of the correct color to keep above high limit. Can be adjusted per RGB-HSV Hue
"KeepFactorHueCurve": {
"CurveType": "Linear",
"CurveHandles": [ [ 0,0.40 ], [ 360,0.40 ] ]
},
The center image below is with 0.20, and the right is with 0.40 (the left is ACR, ignore that). Look at the sky to see the difference. The right also contains other look changes, such as the warmup discussed previously, you would not use that :)
(http://www.ludd.ltu.se/~torger/photography/img/cp-acr-nolook-look.jpg)
-
Thanks, the dumps were made with make-icc only, I'll try with make-profile now.
I will test your suggestions, so far the 0.97 behaviour along with a slight desaturation of close to neutral colors (custom look) is giving me a rather good starting point.
-
will test your suggestions, so far the 0.97 behaviour along with a slight desaturation of close to neutral colors (custom look) is giving me a rather good starting point.
Great. Did you have any example of the black-point problem mentioned a few posts back so I can look into that, or has it been fixed? I suspect that it could be a "custom look" type of thing rather than a bug, but in any case it's worth investigating.
-
I fixed this one by trial and error, changing values in the json file rather than trying to duplicate the curve in C1 and using those values.
When it comes to mimicing a curve the best approach may be to roughly get there with C1 curve editor, and then fine tune the values in the json file.
-
I fixed this one by trial and error, changing values in the json file rather than trying to duplicate the curve in C1 and using those values.
When it comes to mimicing a curve the best approach may be to roughly get there with C1 curve editor, and then fine tune the values in the json file.
Ok, great. I'll add that tip to the tutorial.
-
My setup (OS X + aquaterm) doesn't let me navigate the plot and locate the bad patches precisely. Unfortunately I won't have time to investigate further in the coming days.
Time to let someone else keep Anders busy ;)
-
The upcoming 0.9.10 release will have a gamut compression function as a look operator.
Attached a ProPhoto jpeg demonstrating the result.
EDIT: Arggghhh after the forum update the ICC profile is stripped from the uploaded JPEG, so I can't use that any longer. Removed the attachment. Here's a direct link to the image:
http://torger.dyndns.org/gamut-compression.jpg
Embedded:
(http://torger.dyndns.org/gamut-compression.jpg)
-
If you like the concept of gamut compression in the profile, it's probably still a wise thing to compress to a larger gamut than the target gamut, as you still have good tonality and decent hue stability even with parts of one channel clipped.
The posted example above has the target gamut to exactly the border (AdobeRGB and sRGB in the example), which means that there's no clipping. This makes it unnecessarily desaturated. The AdobeRGB result looks quite good when clipped to sRGB, but with DCamProf you can simply add say 20% extra range to the sRGB gamut which is what you would do.
-
The upcoming 0.9.10 release will have a gamut compression function as a look operator.
Good idea. I noticed you have lowered the adjustfactor for saturated colours, was it related to the curve change ?
-
I noticed you have lowered the adjustfactor for saturated colours, was it related to the curve change ?
No, I just re-adjusted it because it was a bit too high to my eyes. It only makes a change to really super-saturated colors.
The purpose of the neutral tone reproduction operator is to maintain color appearance from the linear colorimetric rendering. So what I use as a reference is the linear colorimetric rendering, push exposure a bit to make it easier to compare (important with linear exposure push, I use RawTherapee for this), and the do an A/B swap and look globally over the whole image and think "did the saturation increase or decrease?" and the adjust these factors until the saturation appearance is the same.
For an ideal result I would probably need a more complex function than the current simple rolloff, but I think it's adequate as is. A 100% perfect result is in any case not possible as human color perception is much too complex to just model with a curve and static saturation adjustments.
-
I've now uploaded 0.9.10. The only new stuff is the gamut compression look operator. The ntro_lookop_conf.json contains a commented example, so it should be easy to figure out from that.
Hopefully it will now be a bit wider space between releases again. Now after gamut compression is done I have no (important) features left on the todo list.
-
I'm shooting way too few sunsets obviously.
I've become aware of that the tone reproduction operator, and the new gamut compression feature, has some pretty severe look issues around the sun, or more general the very last step into clipping. It can clip into white too early, or clip into white and then go back into color, and then clip again.
This is a bug of the severe kind considering how popular sunsets are, so it's a high priority to fix this. I don't know yet if it's a hard or easy fix.
It's strange how work intensifies after the software is "finished" and people start using it :)
-
All look things, even "simple" Kodachrome, and I mean film, not imitations, have their issues. It is important that the start has an option of being more or less colorimetric, and looks are achieved in a profile editor or in the image editor / converter. One or even a dozen of fixed looks can't cover it all. Some issues will remain, whatever we try.
-
0.9.7 ICC profiles are unaffected (or if you set keepfactor for curve to 1.0 on 0.9.10), while there is some DCP-specific clip issue also in the older version.
The curve adaptations for the tulip thing was really a can of worms... those that want a bit of stability until I've sorted this out should stay with 0.9.7
This may be one of those cases Iliah mentions, that there's no good way to solve both cases well, but I'll try some more.
So I have three things to fix
1) luminance curve that was added for the tulips has no good clip behavior
2) DCP looktable has some clip issue that can cause a slight artifact near clipping, which is not seen if an ICC profile is rendered
3) I must rethink clipping behavior for the gamut compression feature, it's now no good.
-
those that want a bit of stability until I've sorted this out should stay with 0.9.7
I think I will put 2 binaries in the next build then
-
0.9.7 is (mostly) unaffected (or if you set keepfactor for curve to 1.0 on 0.9.10)
can you please add this line to the manual (web page) may be ? I mean to the news section of it
-
can you please add this line to the manual (web page) may be ? I mean to the news section of it
I think you can stay low with publishing new builds until I get 0.9.11 out. I'm thinking of reverting back to 0.9.7 on the web, I'll do that if it starts to take a long time to get 0.9.11 out.
It's good to get things out early as my users contribute to testing (the "user contract" is to be prepared that new stuff can be buggy, and inform me if they find severe bugs), but now when I have these bugs on the table I don't need more reports for the moment.
I've added a note on the front page.
-
I think you can stay low with publishing new builds until I get 0.9.11 out. I'm thinking of reverting back to 0.9.7 on the web, I'll do that if it starts to take a long time to get 0.9.11 out.
It's good to get things out early as my users contribute to testing (the "user contract" is to be prepared that new stuff can be buggy, and inform me if they find severe bugs), but now when I have these bugs on the table I don't need more reports for the moment.
I've added a note on the front page.
may be also put a prominent contact web form how the users can submit bugs somewhere @ the top of the manual itself (may be even above the news section for lazy ones), listing what shall be supplied to you...
-
2) DCP looktable has some clip issue that can cause a slight artifact near clipping, which is not seen if an ICC profile is rendered
This was not a bug, except from a poor default. The -h 90,30,15 with gamma encoding leaves too few table entries close to clipping. Either disabling gamma encoding (-G) or increasing to -h 90,30,30 removes that issue. I'll investigate the best default and hard-code that to the next release.
-
MS Windows build v0.9.7 back for downloading @ https://app.box.com/DCamProf-Windows-MingW
-
That last sample tulip image you posted inline (not linked) has the (RT) profile embedded. I extracted it using Colorsync script and it appears to be a 2.2 gamma version of ProPhotoRGB going by Photoshop's CustomRGB in Color Settings dialog box.
The 3D gamut plot is identical to ProPhotoRGB viewing in Colorsync Utility.
The DCamProf tulip preview on my sRGB display viewed in Photoshop shows the tulips as blobs of red, no petal distinction detail as in the other two to the right. I just assigned 1.8 gamma ProPhotoRGB to the image and the petal distinction was restored but the entire image of course lightened accordingly from the 1.8 gamma assigned to 2.2. gamma encoded data.
-
I'm having good progress in fixing the issues, hopefully I can release a 0.9.11 today.
-
I have now released 0.9.11. Now everything should be back to normal, and I urge 0.9.7 users to move to 0.9.11 and test that.
The main problem was a clipping bug that was introduced in 0.9.8 and thus exists in 0.9.9 and 0.9.10 as well. This is now fixed.
This means that those of you that preferred the "old curve" of 0.9.7 should still try 0.9.11, as the desaturation (such as seen in the traffic lights of Frederic's example a while back) should now be gone. The new curve may be a little bit desaturated still, but it should not be a big difference. The advantage of the new curve is same as before, tonality of high saturation colors (mainly reds) are significantly improved, which can be demonstrated with Bart's tulips. If you still prefer the 0.9.7 curve you can reconfigure by setting KeepFactor to 1.0 as discussed before, so there's no reason to stay with 0.9.7 now (assuming I've not missed some other severe bug, but then I'm glad to get help finding it :) ).
The Gamut compression has also been fixed by adding a compress limit configuration (suitable defaults in provided example config), so we let the colors go outside our destination gamut near clipping, which is necessary for making a smooth transition into the sun.
You may still get a "fried egg" effect around the sun depending on how your lens has reacted because DCamProf keeps more color in highlights than say ACR. The fried egg is there in the raw (so it's not a profile artifact as such), but if you keep more color it becomes more visible. This can be re-configured if you're a frequent sunset shooter, but I prefer the current tradeoff.
-
On a different topic, can targets with different spectral resolutions be merged into a single ti3 ? One chart is in 5nm increment and goes up to 730nm, the other in 10nm increment and up to 780nm.
I'm not getting error messages while attempting to merge, but have seen the end of certain lines filled with 0s in the resulting file.
-
On a different topic, can targets with different spectral resolutions be merged into a single ti3 ? One chart is in 5nm increment and goes up to 730nm, the other in 10nm increment and up to 780nm.
I'm not getting error messages while attempting to merge, but have seen the end of certain lines filled with 0s in the resulting file.
Yes it's fine. DCamProf fills out with zeros such that all patches cover the same range after merge.
-
0.9.11 build for Windows ( mingw = dcamprof.exe + libgomp_64-1.dll + both manual & tutorial / = copies of Torger's web pages / in 3 formats : IE archive .mht, Mozilla archive .maff and regular .pdf ) : https://app.box.com/DCamProf (https://app.box.com/DCamProf)
-
I'll try to get a 0.9.11 MacOSX static build out tonight (after parent/teacher conference :-[)
Do people find the MacOSX build useful or am I wasting my time?
-
I'll try to get a 0.9.11 MacOSX static build out tonight (after parent/teacher conference :-[)
Do people find the MacOSX build useful or am I wasting my time?
I think a download counter would be more reliable than getting replies in this thread, I've now put up a link to this thread from my web page, but I guess most people that downloads won't be members in the forum, they just read, find the link and download.
I will go for a vacation over the weekend now so there won't be any 0.9.12 for at least a few days at least ;D
-
Oh, I suppose.
But that would require effort ::)
It'd be nice if box.com made that available w/o spending $45/month which aint gonna happen.
Maybe some other method. Anyone got a suggestion?
-
It'd be nice if box.com made that available w/o spending $45/month which aint gonna happen.
why do you need to pay ? this is not a gigabyte traffic... I keep builds in my free box.net/box.com account... you do not have thousands of people downloading it... I think the most ever downloads for my build was ~25 recently... a lot of people will build themselves... so just use a free account... and free account has some counter where you can see how many times somebody downloaded or at least attempted to download the file
-
I didn't see any free option for download counting or stats. It wanted a business plan upgrade at $15/month x 3
Anyway, I signed up for bitly.com which does what I need and future links will (assuming I remember) be bitly.
-
I didn't see any free option for download counting or stats. It wanted a business plan upgrade at $15/month x 3
Anyway, I signed up for bitly.com which does what I need and future links will (assuming I remember) be bitly.
there are no special options, it is just there in a simple form, w/o any details - see the simple counter ? "3 downloads or attempts"
(http://s29.postimg.org/5f9o9j76v/box.jpg)
-
Thanks much, I had not noticed that and I do wish web designers would stop with the 'light grey text on white' meme. Usability sacrificed on the alter of trendiness.
Apparently 6 brave souls have downloaded 0.9.7
-
I have about 50 downloads of the 0.9.7 source archive. It should have spanned only one week, but due to the bugs in 0.9.8 - 0.9.10 it got some more downloads. The page itself got 650 views the first 15 days of September, and the tutorial about 450 in the same period. So nothing massive, but a noticeable interest. The interest seems to be rising after the tutorial was published.
When searching on "camera profiling" on Google (using incognito page to not get personalized result) the dcamprof page is still as far back as page four. I hope and think it will rise as time goes by, as there really isn't much software out there in this genre. DCamProf can do things no commercially available software can do. It's command line though, but Argyll is too and it does pretty well so I think DCamProf will find its users sooner or later :) . Meanwhile it's nice to have some brave early adopters.
-
Seeing hue shifts in skies that weren't so obvious previously, despite a very relaxed LUT.
Will investigate tonight...
-
DCamProf can do things no commercially available software can do. It's command line though, but Argyll is too and it does pretty well so I think DCamProf will find its users sooner or later :) . Meanwhile it's nice to have some brave early adopters.
I for one am following this with a great deal of interest. I'm waiting until a few more of the issues have been sorted out and the weather turns a little colder (the best time for sitting at ones work station running various programs). I don't mind command line as I've been using Argyll for some years now. Keep up the good work here!! I've been reading through the tutorial.
-
Seeing hue shifts in skies that weren't so obvious previously, despite a very relaxed LUT.
Will investigate tonight...
Bad LUT bends are not manifested like this (smooth hue shifts), if you would have that you see things more similar to posterizing. It's quite difficult to make bad LUT bends visible in real images as one have to have a gradient that exactly hits the area of the bad bend. That's why I think plot analysis is an important tool when doing more advanced design otherwise those bad bends are easily missed.
We've had a recent history of troubles in the highlights, rolling off towards clipping is a quite difficult problem, so there can be yet a problem there. First thing would then be to disable the curve we've had issues with by setting KeepFactor to 1.0 (default 0.0) in the Curve section. In the neighbor "dummies" thread there was an issue with magenta cast close to clipping (which does not happen in the latest 0.9.11), but the magenta here seems too far off from clipping to be related to that.
However skies can go from cyan to magenta for real, a bit depending on where the sun is and atmospheric conditions. DCamProf keeps saturation closer to clipping than typical bundled profiles so it may have just made a real sky hue shift more visible.
If it's not there at all with the bundled profile, then it's probably a problem. If it's just more visible it's probably not a bug, but it may be an undesirable look to you. Then letting it desaturate more can be an option. You can then reconfigure the tone reproduction rolloff to this for example:
"Rolloff": {
"KeepFactorHueCurve": { "CurveType": "Linear", "CurveHandles": [ [ 0,0.0 ], [ 360,0.0 ] ] },
"LowSatScaleHueCurve": { "CurveType": "Linear", "CurveHandles": [ [ 0,1.0 ], [ 360,1.0 ] ] },
"HighSatScaleHueCurve": { "CurveType": "Linear", "CurveHandles": [ [ 0,1.0 ], [ 360,1.0 ] ] }
}
(read data-examples/ntro_conf.json for an explanation of the parameters)
Another more advanced configuration would be to use the stretch operator to even out light blue sky colors, like you can even out skin color.
When it comes to these type of color issues, if there is a bug it's most likely related to the tone reproduction operator, so what one can do is to render a linear profile (no tone curve) and have that as a reference to compare with. If the color shift is not there with the linear profile it is a problem. One can of course also compare to the bundled profile and I would do that as well, but then keep in mind that there's a (minor) risk that they have already done things like smoothing out colors.
-
I for one am following this with a great deal of interest. I'm waiting until a few more of the issues have been sorted out and the weather turns a little colder (the best time for sitting at ones work station running various programs). I don't mind command line as I've been using Argyll for some years now. Keep up the good work here!! I've been reading through the tutorial.
Thanks for the interest. When I think it's stable enough for a wider audience, probably meaning that it's been out for a while and used for some period with no needs to update, there'll be a release of version 1.0.0. Hopefully it's quite close :)
-
Even with the former curve or the rolloff changes the shift is still there. It looks a bit better in the clouds with the stock profile (attached) though there's some magenta on the right too.
As you can imagine we're close to clipping, and to make matter worse it's an area corrected by an LCC.
I'll check if it's present in the linear profile too. Your suggestion to even out the blue area looks like the way to go.
edit : apparently not relaxed enough according to the table
-
Note that a ICC 3D LUT consists about 50% or so of out of gamut values, values that are not given by the native profile, values that DCamProf just needs to make up from thin air. Normally those values are never displayed as no real color will cause them to appear, but you can force them by extreme white balance settings.
The jaggies on that curve can be such values, ie not really affecting the image quality. I'm thinking about smoothing each curve and may do that at some point, but it's a quite difficult 3D problem (ie you can't really just smooth that curve in isolation as it affects others in 3D) and it's easy to disturb "real" entries which happened in the first attempts.
Making ICC LUTs that look smooth from every angle in a viewer is quite difficult, and it's not contributing to image rendering, which means I've put it on a low priority. The difficulty to make an ICC LUT due to all those out of gamut entries was a problem that took me by surprise. The DNG profile LUT doesn't have this problem as they work with multiplications and additions rather than absolute output values (but that has other drawbacks such that it's not possible to alter the neutral axis).
-
Commenting on the actual image, if I would guess I don't think the magenta cast or cloud edge is an issue due to a bad LUT bend or a bug, I think the color is there in the original, and it's visibility is magnified due to the shape of the tone curve.
If you look at the stock profile's clouds you can see that it's lower contrast than in DCamProf's rendering. Take the cloud fragment in the bottom left corner as an example, in the stock profile it almost disappears into the surrounding sky, while in DCamProf the cloud edges are more visible.
This means that the manually made LUT curve (the one you make with the curve tool and then copy to a JSON file) does not compress the highlights as much as the stock profile. To get a result more similar to the stock profile try compress highlights more in your curve. This will lower cloud contrast and desaturate the colors, so the "cast" (which I think is a natural sky variation) will be less visible and the clouds will also look more uniform.
(For readers unfamiliar with this manual curve -- this is a special step required by Capture One only. It may be possible to extract it from the LUT and by that copy the stock profile's curve, say by probing the neutral axis, but so far one have to design it manually and I don't think it's that bad as curve design is a matter of personal taste too, the stock curve is not necessarily the best.)
-
v0.9.11 Mac OSX binary is up...........
https://app.box.com/s/4y3kq5laepy1yp18i69t006jt9zpv0bd
Interestingly, the OSX binary is only 6.3MB (and static) whereas the Windows one is 14.
-
Oh, I suppose.
But that would require effort ::)
It'd be nice if box.com made that available w/o spending $45/month which aint gonna happen.
Maybe some other method. Anyone got a suggestion?
Google Drive - does not cost a penny
-
Thanks but the only reason I mentioned that was that box.com didn't *seem* to offer a simple (read: free tier) download counter but that was only because when I looked, no one had d/l'd it yet.
-
Interestingly, the OSX binary is only 6.3MB (and static) whereas the Windows one is 14.
it is not... I include 3 versions of manual and tutorial as standalone self contained files... otherwise it is ~8MB + ~100Kb OMP .dll
-
Commenting on the actual image, if I would guess I don't think the magenta cast or cloud edge is an issue due to a bad LUT bend or a bug, I think the color is there in the original, and it's visibility is magnified due to the shape of the tone curve.
If you look at the stock profile's clouds you can see that it's lower contrast than in DCamProf's rendering. Take the cloud fragment in the bottom left corner as an example, in the stock profile it almost disappears into the surrounding sky, while in DCamProf the cloud edges are more visible.
This means that the manually made LUT curve (the one you make with the curve tool and then copy to a JSON file) does not compress the highlights as much as the stock profile. To get a result more similar to the stock profile try compress highlights more in your curve. This will lower cloud contrast and desaturate the colors, so the "cast" (which I think is a natural sky variation) will be less visible and the clouds will also look more uniform.
(For readers unfamiliar with this manual curve -- this is a special step required by Capture One only. It may be possible to extract it from the LUT and by that copy the stock profile's curve, say by probing the neutral axis, but so far one have to design it manually and I don't think it's that bad as curve design is a matter of personal taste too, the stock curve is not necessarily the best.)
It's there in the prelim profile too (no additional curve) and you're right more highlight compression certainly helps to fade it. However the effect is visible in darker areas such as the sky too. I could try to even out the tones there I suppose.
-
I have now released 0.9.12, with improved smoothing of out of gamut ICC LUT entries, which had some issues before when a curve was used. The jaggies in the LUT curve in Fredric's message a some time back was probably such an effect.
In normal use there should be no difference, but if you have ICC LUT profiles I still recommend to re-render them using 0.9.12
I also added the possibility to control patch set merging in make-target with include/exclude text files (a feature request I got)
-
0.9.12 build for Windows ( mingw = dcamprof.exe + libgomp_64-1.dll + both manual & tutorial / = copies of Torger's web pages / in 3 formats : IE archive .mht, Mozilla archive .maff and regular .pdf ) : https://app.box.com/DCamProf (https://app.box.com/DCamProf)
-
v0.9.12 build for Mac OSX
https://app.box.com/s/p1bkqitlglx2o3xaex3chd2xa46ye8yn
-
In normal use there should be no difference, but if you have ICC LUT profiles I still recommend to re-render them using 0.9.12
To clarify, you just need to re-run the last make-icc command of your workflow.
-
I just built my "standard" profile from the dummies thread again with 0.9.12. While it is pretty good there are definitely still issues with blue hues. I see it in several pictures and just uploaded an illustrative example here: dcamprof_blue (https://www.dropbox.com/sh/di9fwoy8cjz2qn7/AAClWuBcGATt1hG0WEmVv9u1a?dl=0). In there is the raw, OOC jpeg, standard C1 output and one rendered with C1 and dcamprof 0.9.12. I used the preliminary profile without any additional curve.
Note the different blue hues. The one from dcamprof is pretty much off whereas the other three renderings are fairly consistent. Everything in the blue/violet/magenta range is affected. I am aware the example is not really good at judging what is correct but I don't have something more appropriate at the moment. I hope this is enough to guide you. If not I can certainly find other examples.
-
I just built my "standard" profile from the dummies thread again with 0.9.12. While it is pretty good there are definitely still issues with blue hues. I see it in several pictures and just uploaded an illustrative example here: dcamprof_blue (https://www.dropbox.com/sh/di9fwoy8cjz2qn7/AAClWuBcGATt1hG0WEmVv9u1a?dl=0). In there is the raw, OOC jpeg, standard C1 output and one rendered with C1 and dcamprof 0.9.12. I used the preliminary profile without any additional curve.
Note the different blue hues. The one from dcamprof is pretty much off whereas the other three renderings are fairly consistent. Everything in the blue/violet/magenta range is affected. I am aware the example is not really good at judging what is correct but I don't have something more appropriate at the moment. I hope this is enough to guide you. If not I can certainly find other examples.
while there certainly might be bugs with DCamProf code - how good was your input ? you can find a lot of anecdotal cases of people shooting targets with dress throwing some color reflections on target or @ wrong illumination and/or camera angles to target or whatever... if you want to make the case I 'd suggest post the original source (raw of the target used to build a profile) too ... that's again not to say that you made an error, but if you expect the author to help the more details the better chances... my $0.02
-
I just built my "standard" profile from the dummies thread again with 0.9.12. While it is pretty good there are definitely still issues with blue hues. I see it in several pictures and just uploaded an illustrative example here: dcamprof_blue (https://www.dropbox.com/sh/di9fwoy8cjz2qn7/AAClWuBcGATt1hG0WEmVv9u1a?dl=0). In there is the raw, OOC jpeg, standard C1 output and one rendered with C1 and dcamprof 0.9.12. I used the preliminary profile without any additional curve.
Note the different blue hues. The one from dcamprof is pretty much off whereas the other three renderings are fairly consistent. Everything in the blue/violet/magenta range is affected. I am aware the example is not really good at judging what is correct but I don't have something more appropriate at the moment. I hope this is enough to guide you. If not I can certainly find other examples.
To be able to help I need the data to be able to make the profile from scratch.
If there's a color bug it's more likely to be related to the tone reproduction. If there's a color error in the profile without any curve, I'd say it's more likely some error in the profile making process (misdetected color values etc) rather than a bug.
I've downloaded the files and looked at them. Unfortunately I'm at my super-old laptop with the worst imaginable TN panel so I don't see colors that well. But I'd say what may be strange is a woman with back towards the camera with a blue jacket which has some red component in DCamProf and green in C1, and balanced in OOC. The pink bags are clipped badly, but that's probably normal, you could try the new gamut mapping feature if you want to compress high saturation colors.
C1 and many other profiles typically render deep blues much too light (see bottom right CC24 patch), probably as a form of gamut compression, so a large lightness difference is usually seen. C1 also gamut compresses more than most, seen on the pink bags which are shaded in white. With gamut mapping feature you can get the same look.
I have noted that deep blues are quite easily pulled into out-of-gamut positions. If something bad is happening there it could be that some colors are pulled out of gamut where there is estimation and smoothing going on. I'd test the same profile as DCP too. But I have much too little data to go on, I need all data and commands for generating the profile.
-
I realize you (hk1020) has posted before with some workflow, profile from SSF right? Anyway it's very helpful to post all data each time or a link to an old message where the workflow is, I get quite a lot of "support requests" not only visible here also in private messages I can't really keep track of it all.
EDIT: found some old directory with stuff. I can have a look (probably) tomorrow to see if I can find some problem. There's a risk that it will end with that you need to find a real deep blue object, shoot it under a fixed light, and make a render and compare so you have a real reference to see which rendering that's most accurate. What I do know is that C1 probably is quite off, it usually is especially for saturated colors. OOC probably gamut maps too. I can agree though that there's more than expected going on, so I'll have a look.
-
As you're making with SSF, try using a much larger target, like this for example:
dcamprof make-target -c nex5.json -p cc24 -p munsell -p adobergb-grid target.ti3
I don't think there is a bug going on, but a CC24 profile will have limited precision in high saturation colors, and that coupled with unrelaxed LUT can pull high saturation colors into poor directions. The pink plastic bags seems to suffer from that, get too dark and too saturated. With a larger target and/or relax this is fixed.
The difference in dark blues I think can be higher accuracy in DCamProf. I'd suggest to make a fixed reference comparison.
It is a problem today that if you make a LUT of a CC24 without relax it the spline effect may continue far outside (towards higher saturation colors) and may mess up a bit. I may look into that, but all profiles should have some relax anyway.
-
Thanks for looking into this. Unfortunately, I don't have time at the moment for a more detailed reply. Will do soon.
-
Looked at it a bit more, and I can see that with the nex5.json SSF the dark blue patch on the CC24 gets a bit more red in it than it should. Making a profile from a CC24 shot you don't get as much red in there, then it's very closely to Adobe's matrix result, and looking at a real CC24 it matches other cameras well too. There might still be a tiny bit too much red in there though, which I think can be that the camera filter response has a tendency towards pulling deep blues in that direction.
So what does this mean? I think the SSF curves you've got has some issues in the blue range. I think I've seen this with other SSF curves for my 5Dmk2 for example, that is you see that it's not as accurate on the deep blue patch of a CC24, and maybe this is due to some limitations in measurement when measuring short wavelengths? Or simply that there may actually be some difference between Nex5 and Nex6 so the SSFs is not actually matching? SSF may seem scentific and all, but a badly measured SSF curve set is certainly not better than a good measured CC24 target.
It may be possible to manually tune the SSF curves to make it match the blue better, but then you're on your own :). Personally I'd play a little bit with that, at least to start with. Otherwise you can base your profile on a SSF target, or make an ugly thing in this case when Adobe seems to have a colorimetric matrix -- steal their matrix :). You don't get a LUT correction but this sensor matches very well with just a matrix (see DxOMark's metamerism index, 85 for this). Stealing the matrix makes an almost exact match with your own CC24 target, but the Adobe matrix seems to be designed with some larger target as it's more stable on extreme saturation, like the pink bags.
But still it seems like the camera has some tendency towards getting a little more red into the deep blues, probably it would be nice to have a target with more deep blues than the CC24 has. It seems like Adobe's NEX5 profile is designed with a colorimetric base matrix, and then their look table reduces red component a bit (manual tuning?).
Conclusions:
- The SSFs in nex5.json has an issue in the blue range, and cannot be used for good precision there.
- Not using any relaxation/weighting at all is not recommended, as you can worsen things for colors with higher chroma than the target
- With a CC24-based profile you get less problems with the blue, but possibly there is still some small amount. This is probably hardware related and you
need a target with more deep blue and purple patches to get better precision
- The issues is not due to any bug, but specific to these SSFs and this camera. That is no software changes required.
Note that C1 is known for adding in a good amount of yellow, making saturated reds orange (see the H&M text top of image), and most likely that deep blue is affected too, so I wouldn't use that as a reference.
Solution(s):
- Use traditional CC24 target, add weighting/relax, residual problems are then small
- Use a more saturated target like an IT-8 (preferably combine with CC24), better quality
- Try to manually correct the SSF curves for better deep blue matching
- Manually correct the blue range with look operators.
- Steal matrix from Adobe Standard profile (by eye the base matrix seems to have good colorimetric properties), this is the easiest and most fool-proof in
this case. Note that Adobe's profiles are designed differently profile-to-profile, the base matrix is far from always a colorimetric one like here.
Note that if you copy the Adobe matrix the look of the profile will not at all be like Adobe's profile, as the tone reproduction is totally different. I have attached a native profile with the Adobe D65 matrix you can play with, just run your make-icc commands on that.
-
So what does this mean? I think the SSF curves you've got has some issues in the blue range. I think I've seen this with other SSF curves for my 5Dmk2 for example, that is you see that it's not as accurate on the deep blue patch of a CC24, and maybe this is due to some limitations in measurement when measuring short wavelengths? Or simply that there may actually be some difference between Nex5 and Nex6 so the SSFs is not actually matching? SSF may seem scentific and all, but a badly measured SSF curve set is certainly not better than a good measured CC24 target.
reading the other forum about building a monochromator setup it is clear that it is not that easy to measure SSF directly... so quite possible that source of that SSF (from RIT) didn't make a precise work in that part... somebody (Iliar Borg is a natural suspect) who did measure SSFs a lot can comment on RIT's SSFs - whether their blue curves are good or not
-
Maybe it's the red curve that's bad that causes too much red when blue is weak (a deep blue color means little green and red, but blue is also weak, so it's quite easy to throw off balance). I haven't really played much with tweaking SSF curves so I cannot provide a precise analysis on this particular color issue.
-
Maybe it's the red curve that's bad that causes too much red when blue is weak
indeed... in any case somebody needs (as in it will be nice) to comment about SSFs from RIT vs some other measurments...
-
Anders,
First, thank you for your efforts! It's quite liberating to separate one's desired "look" from a particular hardware/software choice.
I successfully followed the easy method using the imaging resource target RAW to make a C1 profile for the A7II. The resulting profile looks very good. I'd like to do the same for Lightroom. When I tried it, though, this happened:
/Applications/dcamprof make-dcp -n "SONY ILCE-7M2" -d "My Profile" -t acr my-profile.json my-profile.dcp
Generating HueSatMap with 90x30 = 2700 entries...done!
9 (0.333%) of the HueSatMap entries have hue shift discontinuity with neighbors.
Hue shift discontinuity between LUT entry neighbors are not allowed. Aborting.
Are there parameters earlier in the process I should relax? Any help much appreciated!
Best,
Matt
-
First a comment on hk1020 test picture, while the blue stuff is no bug, the pink plastic shopping bags in the picture shows that I need to rework the recent gamut mapping algorithm... that will be my next priority.
-
I successfully followed the easy method using the imaging resource target RAW to make a C1 profile for the A7II. The resulting profile looks very good. I'd like to do the same for Lightroom. When I tried it, though, this happened:
/Applications/dcamprof make-dcp -n "SONY ILCE-7M2" -d "My Profile" -t acr my-profile.json my-profile.dcp
Generating HueSatMap with 90x30 = 2700 entries...done!
9 (0.333%) of the HueSatMap entries have hue shift discontinuity with neighbors.
Hue shift discontinuity between LUT entry neighbors are not allowed. Aborting.
Are there parameters earlier in the process I should relax? Any help much appreciated!
I've actually never succeeding in triggering hue shift discontinuity without using look operators, so this is a first :). What this typically means is that there's some strong stretching going on in the LUT, and that's not good. Could it be that there's some problem with the target shot or patch matching or something? Large Delta E values on the matrix matches (say larger than 4 or so) when running make-profile is suspicious.
There's also a possibility that there's some bug that causes hue discontinuity being triggered despite there are no strong LUT stretches, in that case I would need all your data so I can re-run the workflow and trigger the problem myself.
I made a quick-and-dirty A7r-II profile earlier today, posted in this thread:
http://forum.luminous-landscape.com/index.php?topic=104195.0
-
I thought I'd found my mistake, as I was pointed to a wrong file, but pointing to the right file didn't completely fix the problem:
5 (0.185%) of the HueSatMap entries have hue shift discontinuity with neighbors.
Hue shift discontinuity between LUT entry neighbors are not allowed. Aborting.
Ah well. I still have the A7II, not the A7rII, but your profile may be enough incentive to upgrade ;D
--Matt
-
I thought I'd found my mistake, as I was pointed to a wrong file, but pointing to the right file didn't completely fix the problem:
5 (0.185%) of the HueSatMap entries have hue shift discontinuity with neighbors.
Hue shift discontinuity between LUT entry neighbors are not allowed. Aborting.
Ah well. I still have the A7II, not the A7rII, but your profile may be enough incentive to upgrade ;D
If you can share the data in a dropbox or something I can have a look. It seems pretty strange that a normal profile would trigger hue shift discontinuity.
You can force make-dcp to output a profile though by adding the -H parameter.
-
Perhaps my error was using the linearized cc24.tif exported from C1 in the earlier workflow. I used Raw Therapee instead and have gotten further.
Alas - the resulting profile turns everything a bright green.
Here are dropbox links to the linear targets saved from C1 and Raw Therapee. They are clearly different. I am completely new to RT, so have no confidence in that sample. I clicked the "output TIFF for calibration" button.
https://www.dropbox.com/s/evvtmteueabeopn/C1cc24.tif?dl=0
https://www.dropbox.com/s/qnd4mvr7nleieob/RTcc24.tif?dl=0
As an aside, I had a few days access to a Leica S (007) and compared several dozen captures with the A7II and Canon 1DsII. The Leica and Canon colors were fairly close in LightRoom. The Sony quite yellow/green in comparison, but the use of some freely available profiles moved the Sony colors much closer to the other two. This, more than any amount of online discussion, has convinced me that it IS the profile that makes MOST of the difference between camera looks, optics and sensors being otherwise up to the task. (To be more precise, Leica S(007) with APO 120/2.5, Sony A7II with Minolta 100/2, Canon 1DsII with 24-105/4L , this was for color, not corner sharpness).
Best,
Matt
-
Just released version 0.9.13, fixed a clipping bug (=ugly sunsets) and improved/changed gamut compression algorithm.
-
Perhaps my error was using the linearized cc24.tif exported from C1 in the earlier workflow. I used Raw Therapee instead and have gotten further.
Alas - the resulting profile turns everything a bright green.
Here are dropbox links to the linear targets saved from C1 and Raw Therapee. They are clearly different. I am completely new to RT, so have no confidence in that sample. I clicked the "output TIFF for calibration" button.
https://www.dropbox.com/s/evvtmteueabeopn/C1cc24.tif?dl=0
https://www.dropbox.com/s/qnd4mvr7nleieob/RTcc24.tif?dl=0
As an aside, I had a few days access to a Leica S (007) and compared several dozen captures with the A7II and Canon 1DsII. The Leica and Canon colors were fairly close in LightRoom. The Sony quite yellow/green in comparison, but the use of some freely available profiles moved the Sony colors much closer to the other two. This, more than any amount of online discussion, has convinced me that it IS the profile that makes MOST of the difference between camera looks, optics and sensors being otherwise up to the task. (To be more precise, Leica S(007) with APO 120/2.5, Sony A7II with Minolta 100/2, Canon 1DsII with 24-105/4L , this was for color, not corner sharpness).
Yes I agree that profile is the main thing. There's too few photographers that have played around with profiles to discover this though, so the camera color myth lives on...
Anyway, if you are making a DCP you need to export linearly without white balance so you get a green-tinted dark file. RT can do this, there's a little checkbox when you export "Apply white balance", make sure it's unchecked. If you make a DCP with white-balanced shot you will get a green tint I think.
If you make an ICC profile the white balance does not matter, however if you make a profile for Capture One you need to export from Capture One, and since it sometimes does funky pre-processing it's not guaranteed that a profile made using Capture One exported file will work in any other raw converter.
An easy capture one workflow is described here:
http://www.ludd.ltu.se/~torger/photography/camera-profiling.html#the_easy_way_c1
An easy DNG workflow is described here:
http://www.ludd.ltu.se/~torger/photography/camera-profiling.html#the_easy_way
If you export a file from RT without white balance and make a DNG profile according to the above workflow it should work, and if you export from Capture One to make a Capture One ICC you should follow the Capture One workflow. The Capture One tiff file you provided looks fine.
-
0.9.13 build for Windows ( mingw = dcamprof.exe + libgomp_64-1.dll + both manual & tutorial / = copies of Torger's web pages / in 3 formats : IE archive .mht, Mozilla archive .maff and regular .pdf ) : https://app.box.com/DCamProf (https://app.box.com/DCamProf)
-
Yes, the C1 profile is quite nice! I'll keep working on the DCP. Thanks again!
Matt
Edit: Success! Wonderful!
-
>it IS the profile that makes MOST of the difference between camera looks
There are some exceptions, if we discuss overall "looks" and go a little beyond "most". When the CFA is designed for low noise on a noisy sensor, it is often that, being too transparent, it produces not enough separation in certain ranges of colours, mostly green and yellow. You can easily see it reversing the colour transform. If the raw values from a set of RGB samples have the difference within the photon shot noise limits, those colours will not be resolved. Such CFAs are responsible for mashy greens and plastic skin, and also for wrong looking reds on firefighters' engines or problematic shifts between blue and purple.
-
>so quite possible that source of that SSF (from RIT) didn't make a precise work in that part
Don't take those too seriously, it is more of a proof of concept than lab-grade results. Usual academic practice is that you need to show first you are on the right track before nice things happen, grants granted, etc.
-
reading the other forum about building a monochromator setup it is clear that it is not that easy to measure SSF directly
Erm - it is actually quite easy. My first SSF from SLR/n were quite good and resulted in a good quality profile. Where it gets difficult is with the cameras that tend to heat up (older ones like Kodak ProBacks) and a faster exposures are needed. This is what I am working on to improve right now (and also to make it really easy to process results with Hamamatsu micro spectrometer).
In general I found it a lot easier than to get good profiles from CC targets.
-
> My first SSF from SLR/n were quite good and resulted in a good quality profile
In regards to http://www.cis.rit.edu/~dxl5849/projects/camspec/ - single-port sphere resulting in wrong / unmatched measurement geometry between the camera and spectrometer makes things problematic. Also, the SSFs are given for the particular raw extraction / conversion process (not necessarily without it's own quirks) and may be applicable for the same conversion procedure but not for a different one. There are many other unknowns and omissions in the description of the "ground truth" lab work there.
One of the most important things is that the response needs to be measured with the data numbers in raw being much higher than all noise levels. Each time SSF looks spiky this is the first thing to check. Specific damage to results is in the areas when channel curves overlap.
-
Iliah,
There are some exceptions, if we discuss overall "looks" and go a little beyond "most". When the CFA is designed for low noise on a noisy sensor...
With regard to the low noise level of current state of the art sensors in consumer cameras, are the "exceptions" to which you refer...exceptionally rare?
-
I'm also interested in what you (Iliah) know about the CFA. I'm quite new to this and have mostly looked at fairly modern cameras of 36x24mm and MF size, and they all seem to have such good color separation that I find it hard to believe that "plasticky" skin and other color issues would be a hardware-related thing today.
However I've seen results from really old cameras, and mobile phones and other tiny sensor cameras where color separation indeed is a hardware-related limitation. Shooting at high ISO you get noise issues too of course.
I haven't really made any thorough experiments to make a solid verification but my current assumption is that two reasonably modern cameras with reasonably large sensor in good light can be profiled to look almost exactly the same, but when you then change light from the profiled light the colors will modulate differently so the further away from the calibration illuminant you are the more the cameras will differ. I suspect that the difference will not be that big even then though as CFAs often seem to be quite similar between brands/models these days.
-
v0.9.13 for Mac OSX
I've created a folder on box.com to hold the current and past Mac binaries so I dont have to constantly provide URL's.
Pls give it a spin and let me know how well it works...........
https://app.box.com/s/tcnv5km0sh997ueqez7d3zqcqlh456nu
-
Thank you - so far, so good.
kirk
v0.9.13 for Mac OSX
I've created a folder on box.com to hold the current and past Mac binaries so I dont have to constantly provide URL's.
Pls give it a spin and let me know how well it works...........
https://app.box.com/s/tcnv5km0sh997ueqez7d3zqcqlh456nu
-
Iliah,
With regard to the low noise level of current state of the art sensors in consumer cameras, are the "exceptions" to which you refer...exceptionally rare?
It is totally wrong to speak of "brand colours" (Canon colours, SONY colours,...), I always maintained that. However noisy sensors are not that extinct, not to mention lots of people still shoot with the older cameras.
-
It is totally wrong to speak of "brand colours" (Canon colours, SONY colours,...)
_but_ even with CFA SSF/CMF designs you can see somewhat constant distinction between how they look between Canons and Nikons, no ?
-
I've got a good test image with some deep saturated artificial blue lights which show some rendering artifacts also with the latest v0.9.13. So you know it's a known bug. It will probably take a while to fix, the deep blue range has proven to be especially difficult as camera+profile often push artificial blues into undefined space, so you can't just normally gamut compress or something as you have no defined color to work on.
-
> two reasonably modern cameras with reasonably large sensor in good light can be profiled to look almost exactly the same
Two reasonably low noise cameras, yes; if they are designed without CFA shortcuts. Point is, when somebody is speaking of "colour", colour per se is very flexible for all the cameras, notwithstanding the old ones. It is known with scanners, too. But the richness of gradation is a different matter. One of the good tests is shooting plums, like this: http://img01.quesabesde.com/media/img/noti/0092/nikon_j5_dsc_0084.nef
The colour separation is measured close to full well. Even with a monochromator it is often necessary to take 2 shots with different exposure, so that the data for the weak channels is not dug in noise. For a target chart, it may be 3 to 6 shots, depending on the chart design. CC DC takes 5.
-
_but_ even with CFA SSF/CMF designs you can see somewhat constant distinction between how they look between Canons and Nikons, no ?
Mostly, cameramakers change the CFA design very often, sometimes even when using essentially the same sensor but with a different AFE (analogue front end), or to improve over the previous. So, that is not the part of the "brand look". However, the calculation formulas in use are different between companies, but that does not translate into the look of the final photo. One can see those differences in approacesc only if analyzing raw. End result depends on the colour transform from raw, close to 100%.
-
so that the data for the weak channels is not dug in noise.
so how close to saturation in raw (blue channel) you move for a example a "blue" patch under a tungsten light ? within 2 stops ? 3 stops ?
-
I think the source of the blue problem is that to get dark enough blue on screen you need to subtract raw blue, that is the luminance column in the forward matrix gets a negative blue value.
Adobe doesn't seem to put negative values there, but instead let blue render much lighter than realistic, and in return get no negative colors in extreme ranges, ie a more robust profile. Phase One often do this as well, although I do see extreme blue clip to black sometimes if I remember correctly.
I shall investigate this matrix design method, that is avoid negative blue value in the luminance/Y column, it could be a problem solver. The question is if this can be done without sacrificing blue accuracy in the normal range... the LUT can in theory compensate, but perhaps not without too aggressive bends.
-
so how close to saturation in raw (blue channel) you move for a example a "blue" patch under a tungsten light ? within 2 stops ? 3 stops ?
3 stops down from saturation is midtones, that would be the lowest I go.
-
3 stops down from saturation is midtones, that would be the lowest I go.
and rawdigger still can be used to flat field (we do not naturally clip anything in flat field raw) the target shot with some patches clipped in order to extract the data for that "blue" one so that we can replace (after normalization for the differences in exposure between 2 sets) the data in the shot where "blue" patch is "underexposed", right ? just want to check if no hidden issues w/ such merge of shots and flat fielding.
-
and rawdigger still can be used to flat field (we do not naturally clip anything in flat field raw) the target shot with some patches clipped in order to extract the data for that "blue" one so that we can replace (after normalization for the differences in exposure between 2 sets) the data in the shot where "blue" patch is "underexposed", right ? just want to check if no hidden issues w/ such merge of shots and flat fielding.
Yes, straightforward.
-
I think the source of the blue problem
Oh well, I think I was full of sh*t... forget that about negative matrix values. But anyway, I'm on to it.
-
Iliah,
...my current assumption is that two reasonably modern cameras with reasonably large sensor in good light can be profiled to look almost exactly the same...
Two reasonably low noise cameras, yes; if they are designed without CFA shortcuts...But the richness of gradation is a different matter.
Which component(s) of camera hardware contributes to the richness of gradation?
-
As you're making with SSF, try using a much larger target, like this for example:
dcamprof make-target -c nex5.json -p cc24 -p munsell -p adobergb-grid target.ti3
I don't think there is a bug going on, but a CC24 profile will have limited precision in high saturation colors, and that coupled with unrelaxed LUT can pull high saturation colors into poor directions. The pink plastic bags seems to suffer from that, get too dark and too saturated. With a larger target and/or relax this is fixed.
The difference in dark blues I think can be higher accuracy in DCamProf. I'd suggest to make a fixed reference comparison.
It is a problem today that if you make a LUT of a CC24 without relax it the spline effect may continue far outside (towards higher saturation colors) and may mess up a bit. I may look into that, but all profiles should have some relax anyway.
Though this discussion has progressed a lot by now I'd like to add a little more to my problem. Sorry I didn't properly refer to my posts in the dummy thread in the first place. Actually, I should have stayed there as this discussion is now way over my head. Anyway, I also made a profile from a target, namely the one on imaging-resource..com (http://www.imaging-resource.com/PRODS/sony-nex-6/NEX6hVFAI00100.ARW.HTM) you cited in the dummies thread. There is almost no difference to using the ssf. It is marginally better, though, but I wouldn't bother.
Trying the munsell thing however, did have a visible effect. Its blues are better but still not right. I have some pictures with several very sensitive blues which in reality are very similar but always come up as quite different blues for the individual hues. The separation is way too large, the blues are almost always too dark and some might turn almost purple. Unfortunately, I don't like to post the pictures publicly but I would give Anders a link through PM if that would be acceptable.
-
> Which component(s) of camera hardware contributes to the richness of gradation?
Proper separation in CFA (colour filter array, the thing that forms Bayer mosaic), CDS (correlated double sampling, https://en.wikipedia.org/wiki/Correlated_double_sampling), overall noise (read, thermal, reset, etc). Interesting that proper CFA and low noise are somewhat cross-purpose. Some design issues, too: if during a 30-sec exposure with the lens cap on you can see a "cloud" in raw - it is improper heat dissipation, it hurts. Generally, all uniformity issues cause strange artifacts if one starts to employ 10+ stops of dynamic range.
-
Adobe doesn't seem to put negative values there, but instead let blue render much lighter than realistic, and in return get no negative colors in extreme ranges, ie a more robust profile.
It may not be deliberate on Adobe's part, but rendering highly saturated colors as lighter, in-gamut colors is desirable if you model the Helmholtz-Kohlrausch effect (https://en.wikipedia.org/wiki/Helmholtz%E2%80%93Kohlrausch_effect).
-
It may not be deliberate on Adobe's part, but rendering highly saturated colors as lighter, in-gamut colors is desirable if you model the Helmholtz-Kohlrausch effect (https://en.wikipedia.org/wiki/Helmholtz%E2%80%93Kohlrausch_effect).
Some of their newer profiles have only positive values in their matrices and produce a desaturated light look, not very realistic or accurate, but it has some advantages. With DNG you first make a matrix conversion to get into linear prophoto, clip if necessary, and then make a lookup in the LUT.
With a "normal" matrix optimized to match normal saturation colors you will get negative value and clipping etc in the extreme range, and then the matrix value is first clipped and the put into the LUT.
So I think their matrices in this case is a purely mathematical thing. Their blues also in the normal range are rendered much too light, but it's a part of the "Adobe look".
It may turn out that I will eventually be forced to make these type of "dumb" matrices that has nothing to do with color, but just as an intermediate step to get into the LUT. I'm not yet sure it's required though. I would very much prefer to have a "layered" DNG profile as before, ie run just the matrix and get as good as possible linear colorimetric color, further improve that with the HueSatMap LUT, and then run the LookTable and tone curve to get the tone reproduction operator. In RawTherapee you can toggle all these individual elements which make the profile more versatile.
-
Though this discussion has progressed a lot by now I'd like to add a little more to my problem. Sorry I didn't properly refer to my posts in the dummy thread in the first place. Actually, I should have stayed there as this discussion is now way over my head. Anyway, I also made a profile from a target, namely the one on imaging-resource..com (http://www.imaging-resource.com/PRODS/sony-nex-6/NEX6hVFAI00100.ARW.HTM) you cited in the dummies thread. There is almost no difference to using the ssf. It is marginally better, though, but I wouldn't bother.
Trying the munsell thing however, did have a visible effect. Its blues are better but still not right. I have some pictures with several very sensitive blues which in reality are very similar but always come up as quite different blues for the individual hues. The separation is way too large, the blues are almost always too dark and some might turn almost purple. Unfortunately, I don't like to post the pictures publicly but I would give Anders a link through PM if that would be acceptable.
A PM would be nice. I'm a bit choked now so progress is probably going to be slow. Have you tried X-Rite's software or Adobe's DNG profile editor? If you can it would be nice to hear what result you get from that. It would be very useful to get some indication if it's a camera problem (ie you get blue issues from the other softwares too), or it's a problem with DCamProf's blues in some way. Please include those profiles in the PM too if you can. If it's a camera problem it's still an interesting case as I then can test the feasibility to manually adjust the profile, or add some feature that makes it easier to do so.
Please note that you can't really use the bundled profiles as reference, they are generally majorly wrong. So the image you send should have colors you know what they really are, preferably of objects you have near you and can look at again as I'm probably going to ask about their colors.
(I have seen on the A7r-ii and artificial blues that there might be some purple tainting there too, but it could be unrelated (I'm working on a DNG LUT clipping fix there).)
Did you test the file I posted based on Adobe's matrix? Did it work better? If not here's the message where it's attached:
http://forum.luminous-landscape.com/index.php?topic=100015.msg856054#msg856054
-
A PM would be nice. I'm a bit choked now so progress is probably going to be slow.
Don't worry, you are much faster than what I could come up with. I'll prepare something, maybe tonight. If not then maybe Saturday.
-
I haven't tried dcamprof yet but would it be possible to keep a bunch of reference images from common cameras and whenever a new release is out run a script that generates a new set of profiles. Then people could try the new version quickly. You would only need linear and "torger look" versions.
-
I haven't tried dcamprof yet but would it be possible to keep a bunch of reference images from common cameras and whenever a new release is out run a script that generates a new set of profiles. Then people could try the new version quickly. You would only need linear and "torger look" versions.
Good idea. I'll probably do something like that.
It will be quite boring though, because in most circumstances the new version will look exactly like the old :) the gamut compression stuff has changed a bit though, so it's still a good idea.
What seems to be a quite big chunk of work left is extreme colors and clipping handling. I've already spent a lot of time on that, but it's difficult to get right, there's still issues. There will probably be a few incremental releases before all those things are sorted out. Clipping issues are only visible in certain type of images, night scapes with lots of colored artificial lights is something DCamProf currently doesn't handle that well.
-
This blue clipping issue is driving me nuts.
I've attached a pixel peep screen shot of a corner detail of a night scape shot of an artificial narrow band light, made with an A7r-II. It's one of the shots I've got in a PM, I took the freedom of posting this size crop without asking, hope it's okay. I normally always ask, but this size from a high MP shot I thought was ok.
The first is without profile (eg identity matrix after WB) that shows the actual tonality in the file. Then to the right is adobe's (before looktable+curve) which is quite close to the identity matrix, that is tonality is mostly kept (they mess up a bit when adding looktable+curve but not too bad).
And in the middle is DCamProf (before looktable+curve). Here we see it's much bluer, so blue in fact that much of the tonality is lost, and a sharp transition into cyan. Why is this? When the matrix is optimized to match the CC24 a very good match can be had, but that requires a great deal of blue subtraction, otherwise the dark blue patch becomes much to light, and other colors as well. In theory the LUT could compensate that, but that leads to strong bends, and when relaxing you end up with quite large errors.
This can be seen with Adobe's profile, which doesn't match CC24 colors that well, and thus is not a very accurate profile. But in return very robust with extreme colors.
It seems like some subtraction is okay, say if the Yb value is -0.10 or larger you can still have good tonality (which it is on my Canon for example), but for the A7r-ii and some other Sony sensors it's like -0.28, and then blue is pushed down to the ground in the extreme range. I don't know why these sensors have this type of blue sensitivity, someone that dare to speculate?
By limiting the matrix optimizer to not have (large) negative blue value I can get about the same result as Adobe, but then accuracy in the normal range is lost (can't compensate with LUT without too strong bending) and I really don't like the idea to mess up the normal case to be robust in an extreme case.
I'm thinking of the idea to blend between two profiles/matrices, one for the normal range and gradually blending into another for the extreme range. It's a quite big amount of work to test if that even works though, so I'm probably going to add an "known limitations" section and put this issue there.
However, I still have some messy tone operator clipping issues in this range too, that needs separate fixing. So it's certainly going to be better than the current 0.9.13 when finished, but I think Adobe is going to win on "extreme blues" even after completion.
-
So it's certainly going to be better than the current 0.9.13 when finished, but I think Adobe is going to win on "extreme blues" even after completion.
a simple idea will be to provide some command line option (along with a writeup in manual with couple of illustrations) to build profiles just for those extreme cases... I think it is not reasonable to expect one profile to cover everything... if somebody is after extreme colors - let them build profiles that sacrifice regular colors (doubtfully they will be there anyways or play any center role if so much attention is towards the extremes in the frame).
-
a simple idea will be to provide some command line option (along with a writeup in manual with couple of illustrations) to build profiles just for those extreme cases... I think it is not reasonable to expect one profile to cover everything... if somebody is after extreme colors - let them build profiles that sacrifice regular colors (doubtfully they will be there anyways or play any center role if so much attention is towards the extremes in the frame).
Yes I will probably provide the command line option to limit the "blue compensation". I've tested and it seems to work although weighting/relaxing becomes messier. So yes you could then make a "nightscape/narrow band light" specific robust profile which is more about keeping tonality in the extreme ranges than being accurate in normal range.
-
Before LUT in a DNG profile you do a matrix conversion to linear prophoto space using the forward matrix. This conversion is then clipped to 0-1 range before going into the LUT.
A matrix optimized to match normal colors well will cause say 10% of the total space to be lost in clipping. These 10% is what appears when you shoot strange narrow band lights. In order to make a DNG profile that does not render these clipped flat you cannot make a normal matrix, instead of optimizing it to match colors you need to optimize it to not clip, which is what Adobe is doing in many of their newer profiles. The LUT then (hopefully) moves colors into sane positions in the normal range, and stays fairly passive in the extreme range, so the native camera tonality is kept rather than clipped away.
(A pure matrix profile of course must be optimized to match normal colors, and that simply won't work for the extreme range. Tonality is irreversibly lost.)
DCamProf's native format also uses a matrix for base conversion, on which it adds a LUT, just like a DNG profile. This is problematic for the extreme range. Although DCamProf doesn't clip like in the DNG pipeline, still when the matrix enters negative lightness it cannot really compensate without an extreme bend. This makes the format sort of flawed for making profiles that are robust in the extreme range. However, I don't think a major rewrite is necessary just yet, I'll play around with extended optimization techniques.
Controlling the blue Y value like discussed previously is the major aspect of this, if I'm not mistaken it's all DCamProf's native format needs (since then you can make sure lightness never gets negative, negative X and Z are okay since DCamProf makes addition on those axes), but it's not enough for a DNG profile as the matrix can still clip in the Prophoto conversion.
While working on this and other issues I've also discovered clipping issues in the CIECAM02 blue range which causes some headache as I'm using that here and there. LCMS2 CIECAM02 implementation clips uglier than another CIECAM02 implementation I've looked into, so maybe I'll switch, at least for some critical calculations.
Handling the "extreme range" is the largest can of worms I've opened lately. It's not just one place in the software that is affected, it's like 5-10 places and none of them has an easy fix.
-
Experiments are ongoing, very much "back to the drawing board" kind of thing. Looking a bit at alternate matrix optimizations.
For DNG profiles, one obvious method is to simply set forward matrix equal to ProPhotoRGB matrix, which means that what the LUT sees is the "raw" samples (just white balanced), and then gets to stretch that into place. This is quite robust for the extreme range as you don't get any clipping of data before entering the LUT, but to not clip when inside the LUT it must make sure that it reduces scaling towards 1.0 for the gamut border. You will also get some twists in the LUT which may be problematic.
Adobe has changed their profile design technique over the years, and probably vary between models. Looking at Adobe's own A7r-II and Pentax 645z profiles both seem to be designed with the same method. The forward matrix in their case seems to be a remapped AdobeRGB matrix, which fits inside ProPhotoRGB border, and also the spectral locus border. My guess is that the size of the gamut triangle is pre-defined (to something similar to AdobeRGB), then their matrix optimizer rotates it to match hues as well has possible, ignoring saturation. The matrix result is thus a heavily desaturated look, but with good hue match. Then their LUT simply stretches saturation straight outwards from the whitepoint. The gain from doing this instead of just picking the ProPhotoRGB matrix as is, I guess is that you don't get much hue rotation in the LUT, which is a bit messy with DNG LUTs (hue discontinuity problem discussed some time back).
DCamProf is not a DNG-only software so I'm thinking about how to tackle this. For DNG profiles there are indeed many advantages of this type of optimization technique. As DCamProf currently has clipping issues in its pipeline as well, it's actually an advantage also for DCamProf. However, it's painful to ditch the layered elegance - FM matrix gives you as accurate color as possible with a matrix and the LUT just refines that. Since matrix optimization is fast I'm thinking of having two matrices in the native format, one "real" matrix stored for using when a matrix-only profile is generated, and one matrix optimized to be used together with a LUT.
It's quite likely that the next release will introduce some significant behavior change...
-
Before LUT in a DNG profile you do a matrix conversion to linear prophoto space using the forward matrix. This conversion is then clipped to 0-1 range before going into the LUT.
A matrix optimized to match normal colors well will cause say 10% of the total space to be lost in clipping. These 10% is what appears when you shoot strange narrow band lights. In order to make a DNG profile that does not render these clipped flat you cannot make a normal matrix, instead of optimizing it to match colors you need to optimize it to not clip, which is what Adobe is doing in many of their newer profiles.
All Kodak SLR cameras and Pro Backs used the same approach - the ERIMM (input referenced ProPhoto) as interconnecting space. The code also did some clipping and standard matrix (for ideal camera from Photodesk profiles) was mapping it to 0..1 range perfectly. However each camera came with its own calibrated set of matrices for that specific sensor which were almost always not fitting the 0..1 range in ERIMM space (those were pacjed into every raw file). What Kodak software did if those were used was to also bundle exposure correction factor - this was really used to bring the output within the range of 0..1. Just thought that might be useful.
-
> The forward matrix in their case seems to be a remapped AdobeRGB matrix
As some here know ;) I map to BetaRGB http://brucelindbloom.com/BetaRGB.html
-
While working on this and other issues I've also discovered clipping issues in the CIECAM02 blue range which causes some headache as I'm using that here and there.
CIECAM02 has significant practical issues with colors near to or beyond the spectrum locus, particularly near blue. I think that the basic nature of CIECAM02 makes any elegant solution impossible. (I get passable results with a number of hacks to it). A "nice" solution would involve re-formulating the equations used to accommodate extrapolation, and then re-fitting them to the original data CIECAM02 data sets.
Timo Kunkel and Erik Reinhard's simplified and improved model (http://www.cs.bris.ac.uk/publications/Papers/2001006.pdf) inspired by CIECAM02 might be a place to start.
-
CIECAM02 has significant practical issues with colors near to or beyond the spectrum locus, particularly near blue. I think that the basic nature of CIECAM02 makes any elegant solution impossible. (I get passable results with a number of hacks to it). A "nice" solution would involve re-formulating the equations used to accommodate extrapolation, and then re-fitting them to the original data CIECAM02 data sets.
Timo Kunkel and Erik Reinhard's simplified and improved model (http://www.cs.bris.ac.uk/publications/Papers/2001006.pdf) inspired by CIECAM02 might be a place to start.
Thanks, very interesting paper! At some point I'd like to revisit color modeling and standard observers etc. I think however that the problems I have today with extreme values can be solved with an "inelegant hack", blending over to some other type of approximation. One likely way I'll end up is that before CIECAM even starts getting used the gamut values are compressed into a range it understands.
Close to the locus I don't expect good accuracy anyway, and good accuracy and avoiding clipping are conflicting goals. Camera SSF and Observer CMF are different enough that a good translation can only be had in normal range colors. For extreme colors, as demonstrated by the artificial light some posts back, the matching problem becomes impossible. In that range it's more important to just output some color and have smooth gradients.
I'm battling with how to achieve that goal. I've almost completed a new matrix optimizer that fits the matrix withing ProPhotoRGB to avoid DNG pipeline clipping (that matrix has good hue match, but is strongly desaturated). That matrix will be used for DNG profiles with LUTs. It can be used as input to DCamProf's native LUT too, and what then happens is that the LUT will stretch like crazy to match chroma in the normal range colors, and then compress strongly towards the edge of the "gamut triangle". It achieves the goal of a good match in normal range colors while not clipping in the extreme range. However I think there's a better way to do it, which I'm investigating.
I'd like to use the "good match" matrix as input to DCamProf's native LUT, and somehow transition into an extreme value handling. I'm a bit worried about using a compressing matrix for DCamProf's native LUT as the LUT then has to stretch long distances within the normal range, and I'm not sure the special "LUT space (chromaticity + lightness)" is suited for that, ie inbetween colors might drift in hue. Having an input matrix that requires as low stretching as possible in the LUT seems more healthy, and it makes weighting/relaxing much easier to work with. If I change to a compressing matrix the behavior of weighting will change to a major extent and I'd like to avoid that if possible. Having to rework the tutorial docs etc would not be fun.
(DCamProf's native LUT does not need to work with the same input matrix as the DNG profile LUT, as it's only about encoding)
-
A note to those sending me PMs about the software; I'll likely be a bit slow replying for a while, I'd very much like to sort out this extreme value issue so I'll see where the software ends up first, and it's a challenging problem. I don't want to sacrifice performance in the normal range, then I rather have it work a bit worse than say Adobe in the extreme range, but it must become better than it is today.
-
Experiments are continuing, rejecting idea after idea.
There is a risk that the problem is "unsolvable". We can see that for example Adobe's profiles for A7r-II and Pentax 645z are unrealistically desaturated and have a very light blue. This makes it possible to render smooth gradients with ever-increasing intensity all the way to the extreme values.
If you instead render realistic saturation and blue in the normal range, you get a very difficult transition into the extreme values.
If the problem proves unsolvable there's very uncomfortable changes that needs to be made, namely that the base profile should not be colorimetric, that is you should not try to match the target, but instead be "inspired" by the target but let the camera hardware take the upper hand. Hues can still be matched, but saturation and lightness would be up to the hardware. I don't want to end up there, as then the "look" will be quite random and hardware-specific.
I'm not out of ideas yet though, more things to try...
-
To structure the thoughs a bit; here's the current challenges:
- DCamProf builds on the idea that you have an accurate colorimetric base profile, which you apply tone reproduction (tone curve) and gamut compression on top
- Problem is that colorimetric base profile is only well-defined within the target range (typically CC24) and some distance outside that, for extreme saturation it's not well defined and will typically clip. You can recover clipped data when gamut compression is applied afterwards.
- Some of the recent Sony sensor models like those in 645z and A7r-II have a high sensitivity blue which accentuates this effect
- The obvious solution to pre-compress the gamut already in the colorimetric stage to avoid clipping is hard to make good looking, especially for the Sonys as you need to compress a lot in a short range, or do like Adobe and desaturate the whole profile (ie sacrifice color realism also in the normal range)
- A related but different issue is that maybe accurate deep blues (well-represented by the deep blue CC24 patch) is not what people want, as better tonality is had if you lighten it. Currently DCamProf has no controls for this type of strong subjective adjustment already in the colorimetric stage, it may be a huge effort to implement support for this.
- Clipping issues in the tone reproduction operator by pushing CIECAM02 too hard, but some of it may be solved by itself if some sort of pre-compression is made, so I've saved those issues to later
I'm not particular fond of the idea and do like Adobe, always prioritize robustness over nice colors as robustness is more commercially important for the mass market, so if the problem turns out "unsolvable", I'll clip "nicely" as a first stage (current 0.9.13 clips ugly sometimes with color shifts, sometimes to black), and in the longer term make it possible to make a desaturated profile particularly targeted at rendering extreme range colors without clipping. Interesting enough it's probably only certain cameras that are going to benefit from that, such as the Sonys, while I think my Canon and Hassy would be fine without as it doesn't have that kind of blue sensitivity.
I'm perfectly aware of that artifacts in the extreme range will give me more "support tickets" than having less-than-excellent color in the broad normal range, as the first is easily seen (in scenes where it's triggered), the other is just a matter of taste, so I'm not surprised Adobe's profile is the way it is.
-
All I can say is good luck with it, the results achieved in the normal range were certainly promising.
-
Keep going guys. You're almost at 1,000 replies.
-
By the way I tried implementing that alternate CAM presented in the paper "A neurophysiology-inspired steady-state color appearance model" by Kunkel and Reinhard.
I did not analyze it or anything just a quick copy paste from the paper and see what numbers I got compared to CIECAM02. I was hoping that perhaps it could handle the extreme range better, but actually it's the opposite. That model works in a smaller gamut so for my uses there's no obvious advantage over CIECAM02.
I'm thinking about making a special-case CAM which is a blend between CIECAM02, CIELAB and/or home-made RGB. For the extreme range one need to be pragmatic, no colorimetric model will yield "correct" results, we just need something that looks good.
The main use for these color appearance models in DCamProf is to have a coordinate system that can scale chroma in a decently uniform way while keeping hue and lightness constant. CIECAM02's JCh has less color shift issues than CIELab, but Lab is more robust for extreme values. Some have munsell corrected Lab (RawTherapee have that) or Bruce Lindbloom's UPLab which is the same thing - non-linear correction of the Lab coordinate system to provide better uniformity and less colorshift. These spaces are probably even better than CIECAM02 concerning uniformity, I don't know how they perform in the extreme range though.
I haven't find any strong need to further improve the uniformity of CIECAM02 for its use in DCamProf, but I can't really use it for say gamut-compressing ProPhoto to AdobeRGB, as it can't represent all colors in ProPhoto (which indeed has a blue corner outside the locus). There's another issue which is that for extreme range of blue the lightness drops to near zero (same would be for red too, but due to common RGB gamut orientations we never get there). So if we say have an extreme blue with ~0 lightness and then just compress chroma axis the lightness is still ~0. For this reason I'm considering a model where you stop doing normal colorimetric model for the extreme range but instead just pretends it's some RGB space (with brighter blue, probably sRGB-bright), and then do some suitable transition between them both. It will be a very application specific space, specifically made for general purpose camera profile making.
-
By the way I tried implementing that alternate CAM presented in the paper "A neurophysiology-inspired steady-state color appearance model" by Kunkel and Reinhard.
I did not analyze it or anything just a quick copy paste from the paper and see what numbers I got compared to CIECAM02. I was hoping that perhaps it could handle the extreme range better, but actually it's the opposite. That model works in a smaller gamut so for my uses there's no obvious advantage over CIECAM02.
Interesting result. It might be easier to straighten out though, since I think it is a bit simpler (one less matrix transform ?).
The main use for these color appearance models in DCamProf is to have a coordinate system that can scale chroma in a decently uniform way while keeping hue and lightness constant. CIECAM02's JCh has less color shift issues than CIELab, but Lab is more robust for extreme values. Some have munsell corrected Lab (RawTherapee have that) or Bruce Lindbloom's UPLab which is the same thing - non-linear correction of the Lab coordinate system to provide better uniformity and less colorshift. These spaces are probably even better than CIECAM02 concerning uniformity, I don't know how they perform in the extreme range though.
A much simpler space with good hue linearity that has been used as an alternative to L*a*b* and CIECAM is IPT space:
F. Ebner and M.D. Fairchild, “Development and testing of a color space (IPT) with improved hue uniformity”, Proc.6th IS&T/SID Color Imaging Conference, 8-13 (1998).
I think that iCAM also uses IPT as its underlying appearance space.
It is missing the chromatic adaptation machinery though, so needs a little adaptation to be practical.
Email me if you want more details about it.
-
IPT looks really interesting at least for the gamut mapping task.
The problem with many of the color appearance models in the extreme value range is that they're not designed to handle situations where you get negative value output in the various matrix conversions (for example in XYZ->LMS). Looking at the IPT equations I see that have special case handling for negative values, which should mean that it will work in the extreme value range. I'll try to implement it.
I'm thinking that I could use IPT to always do a little gamut mapping to get within CIECAM02 range in the tone reproduction operator / look design engine.
IPT is locked to the D65 whitepoint (I'm assuming the reason is because you get straighter hue lines with that white point). DCamProf works with D50 whitepoint as that's the whitepoint for both ICC and DNG camera profiles, but I think a CAT (probably Bradford CAT as CAT02 also has extreme range issues) D65<->D50 will yield satisfactory results.
-
How does Photoshop implement Lab->RGB out-of-gamut conversion?
-
I don't know how Photoshop does such things.
I've done some more experiments and at last I'm feeling that things are moving a little bit forward rather than around in circles. There will be gamut mapping in several stages, and they will be of different type.
In the first stage it's in the colorimetric native profile, where the issue is the linear matrix that push extreme values to negative ones that get clipped, which is no good. Here I've experimented with compression using lots of techniques and color spaces, like IPT, and what I seem to end up with in this stage is RGB-HSL(!). That is compression is done of the raw camera values before conversion on the Saturation axis of HSL. This is by no means a "perceptually uniform" space, but as it's on the raw values hue stability seems to be quite good and as it's HSL it transitions nice and smooth (smooth gradients) to the clipping end of the extreme range. Any colorspace based on some sort of conversion has issues with meeting up smoothly with clipping.
In later stage compressions there will be other techniques used.
-
Although the project may seem stalled, I am working on it. I expect to be able to deliver the next release in two weeks or so.
When programming on spare hours here and there sometimes the entire day's programming may be spent on some silly thing, like yesterday when the convex hull algorithm I've got off the 'net turned out to be buggy.
Anyway I'm overall seeing steady progress now and the next release will have some news, the weighting is remade, gamut compression is remade, it will be possible to manually tune also the colorimetric base profile, and various minor updates.
-
Hi,
"Invalid JSON data" error is returned when "dcamprof-0.9.13/data-examples/spectrum.json" is fed to make-target:
$ dcamprof make-target -p cc24 -i spectrum.json target.ti3
Invalid JSON data in file "spectrum.json"
$
Changing line 13 from '"scale:"' to '"scale":' (mind the colon) makes make-target accept the spectrum data file.
Another issue is that having 'scale' set to 100 (which is the default for provided sample file) or 1.0 results in XYZ values not being recalculated in ti3. Removing 'scale' from spectrum data file entirely (along with providing properly scaled data of course) makes make-target recalculate XYZ values as expected.
Is this a [known] bug?
-
Is this a [known] bug?
It's a bug, it was unknown to me, thanks for reporting. Will fix to next release.
-
A progress report; I've come as far as I'm rewriting the docs affected by the new behavior. Usually one discovers issues when testing the workflows though so it may still be a while.
Gamut compression which now happens in several places seems to work well, and the new weighting and target adjustment functionality gives better control of the colorimetric base profile result than before.
Tone reproduction look remains largely the same, with a liiiittle lighter blues per default (the new weighting allows making Adobe/C1 style light blues if one likes that, but being plain wrong per default is too much even for me :) ). Clipping behavior of bright colors (with in-gamut chromaticity) has changed to the better.
I'm still working on a very difficult problem though. I've got an excellent test file which is a sunset in haze, producing a loooong smoooth gradient from reds to yellows and into clipping at the sun center. ACR profiles renders this quite well although they "cheat" a bit by letting many neighboring colors become yellow and lighten them a fair bit which hides gradient issues. The file does look like a "fried egg" without a curve like sunsets do so a 100% perfect rendering with no visible bands in the gradient is not possible, and ACR doesn't do it either but it's as good as it can be.
The bands are not "posterizing", but simply that one channel starts changing a fair bit slower, so there's mathematically nothing wrong with the gradient, but the eye sees the flattening as a band. These bands often occur as well when one channel reaches clipping and the other continues to grow. You can't solve these issues 100% with a profile, but the profile can reduce or increase the effects.
With DCamProf I have quite clearly the "fried egg" issue with sunsets with such bands.
The source of the problem seems to be that models of luminance are not precise enough when translated to RGB. As soon as you do anything with some other color model than RGB (like JCh, IPT etc), gradient issues are likely to occur especially when you have a transition in hue at the same time as the luminance increases, like in a sunset. The eye is extremely sensitive when it comes to detect gradient issues, sub DE 1 for sure. I'm looking at how to improve the situation, without falling back to an RGB-oriented model like I think both Adobe and C1 has. An RGB-oriented model is better at handling extreme ranges, clipping and gradients, but it's worse when it comes to the general look of colors.
DCamProf is already today a hybrid, using more RGB models in the extreme ranges and clipping, but it still doesn't work as well as I'd like.
Evalutation is a bit tricky though as it's affected a bit by the screen, color management can mess up gradients too.
-
torger,
I've got an excellent test file which is a sunset in haze, producing a loooong smoooth gradient from reds to yellows...
What additional test-file phenomena, if any, would help you?
-
torger,
What additional test-file phenomena, if any, would help you?
Test files that fails usually give me lots of work so I like it when they come far in-between :-)
Currently I think I have a quite good set of files, but once in a while a user sends me a problem file which in some cases gets added to the ones I use for testing for specific problems.
I'll probably make some high saturation color test photos later, but I need to do them myself as I need to have a real-life reference that I can look at in that case.
Anyway after some work today I think I'm closing in on getting the hazy sunset to work satisfactory. I still think Adobe makes noticeably better result on it, but at the cost of other aspects (hue separation and accuracy) that I don't want to sacrifice. There's always a tradeoff. In this particular case noone can make it perfect (as the camera clips on the raw level) and it doesn't feel as bad to be 90% perfect when the best are only 95% perfect anyway :).
To make it work I had to drop the old rolloff-to-whitepoint algorithm and replace it with a new (still sorting out details on the new). It's dangerous stuff, there's a risk I break the look of other stuff so I need to test thoroughly on some of the old reference images I have. I hope it will work out well as the new rolloff is simpler than the old.
-
could anyone with better software chops than me post an osx executable of the latest version (0.9.13) pretty please?
-
could anyone with better software chops than me post an osx executable of the latest version (0.9.13) pretty please?
I think somebody posted OSX binaries couple of pages earlier in this thread...
-
Torger a big THANK YOU for DCamProf. I was very unhappy with/disagreed with C1's interpretation of colour for the new 5DSR. My photo partner has tried other software to generate a custom ICC profile for our camera and C1 with limited success. Unfortunately, the previous profile he attempted with other software disallowed the use of the color editor in C1.
DCamProf though has given us what looks to be an excellent profile to work from with full functionality in C1. We only created the profile yesterday and much of it is over my head but I've been reading through much of your text to better understand it all.
Thank you for creating such a comprehensive resource in understanding and applying custom profiling and I look forward to working with my custom profile.
-
Nice to get some feedback, I'm very glad when people find it useful.
The current version works well except in extreme ranges, that is it has some issues with clipping (like sunsets) and super-saturated colors, especially deep blue. That will be fixed in the coming version, which however takes forever to complete :-\ I'm guessing 2 - 4 more weeks now.
Wohoo, reply #1000!
-
Nice to get some feedback, I'm very glad when people find it useful.
I must join then and say that I am finding it very useful - thanks for writing and maintaining it!
So far I used it to build a very good matrix profile from captured SSF (simulating CC SG target) and that was really an easy thing to do.
-
+1, no += N!
Erik
Torger a big THANK YOU for DCamProf. I was very unhappy with/disagreed with C1's interpretation of colour for the new 5DSR. My photo partner has tried other software to generate a custom ICC profile for our camera and C1 with limited success. Unfortunately, the previous profile he attempted with other software disallowed the use of the color editor in C1.
DCamProf though has given us what looks to be an excellent profile to work from with full functionality in C1. We only created the profile yesterday and much of it is over my head but I've been reading through much of your text to better understand it all.
Thank you for creating such a comprehensive resource in understanding and applying custom profiling and I look forward to working with my custom profile.
-
Sorry,
I of course mean *= N! and make that N big!
Best regards
Erik
+1, no += N!
Erik
-
I've now released v0.10.0.
Going from 0.9 series to 0.10 may seem likes it's going from 90% complete to 10% complete, but hopefully I'll do a jump to 1.0 sooonish. I just had to make such big changes under the hood for the extreme value handling so I had to at least bump minor version.
There's a long list of changes on the home page. There's a bit of behavior change.
In terms of look it should be the same except clip bug fixes, plus on major change in blue handling. Cameras with "over-sensitive" blue channels (many sony sensors) will now render blue considerably lighter than before, and all cameras will render it a little lighter. The reason is a default negative factor matrix limiter, which can be turned off if you really want the dark blues traded for strong LUT bends at the gamut edge.
There's so much changes I expect a few minor updates before it fully stabilizes.
-
May be something with my system (OS X) but make fails on argyllio.c
I'll have a look tonight.
argyllio.c:86:9: error: unknown type name 'ssize_t'; did you mean 'size_t'?
ssize_t linefeed = -1;
^~~~~~~
size_t
/usr/include/sys/_types/_size_t.h:30:32: note: 'size_t' declared here
typedef __darwin_size_t size_t;
^
argyllio.c:87:14: error: use of undeclared identifier 'ssize_t'; did you mean 'sizeof'?
for (ssize_t i = 0; i < (ssize_t)data_size; i++) {
^~~~~~~
sizeof
argyllio.c:87:22: error: use of undeclared identifier 'i'
for (ssize_t i = 0; i < (ssize_t)data_size; i++) {
^
argyllio.c:87:29: error: use of undeclared identifier 'i'
for (ssize_t i = 0; i < (ssize_t)data_size; i++) {
^
argyllio.c:87:34: error: use of undeclared identifier 'ssize_t'
for (ssize_t i = 0; i < (ssize_t)data_size; i++) {
^
argyllio.c:87:53: error: use of undeclared identifier 'i'
for (ssize_t i = 0; i < (ssize_t)data_size; i++) {
^
argyllio.c:88:22: error: use of undeclared identifier 'i'
if (data == '\n' || data == '\r') {
^
argyllio.c:88:41: error: use of undeclared identifier 'i'
if (data == '\n' || data == '\r') {
^
argyllio.c:89:48: error: use of undeclared identifier 'i'
if (linefeed == -1) linefeed = i;
^
argyllio.c:91:52: error: use of undeclared identifier 'i'
if (linefeed != -1 && isblank(data)) {
^
argyllio.c:92:40: error: use of undeclared identifier 'i'
while (linefeed != i) data[linefeed++] = ' ';
-
Thanks for testing, fixed that. Re-download it again.
-
Fixed, thanks. Will test over the weekend.
-
For those making their own custom looks using look operators I've become aware of that if you change increase saturation or modify hues etc close to the neutral axis there's a risk that you disturb noise reduction algorithms. I've only observed this as a real problem in Lightroom, but I guess it can happen elsewhere too.
A way to fix this is to roll off your effects towards the neutral axis. A common subjective adjustment is to make "neutrals more neutral" and if you have that type of adjustment you have already fixed this as a side effect. Reducing chroma of close-to-neutral colors help noise reduction as a side effect.
-
Version 0.10.0 for Mac OSX now available.
-
0.10.0 build for Windows ( mingw = dcamprof.exe + libgomp_64-1.dll + both manual & tutorial / = copies of Torger's web pages / in 3 formats : IE archive .mht, Mozilla archive .maff and regular .pdf ) : https://app.box.com/DCamProf (https://app.box.com/DCamProf)
-
The tutorial has quite much updates, a new section on active matrix design:
http://www.ludd.ltu.se/~torger/photography/camera-profiling.html#step6_matrix
which you did not do before. It's rarely something you need to care about though unless you are specially interested and like to make your own tradeoffs.
However I've come across one camera that required some manual matrix design to get good results, the Sony NEX 6. The strongest blue subtraction factor ever and quite noisy too, a normally/automatically designed profile caused the deep blues to be incorrectly pulled against purple. By using the new target adjustment feature (pushing blue against cyan to give it some margin) and the matrix limiter it's possible to control the color also for that though.
-
Built and tested a simple neutral profile on several pics, and blues look more saturated than other hues. Is it an expected side effect of the recent changes ?
-
Built and tested a simple neutral profile on several pics, and blues look more saturated than other hues. Is it an expected side effect of the recent changes ?
Thanks for taking time to post images, it's really worth a thousand words as they say :)
As usual I'm not at my workstation with the calibrated screen so I must look at it there when I get there later. On my less-than-perfect screen here it also looks like the sky is considerably less magenta. The difference looks pretty big, so something must have happened.
I don't know what though. I shall look at some plain sky test shots and see if I can figure it out.
I've got one more complaint on "too saturated blues", but not in skies but on more saturated blues. That's on a 645z where the blues got a lot lighter, increasing lightness does increase saturation as a side effect but that should have been compensated for.
Considering hue, do you know which one of your shots that is most true to the original scene? I know P1 renders stuff quite warm (yellow), but anyway the blue in skies can turn towards magenta, cyan and indeed yellow depending on atmospheric conditions and sun position.
I think the change in hue may be due to a change in rolloff algorithm, but it changed as early as 0.9.3. In earlier versions it was more pure RGB-HSV-hue-oriented and HSV-hue has specifically an issue in the blue range where blue turns magenta. Since some time that is stabilized and hue should be kept more constant. It might be that effect we see regarding hue, but it doesn't match the claimed version so it's probably something else then.
I need to investigate a bit anyway.
-
I'd say P1 and dcamprof 0.10 are closer to the 'true' sky hue, dcamprof 0.9.10 (or 11) is too magenta. I'm used to correcting skies as LCCs seem to mess a lot with hues in this range (take them to magenta territories), so the hue shift is not what strikes me most here.
The brighter part of the sky, the first half starting from the horizon here, feels almost electric with dcamprof 0.10. According to my tests on different pics it looks like only a limited range exhibits this over-saturated blue phenomenon, maybe it's only a matter of smoothing that new specific blue handling.
-
I'd say P1 and dcamprof 0.10 are closer to the 'true' sky hue, dcamprof 0.9.10 (or 11) is too magenta. I'm used to correcting skies as LCCs seem to mess a lot with hues in this range (take them to magenta territories), so the hue shift is not what strikes me most here.
The brighter part of the sky, the first half starting from the horizon here, feels almost electric with dcamprof 0.10. According to my tests on different pics it looks like only a limited range exhibits this over-saturated blue phenomenon, maybe it's only a matter of smoothing that new specific blue handling.
Looking at my workstation screen now, and I can see now. There's been more than one trick to handle the blue range so this may be a side effect from that. I'm going to investigate.
Will investigate.
-
The change in sky blue saturation sits in the updated rolloff handling in 0.10.0. The change was not intended. To quickly restore the old behavior, make this change in look.c (any large number will do so the alternate rolloff is never triggered):
--- a/look.c
+++ b/look.c
@@ -887,7 +887,7 @@ look_tone_rep_op(v3 *out,
// (test on skintone highlights), so we make a fade between
const double acr_chroma_lo_sat = acr_jch.v[1];
const double acr_chroma_hi_sat = jch.v[1] * (acr_jch.v[1] / orig_jch.v[1]);
- const double lolim = 10;
+ const double lolim = 1000000000;
const double hilim = 60;
if (orig_jch.v[1] < lolim) {
acr_chroma = acr_chroma_lo_sat;
The old has issues with gradient smoothness in sunsets though, so I need to make some better fix.
-
It looks better now (new version on the right).
-
I see a hue shift in your sky blues, not saturation. I do see the P1 sky blue as having too much green.
DcamProf nails the UV bias the sun provides in cloudless daylight sky blue. Greenish skies are inaccurate, unless you have a lot of air pollution where that was taken.
-
After a little gradient testing I have now released v0.10.1 which contains the hotfix of the rolloff (important fix!), in essence the same result as in the one-liner patch a few posts back.
-
Version 0.10.1 for Mac OSX now available.
-
0.10.1 build for Windows ( mingw = dcamprof.exe + libgomp_64-1.dll + both manual & tutorial / = copies of Torger's web pages / in 3 formats : IE archive .mht, Mozilla archive .maff and regular .pdf ) : https://app.box.com/DCamProf (https://app.box.com/DCamProf)
-
Thanks Torger for Version 0.10.1 and AlterEgo for the Windows build.
I had noticed some clipping in the previous version but otherwise has been excellent.
-
I'm now working on refining high saturation color handling of the neutral tone reproduction operator. It's currently a little bit over-saturated, at least in some hue ranges (got some issue report about over-saturated blues, it's always about the blues it seems :) ). I'm hoping to be able to keep the saturation control hue-independent like it is today, but I'm not sure, maybe blue range needs (yet) some extra handling.
The tone reproduction has so far been tuned mostly against normal range colors, faces and such and in that range the principle has been that rather have a tiny bit too much saturation than too little, as the underlying luminance curve has a tendency to make things look grayer than they are. However I'm probably going to do the other way around for high saturation colors, rather present them with a tiny bit too low saturation.
-
Found the cause of the over-saturation, mainly seen in dark blue colors. In the tone reproduction operator there's a chroma push of shadows that is there to counter-act the grayness of the luminance curve. This works well on faces and other low saturation material, but is no good on higher saturation colors. Blue is most affected as blues are dark and thus get into the shadowy range.
I will fix this to the next release (the fix is easy, simply rolling off the effect as saturation increases).
-
Hi Anders,
Are you having the "blues" blues today?
Best regards
Erik
I'm now working on refining high saturation color handling of the neutral tone reproduction operator. It's currently a little bit over-saturated, at least in some hue ranges (got some issue report about over-saturated blues, it's always about the blues it seems :) ). I'm hoping to be able to keep the saturation control hue-independent like it is today, but I'm not sure, maybe blue range needs (yet) some extra handling.
The tone reproduction has so far been tuned mostly against normal range colors, faces and such and in that range the principle has been that rather have a tiny bit too much saturation than too little, as the underlying luminance curve has a tendency to make things look grayer than they are. However I'm probably going to do the other way around for high saturation colors, rather present them with a tiny bit too low saturation.
-
I've just released v0.10.2.
Mostly tunings of the tone reproduction operator, including the important over-saturated shadows fix.
-
Thanks, looks good, blues are still a tad more saturated than the rest but it probably won't bother many people.
I'll wait it 'stabilizes' a bit before attempting to design looks again.
-
0.10.2 build for Windows ( mingw = dcamprof.exe + libgomp_64-1.dll + both manual & tutorial / = copies of Torger's web pages / in 3 formats : IE archive .mht, Mozilla archive .maff and regular .pdf ) : https://app.box.com/DCamProf (https://app.box.com/DCamProf)
-
Version 0.10.2 for Mac OSX now available
-
+2 to AlterEgo and howardm!
-
The design principle I still go by with the tone reproduction is to assume that the base colorimetric profile is accurate, and then the purpose of the tone reproduction is to maintain the same color appearance, not increase or decrease saturation. That is I'm not really designing "by taste", but by color appearance matching. Then there's some taste and judgements involved anyway of course as it's not possible to make perfect and then one must make tradeoffs.
With the ACR tone curve doing A/B swaps with a curve-less (colorimetric) rendering pushed 0.7 stops to equalize brightness is a base technique I'm using. The adjustments I've done recently has been possible to demonstrate as mismatch when doing these swaps on new test images I've received.
Blue is a little bit special as discussed in several past posts which has made it necessary to work with more tradeoffs. However with this last release I think I have quite good A/B matching with all the test material I have got. The error fixed in this last release was demonstrated in a small deep blue traffic sign in a Pentax 645z shot that was sent to me.
So if any of you has a test shot that demonstrates a clear mismatch I'm interested to have it if I may. The recent fixes has all been initiated by user contributions.
There are two major aspects to know about when it comes to camera profiles and hue accuracy. Nearly all bundled/commercial profiles renders colors warm (more yellow) than accurate, and nearly all renders blue much lighter and not rarely more cyan than accurate. In the tone reproduction operator I don't design towards these targets, the aim is accurate matching with the colorimetric profile as long as possible. With blue I had eventually to go lighter for technical reasons, but it's still rendering darker than most bundled profiles that go even lighter.
Then the idea that a custom look is designed on top of this, my "Neutral+" profiles add more yellow etc. I haven't added any specific blue adjustments to it though, I haven't really decided on that yet.
From the same person I got the blue road sign over-saturated issue, I also got a forest greens lacking yellow issue, I did not do anything about the latter as I judged that to be one of those bundled yellow-rendering profiles things.
Hue is not likely to be bad, except possibly in extreme ranges and such. Saturation matching is more tricky, and then we have this rolloff (desaturation) into clipping which is sort of all about taste. Saturation handling in the extreme range is also a bit about taste more than about matching.
-
Thanks, looks good, blues are still a tad more saturated than the rest but it probably won't bother many people.
Despite forced lightening due to technical reasons as discussed in past posts, DCamProf's default blues are typically rendered considerably darker (and as such more realistic) than the typical bundled profile. However a darker (deeper) blue may look more saturated, especially when we've used to see lighter blues from bundled profiles. I've tried to under-saturate it a bit to compensate, but then it goes into grayness instead. For the "blue" test shots I have I'm pleased with the level of saturation produced by v0.10.2. However if you have some shot where over-saturation is seen I'd love to have a look to see if further tuning can be made.
There's a nagging thought in the back of my head that tells me that maybe we must render blues light light light (and even cyan-polluted) like most do in order to get pleasing tonality etc, so maybe even more "hacks" need to be made. So far I don't really think so (blues are already considerably lightened), but it's too early to say that I've "landed" the issue.
-
Actually I was comparing the rendering against 0.9.x profiles. Saturation looked better balanced across the whole hue range.
You are right it may be darker blues mistaken for more saturated ones though, will keep on testing and send files if needed.
-
From a personal message
I've now released v0.10.0 so you can play with the adjustments yourself.
Based on a cc24 imaging resource shot (rather than nex5.json) I got good results wiith this command:
dcamprof make-profile -a target-adjust.json cc24.ti3 my-profile.json
Where the target-adjust.json contains the following:
{ "PatchAdjustments": [ { "Name": "C01", "ScaleRGB": [ 0.8, 1.0, 1.0 ] } ] }
If you use nex5.json you may need more aggressive scaling, like increasing green to 1.2 from 1.0.
I haven't tried 0.10.2 yet but proceeding as above with 0.10.1 I get much better results than before. But certain blues still tend toward purple. My aim is to get usable profiles for C1 as their profile for NEX6 are terribly reddish, so much so that I find them unusable. But if I compare them now with dcamprof they have a remarkably good separation of various blues, which dcamprof doesn't quite match yet. I will play a bit more with above patch adjustments. I feel some blues are really wrong now but it is in situations where I can't say what is right.
I did notice darker blues as well but found them quite good and closer to reality. I'll see if I can find more examples to post. And considering saturation the new profiles are generally more saturated it seems. I don't think that is an improvement. Again I might find an example to post later. Just thought to let you know what I found so far. Thanks for all the work.
-
From a personal message
I haven't tried 0.10.2 yet but proceeding as above with 0.10.1 I get much better results than before. But certain blues still tend toward purple. My aim is to get usable profiles for C1 as their profile for NEX6 are terribly reddish, so much so that I find them unusable. But if I compare them now with dcamprof they have a remarkably good separation of various blues, which dcamprof doesn't quite match yet. I will play a bit more with above patch adjustments. I feel some blues are really wrong now but it is in situations where I can't say what is right.
I did notice darker blues as well but found them quite good and closer to reality. I'll see if I can find more examples to post. And considering saturation the new profiles are generally more saturated it seems. I don't think that is an improvement. Again I might find an example to post later. Just thought to let you know what I found so far. Thanks for all the work.
0.10.1 has an over-saturation issue in shadows (and deep blue reaches down in that range), so do test 0.10.2 before evaluating the level of saturation.
There's a tonality problem with rendering blues accurately dark which is that the eye is less sensitive there and thus you don't get as good tonal separation as if you transform the color into lighter more cyan tones. DCamProf does lighten blues today related to technical issues, but still keep it at a fairly realistic level. My intention is to not pull it further from realistic than technically required, and then if you want to improve tonality at expense of accuracy/realism you do that with possibly pre-adjusting blue (through target adjustment) and the rest with a look operator.
Your camera is of special interest though as it seems it's hard to measure accurately and/or make a suitable matrix for that keeps blues stable, that is there may be issues directly in the colorimetric aspect of the software rather than in tone reproduction. It could also be that the CC24 does not work well with the camera due to its special blue response, that is there would have to be more blue patches in order to make more non-linear LUT corrections.
Anyway, let us know what you find.
-
When calibrating using a CC24, the deep blue is much decided by the C01 deep blue patch. This patch is very dark, and as such it's very sensitive to glare. If the measurement contains a large amount of glare then the software will think that the camera renders blues light and desaturated (as they become by glare) and will thus compensate by darkening and saturating.
One theory is that the blue issues some are experiencing could be related to this, that is that the tone reproduction operator may translate the blues from colorimetric profile accurately, but that the blues are too dark and saturated in the colorimetric profile itself due to measurement error.
If this is the case systematically I may need to revisit the glare compensation feature. When I did glare compensation (linearization) a few months back my conclusion was that it did not really work unless the glare was small in the first place, and even then it was not stable. You can play with glare compensation in testchart-ff command, but in normal workflows there's no glare compensation made. I'm thinking if you put the requirement that glare must be low one might be able to make some strongly simplified model which is simply modifying saturation and lightness of the reference values depending on their original lightness. Before I put an effort into that I must know that there actually is a measurement issue concerning blues.
If you do suspect an issue, I strongly suggest to also render a pure colorimetric profile (that is without a tone curve) and test that too to see if the problem is that the color is wrong in the colorimetric profile, or if the problem is due to the tone reproduction (that is the color transformations that take place when the curve is applied). As a curve generally brightens the image a lot too, to be able to compare before / after curve (switching between your two profiles) pushing the exposure with about 0.7 stops of the colorimetric profile is often helpful.
-
Here's a screenshot from http://colorizer.org/ (I recommend that warmly if you need RGB sliders to play with) showing the issue of tonality and rendering accurate deep blues.
Here I'm comparing the transition from blue to cyan with blue to black. If you lighten blue and push it towards cyan you get more of the cyan range, while if you render deep blue more accurately you get more towards the blue-to-black range. It's much easier for the eye to separate tones in the lighter blue-to-cyan range.
In this screenshot you can also see why it's better to push deep blue towards cyan (that is add in more green) than it is to push it towards purple (add in more red). While cyan sort of looks like a "brighter/overexposed blue" and makes up a nice transition, if you taint blue with red it becomes purple which clearly is a vastly different color.
You can also see in the left range of the R/G sliders (purple and cyan scale) that blue with little red in it looks exactly the same as blue with very little green in it. There's simply many deep blue colors that look virtually the same.
-
Version 0.10.1 for Mac OSX now available.
hi howardm,
I downloaded your complied DCamProf to my OSX (10.10.4), but it is not executable. Do I miss something?
-
I doubt you did something wrong. Have you downloaded other versions that I've created?
What appears to have happened, and this is not overly unusual when doing file transfers is that the Unix 'execute' bit
gets turned off (for security purposes).
So, you often have to turn it back on.
If you do a 'ls -lt ./dcamprof-v0.10.2' command, does the left side show something like rw-r--r-- ???
If so, then do a 'chmod 755 ./dcamprof-v0.10.2' and then you should be able to run it.
I will say that the various versions have been downloaded a total of 34 times and this is the first 'problem' report so
I dont think it's an actual problem, just a user education/knowledge issue.
I suppose if I 'zip'd' the file before I uploaded then the bit would remain set but that would force people to unzip. Potato-Potatoe
-
yes! permission was the problem. it works now!! thanks!! ;D
I doubt you did something wrong. Have you downloaded other versions that I've created?
What appears to have happened, and this is not overly unusual when doing file transfers is that the Unix 'execute' bit
gets turned off (for security purposes).
So, you often have to turn it back on.
If you do a 'ls -lt ./dcamprof-v0.10.2' command, does the left side show something like rw-r--r-- ???
If so, then do a 'chmod 755 ./dcamprof-v0.10.2' and then you should be able to run it.
I will say that the various versions have been downloaded a total of 34 times and this is the first 'problem' report so
I dont think it's an actual problem, just a user education/knowledge issue.
I suppose if I 'zip'd' the file before I uploaded then the bit would remain set but that would force people to unzip. Potato-Potatoe
-
AlterEgo,
what is the latest/best ColorChecker SG spectral measurement data you have? Can I include and distribute with DCamProf as a ccsg_ref.cie file?
-
AlterEgo,
what is the latest/best ColorChecker SG spectral measurement data you have? Can I include and distribute with DCamProf as a ccsg_ref.cie file?
didn't I post the measurements here ? I need to check, but I think I did and did also after X-Rite replaced my original "defective" chart following some protracted dispute between me and their CSRs about the measurements (numbers) of the black patches ... in any case all the files I ever attached here in LuLa forum with whatever measurements _I did_ are absolutely free to use by anybody for whatever purposes...
-
didn't I post the measurements here ? I need to check, but I think I did and did also after X-Rite replaced my original "defective" chart following some protracted dispute between me and their CSRs about the measurements (numbers) of the black patches ... in any case all the files I ever attached here in LuLa forum with whatever measurements _I did_ are absolutely free to use by anybody for whatever purposes...
The thread is quite long hard to find. I think this is the latest but I'm not sure:
http://forum.luminous-landscape.com/index.php?topic=100015.msg848342#msg848342
-
...
-
...
Thanks. Which one of the two do you think is more accurate? The Spectrolino or the Ipro? If I remember correctly there are quite some difficulties to measure the black patches dark enough.
-
Thanks. Which one of the two do you think is more accurate? The Spectrolino or the Ipro? If I remember correctly there are quite some difficulties to measure the black patches dark enough.
I used the ones from Iliah MakeInpitICC (I believe they were measured with Spectrolino) to make profiles from SSF - they worked great.
-
Thanks. Which one of the two do you think is more accurate? The Spectrolino or the Ipro? If I remember correctly there are quite some difficulties to measure the black patches dark enough.
I really do not know (I resorted back to using x-rite passport target shots for CMF/SSF approximation)... I, for example, do not even know whether XRGA or non XRGA mode is more valid ;D ... that XRGA was brought by X-Rite to make the results from instruments from 2 different companies (GmB and X-Rite, after GmB was purchased by X-Rite) closer... but that brings a question - who was more precise in the first place :o ( https://www.xrite.com/documents/literature/en/L7-462_XRGA_WhitePaper_en.pdf ) ?
-
I used the ones from Iliah MakeInpitICC (I believe they were measured with Spectrolino) to make profiles from SSF - they worked great.
the questions is - can you trust your target... the one I got first turned out to be defective... now looking @ manufacturers of the targets, X-Rite, with all fine print, seems to be the best one, but can you trust w/o measuring and when measuring - can you trust the instrument ?
Iliah's data close to GmB's, but in one patch quite different from GmB's data for SG
my Spectrolino (the really old one device) measurements of my (X-Rite made in 2014) target is closer to Iliah's data than my new i1Pro2 (in XRGA mode, mind you) to the same
but both of my measurements are as close (or may be even closer) between themselves as Iliah's vs GmBs... so may be, may be... the culprit is really the target(s) that we are measuring ?!
PS: I also want to confess that I did not gather a strength to do multiple measurements of SG and average those...
-
here is comparison between one of my Spectrolino's in non XRGA mode and i1Pro2 in XRGA mode with my SG target :
===
PatchTool COMPARE TOOL - COMPARE STATS REPORT
This file combines and compares the data of two files which have the same number of samples.
This report presents statistical data derived from the differences between the color values of these two data sets.
Date: "2015-10-29"
Time: "03:45:08 PM"
Version: "4.7.0 b362"
REFERENCE
- Name: "CCSG(v2014.11)-10.0nm-i1Pro2-XRGA (SN#1044184,2015-07-09).cie (M0)"
# The SOURCE data type is < spectrum >.
SAMPLE
- Name: "CCSG(v2014.11)-10.0nm-Spectrolino (SN#17984,2015-07-09).cie (M0)"
# The SAMPLE data type is < spectrum >.
STATS-SETTINGS
Delta parameter: "E*"
Illuminant: "D50"
Observer: "2 deg."
Delta-E* formula: "CIEDE2000"
Absolute values: "NO"
Separate Neg./Pos. stats: "NO"
Negative samples: 0
Positive samples: 140
Number of samples: 140
AVERAGE
All samples: 0.64
Best 90%: 0.60
Worst 10%: 0.95
STANDARD-DEVIATION
All samples: 0.19
Best 90%: 0.17
Worst 10%: 0.05
MAXIMUM-ERROR
10th percentile: ± 0.38
Median (50th perc.): ± 0.64
90th percentile: ± 0.89
95th percentile: ± 0.93
Of all samples: ± 1.06
HISTOGRAM-DATA
Bin-size: 0.2000
Bin center 0.1000 0.3000 0.5000 0.7000 0.9000 1.1000
No patches 2 13 43 53 27 2
CUMULATIVE RELATIVE FREQUENCY (CRF) DATA
CRF of all-samples:
Max-error 0.1393 0.3129 0.3830 0.4242 0.4484 0.5067 0.5377 0.5636 0.5943 0.6248 0.6438 0.6581 0.6804 0.7226 0.7456 0.7722 0.8126 0.8463 0.8856 0.9325 1.0585
CRF 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
-
and here is Iliah's vs GmB's own :
===
PatchTool COMPARE TOOL - COMPARE STATS REPORT
This file combines and compares the data of two files which have the same number of samples.
This report presents statistical data derived from the differences between the color values of these two data sets.
Date: "2015-10-29"
Time: "03:47:45 PM"
Version: "4.7.0 b362"
REFERENCE
- Name: "ColorCheckerSG_spectral.cie (M0)"
# The SOURCE data type is < spectrum >.
SAMPLE
- Name: "Iliag Borg = DigitalColorCheckerSG.cie (M0)"
# The SAMPLE data type is < spectrum >.
STATS-SETTINGS
Delta parameter: "E*"
Illuminant: "D50"
Observer: "2 deg."
Delta-E* formula: "CIEDE2000"
Absolute values: "NO"
Separate Neg./Pos. stats: "NO"
Negative samples: 0
Positive samples: 140
Number of samples: 140
AVERAGE
All samples: 0.67
Best 90%: 0.60
Worst 10%: 1.32
STANDARD-DEVIATION
All samples: 0.29
Best 90%: 0.19
Worst 10%: 0.30
MAXIMUM-ERROR
10th percentile: ± 0.39
Median (50th perc.): ± 0.59
90th percentile: ± 1.08
95th percentile: ± 1.22
Of all samples: ± 2.32
HISTOGRAM-DATA
Bin-size: 0.5000
Bin center 0.2500 0.7500 1.2500 1.7500 2.2500
No patches 40 84 15 0 1
CUMULATIVE RELATIVE FREQUENCY (CRF) DATA
CRF of all-samples:
Max-error 0.0932 0.3225 0.3893 0.4479 0.4766 0.4903 0.5066 0.5382 0.5627 0.5827 0.5917 0.6119 0.6391 0.6698 0.7114 0.7769 0.8541 0.9236 1.0752 1.2188 2.3185
CRF 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
-
I'm working on a new glare modeling algorithm and had some promising results. The old linearization (available in the testchart-ff command with the -L flag) does not work that well, but with the new one it seems like some sort of results can be had even with 1+ stop glare in the CCSG shot, like there is in for example Imaging Resource's raw files.
I'll also make it a bit easier than before to make use of the white-gray-black calibration patches along the CCSG border.
In my new glare approach I don't linearize the camera raw data, but instead add glare to the reference values (or reference spectra when available) to match the camera's glare. It's not as elegant and not how I'd want to do it, but it seems to be more robust. The key advantage of doing this way is that you know the CMF of the observer, you don't know the SSF of the camera (well if you did you wouldn't need glare modeling), and by that you can make demarcations in the model like keeping hue constant.
Glare modeling will still be no replacement for shooting the target properly though. I don't think I have seen a single CCSG shot with less than at least 1 stop of glare. Shooting glossy targets are tricky indeed. A CC24 shot with low amounts of glare will most likely make a more accurate profile than a CCSG shot with high amounts of glare, even if compensated with a glare reduction model, as there's too many unknowns when it comes to glare to make an accurate model.
-
the questions is - can you trust your target... the one I got first turned out to be defective... now looking @ manufacturers of the targets, X-Rite, with all fine print, seems to be the best one, but can you trust w/o measuring and when measuring - can you trust the instrument ?
If you read what I said then the answer is right there ;)
I can because my target is synthetic - generated from CCSG data, illuminant and SSF by dcamprof. I used that synthetic target to build and optimise matrix profile and results came out very good and close to the one Iliah built and tuned by hand (for Kodak SLR/n)
-
my target is synthetic
right, then it does not matter much which data - you can as well use GmB's own spectral data too (from profilemaker distribution).
-
Thanks for making available this tool!
I am just starting out and followed the "easy way" tutorial to build DCP profiles.
After reading through the documentation I am still unsure about two issues:
1. I use a Colorchecker Passport as a target. In your documentation you write that the cc24_ref.cie from DCamProf should be used in combination with ColorChecker.cht from Argyll. However, Argyll comes also with a ColorcheckerPassport.cht file, wouldn't this be a better choice for the passport target, or would this not be compatible with the cc24_ref.cie you provide?
2. Should the -C parameter also be used when creating a Tungsten profile, or is this only meant for the D50 one?
dcamprof make-profile -i StdA -C cc24.ti3 my-profile.json
Would this then be the correct command for merging the profiles?
dcamprof make-dcp -n "Camera Name" -d "Displayed Name of Profile" -i StdA -I D50 -t acr -g adobergb my-profile2.json my-profile1.json my-profile-dual.dcp
-
I don't have a spectral reference file for the color checker passport, so you have to use only the CC24 part of it, and then you use ColorChecker.cht.
If someone has a high quality spectral measurement file for the passport, it would be nice if I could get it for inclusion. But anyway, there are a few more patches on the passport but I don't think they add much to the CC24 as they are of similar color.
-C is used if you want the colors to look as if shot under D50. I only use it for light sources close to D50 or if I have some special need to match D50 despite different light. So the default choice is not to use it.
Say if you do reproduction work and use tungsten lights instead of flash, it's likely that you want it to look the same as if you where using a D50-ish flash, and then -C would be appropriate. For general purpose profiles I think it makes more sense to simulate the colors we actually see under that light. Tungsten is a bit complicated though as it's often so low in temperature that we don't do a full chromatic adaptation (ie we see that the light is yellow) and raw converters are not really designed to simulate that.
The merge command look correct just make sure that the profiles are listed in the right order, stda first then d50.
Thanks for making available this tool!
I am just starting out and followed the "easy way" tutorial to build DCP profiles.
After reading through the documentation I am still unsure about two issues:
1. I use a Colorchecker Passport as a target. In your documentation you write that the cc24_ref.cie from DCamProf should be used in combination with ColorChecker.cht from Argyll. However, Argyll comes also with a ColorcheckerPassport.cht file, wouldn't this be a better choice for the passport target, or would this not be compatible with the cc24_ref.cie you provide?
2. Should the -C parameter also be used when creating a Tungsten profile, or is this only meant for the D50 one?
Would this then be the correct command for merging the profiles?
-
right, then it does not matter much which data - you can as well use GmB's own spectral data too (from profilemaker distribution).
I prefer to use a real measured one from which good profiles were made. And Xrite has not published spectral data for them - only LAB that I could find
-
And Xrite has not published spectral data for them - only LAB that I could find
spectral data was supplied for example with ProfileMaker distro (the same data comes with BabelColor Patchtool)
also, for those who do not use SSF/CMF (and hence any reasonable variations not important - I mean for SSF/CMF case they are not important) and actually use the real SG target for profile making = http://xritephoto.com/ph_product_overview.aspx?ID=938&Action=Support&SupportID=5884&catid=28
X-Rite finally bothered to post this info officially this week
===
difference between Iliah's spectral data and and X-Rite LAB data for new SG targets (I have /and measured/ a new one, batch from Nov 2014) :
PatchTool COMPARE TOOL - COMPARE STATS REPORT
This file combines and compares the data of two files which have the same number of samples.
This report presents statistical data derived from the differences between the color values of these two data sets.
Date: "2015-10-30"
Time: "11:34:00 AM"
Version: "4.7.0 b362"
REFERENCE
- Name: "Iliag Borg = DigitalColorCheckerSG.cie (M0)"
# The SOURCE data type is < spectrum >.
SAMPLE
- Name: "ColorCheckerSG_After_Nov2014.txt (M0)"
# The SAMPLE data type is < XYZ >.
- Observer: "2 degree (fixed)"
STATS-SETTINGS
Delta parameter: "E*"
Illuminant: "D50"
Observer: "2 deg."
Delta-E* formula: "CIEDE2000"
Absolute values: "NO"
Separate Neg./Pos. stats: "NO"
Negative samples: 0
Positive samples: 140
Number of samples: 140
AVERAGE
All samples: 0.87
Best 90%: 0.78
Worst 10%: 1.68
STANDARD-DEVIATION
All samples: 0.44
Best 90%: 0.35
Worst 10%: 0.32
MAXIMUM-ERROR
10th percentile: ± 0.37
Median (50th perc.): ± 0.75
90th percentile: ± 1.39
95th percentile: ± 1.52
Of all samples: ± 2.39
HISTOGRAM-DATA
Bin-size: 0.5000
Bin center 0.2500 0.7500 1.2500 1.7500 2.2500
No patches 28 60 44 6 2
CUMULATIVE RELATIVE FREQUENCY (CRF) DATA
CRF of all-samples:
Max-error 0.0837 0.2188 0.3742 0.4567 0.4963 0.5528 0.5869 0.6166 0.6403 0.7193 0.7514 0.8216 0.9676 1.0391 1.1621 1.2296 1.3036 1.3256 1.3940 1.5171 2.3893
CRF 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
===
difference between my (one time, not averaged) i1Pro2 XRGA measured spectral data and X-Rite LAB data for new SG targets (I have /and measured/ a new one, batch from Nov 2014) :
PatchTool COMPARE TOOL - COMPARE STATS REPORT
This file combines and compares the data of two files which have the same number of samples.
This report presents statistical data derived from the differences between the color values of these two data sets.
Date: "2015-10-30"
Time: "11:36:29 AM"
Version: "4.7.0 b362"
REFERENCE
- Name: "CCSG(v2014.11)-10.0nm-i1Pro2-XRGA (SN#1044184,2015-07-09).cie (M0)"
# The SOURCE data type is < spectrum >.
SAMPLE
- Name: "ColorCheckerSG_After_Nov2014.txt (M0)"
# The SAMPLE data type is < XYZ >.
- Observer: "2 degree (fixed)"
STATS-SETTINGS
Delta parameter: "E*"
Illuminant: "D50"
Observer: "2 deg."
Delta-E* formula: "CIEDE2000"
Absolute values: "NO"
Separate Neg./Pos. stats: "NO"
Negative samples: 0
Positive samples: 140
Number of samples: 140
AVERAGE
All samples: 0.63
Best 90%: 0.59
Worst 10%: 1.02
STANDARD-DEVIATION
All samples: 0.23
Best 90%: 0.19
Worst 10%: 0.18
MAXIMUM-ERROR
10th percentile: ± 0.30
Median (50th perc.): ± 0.65
90th percentile: ± 0.89
95th percentile: ± 0.93
Of all samples: ± 1.60
HISTOGRAM-DATA
Bin-size: 0.5000
Bin center 0.2500 0.7500 1.2500 1.7500
No patches 37 99 3 1
CUMULATIVE RELATIVE FREQUENCY (CRF) DATA
CRF of all-samples:
Max-error 0.0792 0.1841 0.2959 0.4175 0.4697 0.4942 0.5256 0.5392 0.5822 0.6039 0.6505 0.6760 0.6970 0.7164 0.7524 0.7763 0.7959 0.8382 0.8854 0.9291 1.6020
CRF 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
-
Just released v0.10.3
Included CCSG data. The main new feature is "glare matching", a (for me) new approach to deal with glare. Instead of trying to linearize the raw samples glare is simulated and added to the reference values/spectra to match the camera.
It's now also possible to do flatfield and glare matching directly in the make-profile command by providing a target layout JSON pointing out white, black and middle gray patches. I've added example for the CCSG (ccsg-layout.json) and CC24 (cc24-layout.json). The SG with its border of white/gray/black patches supports full flatfield correction and locally varying glare matching. The CC24 with only one white patch doesn't support flatfield correction, but a global glare matching can be made. CC24 shots generally have low amounts of glare as it's a matte target but still I've added the use of glare matching in the basic use case examples. It makes a little difference, making dark colors a tiny bit lighter and a tiny tiny bit lower saturation.
To see stronger effect of glare matching you need to use a glossy target like the CCSG. I've attached an animated GIF that makes an A/B swap between the results of two profiles, one with glare matching one without (click on the image to start the animation). The shot comes from Imaging Resource and suffers quite some glare on the right side which you can see by that despite even light (all white patches equally bright) the black patches considerably brighter on the right side.
If you don't compensate glare make-profile will think that the camera renders dark colors very light and overall a bit low contrast and saturation and will thus make a profile that compensates, and thus dark colors get very dark and the profile becomes a little over-saturated (hard to see in the animated gif which is limited to 256 colors). This is most easily seen in the top right corner.
The other is the profile made on this target shot but with glare matching. I don't own a CCSG so I can't compare with my eyes but I think it makes a quite good job. I still strongly recommend against relying on glare matching rather than shooting your targets properly, but sometimes you get shots from third parties and you just need to make the best out of it.
The patch that really gives it away that one of the profiles have incorrect color is the pink patch in the CC24 section of the target (located in upper center in the target). In the profile without glare matching the pink is rendered much darker pretty close to the neighboring red patch, not at all how the CC24 looks.
Example command for CCSG:
scanin -v -dipn ccsg.tif ColorCheckerSG.cht ccsg_ref.cie
dcamprof make-profile -g ccsg-layout.json -y -100 -l 0 -t acr ccsg.ti3 glare-demo-full-correct.dcp
I included AlterEgo's spectrolino data as ccsg_ref-new.cie, the ccsg_ref.cie is for the old charts before November 2014. For the CC24 I only have for the old charts so far.
-
0.10.3 build for Windows ( mingw = dcamprof.exe + libgomp_64-1.dll + both manual & tutorial / = copies of Torger's web pages / in 3 formats : IE archive .mht, Mozilla archive .maff and regular .pdf ) : https://app.box.com/DCamProf (https://app.box.com/DCamProf)
-
I realize that I actually have November 2014 CC24 chart myself, that is the new one. I have measured it myself for my own calibration but I only have a Colormunki so I don't feel fully confident of the quality to distribute it with DCamProf.
If someone has a November 2014 (or newer) CC24 and have a good pro level gear spectral measurement of it and can share I'd love to have it for inclusion.
-
v0.10.3 available for Mac OSX
-
Another comparison with the C1 daylight profile : same WB, same curve (approximated), built from a CC24 target, patches measured and averaged with an i1 pro1.
Having the scene still in mind, here's what I can say
- P1 blues are too cyan, whereas dcamprof are too magenta. No real surprise here, my CC24 purple-blue and dark purple-blue are among the 3 worst for hue DE. Still, no big deal.
- P1 blues are too light, DCP ones are more realistic (I prefer/am used to the lighter ones though)
- P1 greens are too yellow, and perhaps a bit too saturated. DCP maybe lacks some.
- DCP saturated yellows (leaves) are closer to the true hue, P1 yellows are too green/lacks red. However they look a bit too saturated (not sure, hard to tell).
- the green-yellow-orange range clearly has better separation with dcamprof
The differences are actually very minimal, once the neutral operator rendering is stable enough I'll probably switch to dcamprof and design custom profile variants for my everyday use.
-
Nice example, good demonstration of typical differences.
P1's cyan skies is a result of their warm look which is pretty global, ie there's more yellow in most of their colors.
I'm a little concerned by the sky color not being correct, it does indeed look a bit too magenta. I have noticed that differences that look pretty tiny on the CC24 can translate to very visible hue differences in the sky for example. There's some more experimentation for me to do on the colorimetric accuracy part. It seems to me that those color errors can be pretty "random", that is too magenta when generated for your camera, but better on some other. That's not nice, I'd like the look to be same on all cameras. I'm not sure yet if it is feasible to make good (in this case meaning almost perfect) matching between cameras or we have to accept that on some cameras sky may slant to magenta on some other to cyan. You can always manually tune it of course (I'd look into target adjustment feature first to get your base profile hue pushed more to cyan), but I'd ideally like that you would not have to.
It also seems like the difference in appearance between a D50-designed and a D65-designed profile may be much larger than I've thought. Again doesn't look like much on a CC24 but in a real scene you see pretty obvious differences.
The saturation level is much a matter of taste. I've tried to match appearances but chroma is hard to match, unlike hue and certainly lightness there's a quite wide span that looks sort of matching, and it probably varies a bit from day to day what I find to be the exact right level. I have recently looked at saturation levels though and find the current one pretty good, but if they are anything they would be a tiny bit too saturated, at least in the normal range. It's very easy to adjust to taste though.
I'll probably experiment some with making a "lighter blue look" config through look operators, but I probably won't push blue any lighter than it is today in the default rendering. It's already a fair bit lighter than realistic for technical reasons and I want as little subjective elements as possible in the base profile. Light blues are as common as the warm tone in bundled profiles though so I can't disregard from the desire to have them.
-
The magenta sky may also be the result of the CC24 patches measurements. My i1 pro rev A is said not to be super accurate measuring high densities.
Around mid november I'll shoot different charts in better conditions and with lights closer to D50 (CC24, CCSG, IT8, CMP DT4, ...), both with the A7rII and the IQ. Should let us compare the blue behaviour on devices with different sensitivities.
-
The magenta sky may also be the result of the CC24 patches measurements. My i1 pro rev A is said not to be super accurate measuring high densities.
Around mid november I'll shoot different charts in better conditions and with lights closer to D50 (CC24, CCSG, IT8, CMP DT4, ...), both with the A7rII and the IQ. Should let us compare the blue behaviour on devices with different sensitivities.
That magenta sky might also be a result of the angle of light in the actual scene to your lens. I get changing sky hues shooting in similarly lit scenes depending on whether the sun is at my back, on the side or at an angle and at different times of day. I think lens coating might be adding to this as well.
-
The effect is consistent, whether it's a hazy polluted summer sky or a clean spring/autumn one. The entire blue range is affected by the tint shift, from shadows to skies, where it's the most obvious.
-
Skies do vary naturally from cyan to magenta depending on sun position and atmospheric conditions. LCC shots (if used here?) can also cause a slight white balance shift, but the comparison with P1 (with the same LCC) says it all I think, plus that Frederic has been very precise about color in all previous posts, so I trust his judgment.
Of what I've seen so far this does not occur in all DCamProf profiles though so the problem is probably in the base colorimetric profile. There seems to be some issues with getting blue hues right in some cases and I have not figured out all the reason for that.
-
> I don't have a spectral reference file for the color checker passport, so you have to use only the CC24 part of it, and then you use ColorChecker.cht.
Might be worth it to mention that the paper is different between CC24 and CCP24, CCP24 being closer to semi-gloss. Spectral responses are different, as they are different between CC24 and CCSG 24-patch portion.
From profiling standpoint, there is no additional benefit from using the other portion of CCP, the one that contains white balance adjustment patches. Using it may also make shooting more difficult.
-
I realize that I actually have November 2014 CC24 chart myself, that is the new one. I have measured it myself for my own calibration but I only have a Colormunki so I don't feel fully confident of the quality to distribute it with DCamProf.
If someone has a November 2014 (or newer) CC24 and have a good pro level gear spectral measurement of it and can share I'd love to have it for inclusion.
The sample variation between charts from different batches is sometimes very significant. The best strategy I know for those who do not have accurate measurements of the charts they use is to use the averaged results, like those reported on http://www.babelcolor.com/main_level/ColorChecker.htm or by Robin Myers in his library http://www.rmimaging.com/spectral_library/library_index.html
Goes without saying, if you can afford PatchTool or SpectraShop - get them.
-
Goes without saying
a question = will you use XRGA or non XRGA measurements ? may be it is just an irrelevant difference vs the magnitude of the other errors/tolerances in the process, but still...
-
a question = will you use XRGA or non XRGA measurements ? may be it is just an irrelevant difference vs the magnitude of the other errors/tolerances in the process, but still...
The question you are asking amounts to who's calibration standard is better, Gretag's or X-Rite's. My answer is Gretag's, and thus no XRGA if I can avoid it. Details: XRGA uses different weighting (for reds, mostly). Thus spectral data can be converted from one standard to another. dE worst case 1.4. The slightest differences in aperture and measuring geometry have more effect than the difference between Gretag and X-Rite. Gretag geometry is generally more accurate, with better conformance to 45/0. In some ways one can view XRGA as an attempt to correct for systematic geometry error.
PS: Please have a look at Tom Lianza's account on the matter http://www.color.org/events/frankfurt/Lianza_ICCFrankfurt2013instrument_compare.pdf
Few things about Mr. Lianza here: http://patents.justia.com/inventor/thomas-a-lianza
-
What of D65 and D50 is the better calibration illuminant for an all-around "daylight" profile?
Adobe uses StdA and D65 in their dual-illuminant profiles.
D50 is a whole lot easier to simulate with artificial light than D65 and is thus more practical. D50 is also the reference white in the connection space of both ICCs and DCPs. D50 is also a typical print viewing standard and closer to flash light.
But is D65 still preferable if you can have it? I don't really know.
Most cameras seems to be tuned to D50, moreso than D65. With D65 a we have this blue subtraction that gets pretty strong. The difference in color reproduction between a D50 and D65 profile seems to be pretty small, at least in SSF simulations. I recently had quite a large difference in a target shooting scenario but I think that was measurement error. I haven't really had the opportunity to compare myself with own shots.
With the difference in look between D65/D50 being as small as it seems, I tend to think that maybe D50 is what we should recommend when making a general purpose profile, even a dual illuminant one, StdA + D50 instead of StdA + D65.
Anyone that has any feedback/ideas on this?
-
PS: Please have a look at Tom Lianza's account on the matter http://www.color.org/events/frankfurt/Lianza_ICCFrankfurt2013instrument_compare.pdf
his presentation does not clearly say that GmB was more precise than X-Rite, but then he was X-Rite employee at the time of that presentation (even he was GmB employee before that).
-
his presentation does not clearly say that GmB was more precise than X-Rite
His presentation says enough on the matter of the relevance, especially given his position with X-Rite at the time. The reason I linked to Mr. Lianza presentation is to give a perspective different from my own.
As to who is more precise, that was not what I was saying - I used the word "better", because of the agreement between the instruments. Generally, repeatability, long-term stability, and inter-instrument agreement are all more important than "precision". I'm off this topic ;)
-
In photography we use D55 and 3200K (ISO Studio Tungsten).
For "interpolation" one wants two points as far apart as possible. D65 was chosen because D75 is nearly impossible to simulate accurately while D65 is not so problematic using xenon flash; while 3200 was too "close" and impractical given the popularity of incandescent bulbs.
-
His presentation says enough on the matter of the relevance, especially given his position with X-Rite at the time. The reason I linked to Mr. Lianza presentation is to give a perspective different from my own.
As to who is more precise, that was not what I was saying - I used the word "better", because of the agreement between the instruments. Generally, repeatability, long-term stability, and inter-instrument agreement are all more important than "precision". I'm off this topic ;)
WAIT, WAIT ! you said = "Gretag geometry is generally more accurate, with better conformance to 45/0", which a regular person can interpret as "more precise", is it not ?
as for "repeatability, long-term stability, and inter-instrument agreement are all more important than "precision"" - for example somebody has one instrument and that person is facing a choice - use XRGA or non XRGA data - something has to be closer to some magical NIST grade everything instrument... it is not about corporate issues with 2 lines of "legacy" instruments producing the data that now that both lines under one roof needs to be in some agreement within small dEwhatever and so XRGA adjustments were a solution (and X-Rite being a surviving company it seems natural /selfish/ that more adjustments were to GmB than to own X-Rite instuments)...
-
> somebody has one instrument and that person is facing a choice - use XRGA or non XRGA data
It is not a real choice. It does not matter which standard you use.
-
> somebody has one instrument and that person is facing a choice - use XRGA or non XRGA data
It is not a real choice. It does not matter which standard you use.
that's difficult to digest for the guts when the adjustments are quit big for non X-Rite legacy instruments (and allegedly consistent across the instruments of the same model)
(http://s14.postimg.org/b3lks7dg1/xrgagmb.jpg)
-
In photography we use D55 and 3200K (ISO Studio Tungsten).
For "interpolation" one wants two points as far apart as possible. D65 was chosen because D75 is nearly impossible to simulate accurately while D65 is not so problematic using xenon flash; while 3200 was too "close" and impractical given the popularity of incandescent bulbs.
with CMF/SSF data then it is logical to create for ACR/LR a dual illuminant DCP/DNG profile with "A" (or even lower) and D75 (or even further up), as you do not really illuminate any real target with any actual light, no ?
-
with CMF/SSF data then it is logical to create for ACR/LR a dual illuminant DCP/DNG profile with "A" (or even lower) and D75 (or even further up), as you do not really illuminate any real target with any actual light, no ?
You are limited to the light sources listed in EXIF standard (that's how Adobe implemented this), so, no, lower than A is not possible. D75 is the highest possible.
-
with CMF/SSF data then it is logical to create for ACR/LR a dual illuminant DCP/DNG profile with "A" (or even lower) and D75 (or even further up), as you do not really illuminate any real target with any actual light, no ?
As Iliah said the lowest possible is StdA (2850K) and the highest possible is D75 (7500K). The "inbetweeners" are interpolated with simple linear interpolation. The interpolation is only about making a smooth transition between the both matrices and LUTs, not about trying to interpolate something "accurate". For single-illuminant profiles the light source tag is not used.
It would be an interesting experiment using SSF to render a 4000K profile and see how much it differs from a dual-illuminant 2850K+7500K profile under 4000K light. I think we would find out that it differs substantially for temperatures close to the low and less for those close to the high due to the non-linear behavior.
I think I'll do some experiments...
-
A related(?) question: What happens in profiling when we are dealing with a different tint (green-magenta) with an otherwise same light temperature?
I am asking because I tried to make some profiles from the Imaging Resource CC24 shots. To check the quality I compared it to a profile made with a shot of my CC24 under midday sun.
Both profiles (for a Pentax K-5 II) used the same command line:
dcamprof make-profile -i D50 -C cc24.ti3 my-profile.json
Both profiles showed similar and rather low DE values, so from this regard everything should be fine. I expected no or only negligible visible difference when applying the profiles in Raw Therapee, but was surprised: The Imaging Resource profile has visibly more magenta skintones, in direct comparison it looks slightly wrong compared to my daylight CC24 profile.
I then checked the white balance of the light gray patch in Raw Therapee. For comparison, normally my camera shows around 0.985 Tint when grey-card measured in sunny midday conditions, but the IR shot white balance shows 1.082 tint, indicating that their light source seems to have a strong magenta tint, even if the color temperature is fine (~5500K).
What are your thoughts on this? Is the difference likely to be caused by the different tint in light sources, or are other factors (different lens, CC24 variations...) more likely to blame?
I can upload the profiles and intermediate steps if needed.
-
not about trying to interpolate something "accurate".
well then where is the "border" then ? if D75 is not more "accurate" in general use of a dual illuminant profile than D65 - why 'd D65 be more beneficial to use (SSF/CMF cases - hence not about getting real light) vs D55 or D50 ? will the transition be smoother with D65 than with D55 or D50 ?
-
well then where is the "border" then ? if D75 is not more "accurate" in general use of a dual illuminant profile than D65 - why 'd D65 be more beneficial to use (SSF/CMF cases - hence not about getting real light) vs D55 or D50 ? will the transition be smoother with D65 than with D55 or D50 ?
I don't know. I'm currently implementing support for spectral images so I can more easily make some practical experiments.
-
>It would be an interesting experiment using SSF to render a 4000K profile and see how much it differs from a dual-illuminant 2850K+7500K profile under 4000K light. I think we would find out that it differs substantially for temperatures close to the low and less for those close to the high due to the non-linear behavior.
Difference between interpolated 4230K and actual fluorescent F2 and similar pairs is what is really telling. Low CRI sources tend to break things, that is why I switched to spectral methods.
-
Difference between interpolated 4230K and actual fluorescent F2 and similar pairs is what is really telling.
but when we are using 4xxxK WB in a raw converter like ACR/LR it is not necessarily because of the fluorescent lights or so - it might as well be still some daylight or rather flash when photographer thinks this is the WB he needs (for his taste) for example, no ?
-
but when we are using 4xxxK WB in a raw converter like ACR/LR it is not necessarily because of the fluorescent lights or so - it might as well be still some daylight or rather flash when photographer thinks this is the WB he needs (for his taste) for example, no ?
When you are using 4xxxK is a raw converter all you are doing is setting white balance, because of the way colour temperature is defined; while we, I think, are talking about the colour.
-
When you are using 4xxxK is a raw converter all you are doing is setting white balance, because of the way colour temperature is defined; while we, I think, are talking about the colour.
OK, if the topic is about getting a better color transform for a raw when we have it from a shot under a fluorescent type (bad spectrum) of illumination using a dual illuminant profile built using a pair of good spectral illuminations like A and D**s then I digress
-
I have now released DCamProf v0.10.4. Some minor bug fix, and a new command si-render to process spectral images. I also added a KeepLightness flag to the Curves look operator so you can adjust hue without causing a lightness change as side effect.
I used that command to do various simulations to find out the "best" calibration illuminant choice. Generic things that can be said about a camera profile used at a too high temperature (say D50 profile at 10000K, or StdA profile at D50) is that blues get brighter and more cyan (increased blue and green content) and reds get darker and more purple (decreased red and green content). You also get a mild saturation increase of many colors. The opposite happens if you go the other direction.
But anyway, the actual choice does not matter that much, and I find it hard to say that StdA + D65 is any better than StdA + D50, but perhaps the other way around due to the blue thing. The StdA + D50 is very close at D65, and even higher it will appear slightly more saturated and have little bit brighter blues.
I'm probably going to be less active with DCamProf development now for some time. Haven't had any major issue reports for a while and I don't have any critical features left on my todo list.
-
0.10.4 build for Windows ( mingw = dcamprof.exe + libgomp_64-1.dll + both manual & tutorial / = copies of Torger's web pages / in 3 formats : IE archive .mht, Mozilla archive .maff and regular .pdf ) : https://app.box.com/DCamProf (https://app.box.com/DCamProf)
-
I have now released DCamProf v0.10.4. Some minor bug fix, and a new command si-render to process spectral images. I also added a KeepLightness flag to the Curves look operator so you can adjust hue without causing a lightness change as side effect.
I used that command to do various simulations to find out the "best" calibration illuminant choice. Generic things that can be said about a camera profile used at a too high temperature (say D50 profile at 10000K, or StdA profile at D50) is that blues get brighter and more cyan (increased blue and green content) and reds get darker and more purple (decreased red and green content). You also get a mild saturation increase of many colors. The opposite happens if you go the other direction.
But anyway, the actual choice does not matter that much, and I find it hard to say that StdA + D65 is any better than StdA + D50, but perhaps the other way around due to the blue thing. The StdA + D50 is very close at D65, and even higher it will appear slightly more saturated and have little bit brighter blues.
I'm probably going to be less active with DCamProf development now for some time. Haven't had any major issue reports for a while and I don't have any critical features left on my todo list.
I have only popped in and out of this thread reading bits and pieces and following the progress with interest, I am hopeful of finding time to understand and use the tool in the near future, but a huge thank you for the work you have done. Have the more recent changes impacted the previous profiles you shared for the SonyA7rII ? They have been brilliant and a much more enjoyable profile to work with for my use so far.
Thanks again
-
I have only popped in and out of this thread reading bits and pieces and following the progress with interest, I am hopeful of finding time to understand and use the tool in the near future, but a huge thank you for the work you have done. Have the more recent changes impacted the previous profiles you shared for the SonyA7rII ? They have been brilliant and a much more enjoyable profile to work with for my use so far.
Thanks again
Yes it has had some minor impact... I'm thinking about making the generation and distribution of example profiles a bit more structured, but haven't come that far on that. Meanwhile I can render an updated profile and post in the A7rII thread... I'll do it during the day.
EDIT: new profiles found in http://forum.luminous-landscape.com/index.php?topic=104195.msg867280#msg867280
-
Version 0.10.4 for Mac OSX now available
-
I have now released DCamProf v0.10.4. Some minor bug fix, and a new command si-render to process spectral images. I also added a KeepLightness flag to the Curves look operator so you can adjust hue without causing a lightness change as side effect.
I used that command to do various simulations to find out the "best" calibration illuminant choice. Generic things that can be said about a camera profile used at a too high temperature (say D50 profile at 10000K, or StdA profile at D50) is that blues get brighter and more cyan (increased blue and green content) and reds get darker and more purple (decreased red and green content). You also get a mild saturation increase of many colors. The opposite happens if you go the other direction.
But anyway, the actual choice does not matter that much, and I find it hard to say that StdA + D65 is any better than StdA + D50, but perhaps the other way around due to the blue thing. The StdA + D50 is very close at D65, and even higher it will appear slightly more saturated and have little bit brighter blues.
I'm probably going to be less active with DCamProf development now for some time. Haven't had any major issue reports for a while and I don't have any critical features left on my todo list.
Thank you for this new version, I'll redo my profiles.
-
I realize that I actually have November 2014 CC24 chart myself, that is the new one. I have measured it myself for my own calibration but I only have a Colormunki so I don't feel fully confident of the quality to distribute it with DCamProf.
If someone has a November 2014 (or newer) CC24 and have a good pro level gear spectral measurement of it and can share I'd love to have it for inclusion.
here we go (attached) :
X-Rite CC24 / ColorChecker Classic, November 2014 edition
averaged 5 passes with X-Rite i1Pro2 using BabelColor PatchTool v5.x in non-XRGA (GMDI) mode
PS: both file & data are free to be used for any purposes in any manner, etc, etc
--- comparing with the official X-Rite LAB data for November 2014+ targets ---
PatchTool COMPARE TOOL - COMPARE STATS REPORT
This file combines and compares the data of two files which have the same number of samples.
This report presents statistical data derived from the differences between the color values of these two data sets.
Date: "2015-11-17"
Time: "09:34:40 PM"
Version: "5.0.0 b397"
REFERENCE
- Name: "CC24 (ColorChecker Classic) 2014-11 = i1Pro2(SN#1044184, 2015-11-17, spectral, GMDI, AVG-5).cie (M0)"
# The SOURCE data type is < spectrum >.
SAMPLE
- Name: "ColorChecker24_After_Nov2014.txt (M0)"
# The SAMPLE data type is < XYZ >.
- Observer: "2 degree (fixed)"
STATS-SETTINGS
Delta parameter: "E*"
Illuminant: "D50"
Observer: "2 deg."
Delta-E* formula: "CIEDE2000"
Absolute values: "NO"
Separate Neg./Pos. stats: "NO"
Negative samples: 0
Positive samples: 24
Number of samples: 24
AVERAGE
All samples: 0.66
Best 90%: 0.58
Worst 10%: 1.50
STANDARD-DEVIATION
All samples: 0.36
Best 90%: 0.26
Worst 10%: 0.20
MAXIMUM-ERROR
10th percentile: ± 0.29
Median (50th perc.): ± 0.59
90th percentile: ± 1.14
95th percentile: ± 1.29
Of all samples: ± 1.70
HISTOGRAM-DATA
Bin-size: 0.1000
Bin center 0.0500 0.1500 0.2500 0.3500 0.4500 0.5500 0.6500 0.7500 0.8500 0.9500 1.0500 1.1500 1.2500 1.3500 1.4500 1.5500 1.6500 1.7500 1.8500 1.9500
No patches 0 0 4 4 2 2 2 4 2 0 0 2 1 0 0 0 1 0 0 0
CUMULATIVE RELATIVE FREQUENCY (CRF) DATA
CRF of all-samples:
Max-error 0.2386 0.2819 0.2902 0.2949 0.3166 0.3650 0.3940 0.4514 0.4585 0.5836 0.5851 0.6265 0.6436 0.7064 0.7123 0.7272 0.8515 0.8606 1.1410 1.2940 1.6992
CRF 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
--- comparing with the official X-Rite LAB data for pre November 2014 targets ---
PatchTool COMPARE TOOL - COMPARE STATS REPORT
This file combines and compares the data of two files which have the same number of samples.
This report presents statistical data derived from the differences between the color values of these two data sets.
Date: "2015-11-17"
Time: "09:41:20 PM"
Version: "5.0.0 b397"
REFERENCE
- Name: "CC24 (ColorChecker Classic) 2014-11 = i1Pro2(SN#1044184, 2015-11-17, spectral, GMDI, AVG-5).cie (M0)"
# The SOURCE data type is < spectrum >.
SAMPLE
- Name: "ColorChecker24_Before_Nov2014.txt"
# The SAMPLE data type is < XYZ >.
- Observer: "2 degree (fixed)"
STATS-SETTINGS
Delta parameter: "E*"
Illuminant: "D50"
Observer: "2 deg."
Delta-E* formula: "CIEDE2000"
Absolute values: "NO"
Separate Neg./Pos. stats: "NO"
Negative samples: 0
Positive samples: 24
Number of samples: 24
AVERAGE
All samples: 0.88
Best 90%: 0.83
Worst 10%: 1.43
STANDARD-DEVIATION
All samples: 0.31
Best 90%: 0.27
Worst 10%: 0.02
MAXIMUM-ERROR
10th percentile: ± 0.41
Median (50th perc.): ± 0.93
90th percentile: ± 1.20
95th percentile: ± 1.41
Of all samples: ± 1.45
HISTOGRAM-DATA
Bin-size: 0.1000
Bin center 0.0500 0.1500 0.2500 0.3500 0.4500 0.5500 0.6500 0.7500 0.8500 0.9500 1.0500 1.1500 1.2500 1.3500 1.4500 1.5500
No patches 0 0 1 1 1 1 3 3 0 4 4 3 1 0 2 0
CUMULATIVE RELATIVE FREQUENCY (CRF) DATA
CRF of all-samples:
Max-error 0.2235 0.3709 0.4091 0.5880 0.6024 0.6199 0.7531 0.7803 0.7909 0.9210 0.9311 0.9515 0.9755 1.0192 1.0429 1.0748 1.1235 1.1356 1.2046 1.4082 1.4525
CRF 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
-
The question you are asking amounts to who's calibration standard is better, Gretag's or X-Rite's. My answer is Gretag's, and thus no XRGA if I can avoid it. Details: XRGA uses different weighting (for reds, mostly). Thus spectral data can be converted from one standard to another. dE worst case 1.4. The slightest differences in aperture and measuring geometry have more effect than the difference between Gretag and X-Rite. Gretag geometry is generally more accurate, with better conformance to 45/0. In some ways one can view XRGA as an attempt to correct for systematic geometry error.
PS: Please have a look at Tom Lianza's account on the matter http://www.color.org/events/frankfurt/Lianza_ICCFrankfurt2013instrument_compare.pdf
Few things about Mr. Lianza here: http://patents.justia.com/inventor/thomas-a-lianza
I am sorry, but back to "how many angels can stand on the point of a pin?" (c).
Consider for example i1Pro2 device... it has both XRGA and non XRGA modes (at least available through software like BabelColor PatchTool and it does not seem that mr. Pascale invented that on his own - just implemented something that X-Rite itself gives out of the driver, no ?).
That means that the device itself is operating either with GmB calibration standard/geometry or with X-Rite calibration standard/geometry and then whatever is implemented inside is then (can be) translated to XRGA (by driver, I'd assume, shall ask mr. Pascale - which I actually did this morning).
So which "blood" runs in i1Pro2 then ?
I run both XRGA and non XRGA measurments yesterday, averaged for both, and run compare in PatchTool :
PatchTool COMPARE TOOL - COMPARE STATS REPORT
This file combines and compares the data of two files which have the same number of samples.
This report presents statistical data derived from the differences between the color values of these two data sets.
Date: "2015-11-18"
Time: "02:13:30 AM"
Version: "5.0.0 b397"
REFERENCE
- Name: "PatchTool Avg-9 (5 files)"
# The SOURCE data type is < spectrum >.
SAMPLE
- Name: "CC24 (ColorChecker Classic) 2014-11 = i1Pro2(SN#1044184, 2015-11-17, spectral, GMDI, AVG-5).cie (M0)"
# The SAMPLE data type is < spectrum >.
STATS-SETTINGS
Delta parameter: "E*"
Illuminant: "D50"
Observer: "2 deg."
Delta-E* formula: "CIEDE2000"
Absolute values: "NO"
Separate Neg./Pos. stats: "NO"
Negative samples: 0
Positive samples: 24
Number of samples: 24
AVERAGE
All samples: 0.10
Best 90%: 0.09
Worst 10%: 0.19
STANDARD-DEVIATION
All samples: 0.05
Best 90%: 0.04
Worst 10%: 0.01
MAXIMUM-ERROR
10th percentile: ± 0.03
Median (50th perc.): ± 0.10
90th percentile: ± 0.16
95th percentile: ± 0.17
Of all samples: ± 0.20
HISTOGRAM-DATA
Bin-size: 0.0500
Bin center 0.0250 0.0750 0.1250 0.1750
No patches 5 6 9 4
CUMULATIVE RELATIVE FREQUENCY (CRF) DATA
CRF of all-samples:
Max-error 0.0293 0.0311 0.0329 0.0391 0.0470 0.0512 0.0726 0.0750 0.0752 0.0980 0.1027 0.1034 0.1040 0.1265 0.1353 0.1366 0.1396 0.1408 0.1568 0.1722 0.1988
CRF 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
"PatchTool Avg-9 (5 files)" in the text above is XRGA mode measurements.
it seems that the difference between XRGA and non XRGA results points ( based on table 3 in https://www.xrite.com/documents/literature/en/L7-462_XRGA_WhitePaper_en.pdf ) that i1Pro2 is operating more like legacy X-Rite devices and then using i1Pro2 in XRGA mode shall actually bring it closer to GmB legacy devices, which you state is "better"...
:o !!!
PS: "table 3"
(http://s14.postimg.org/b3lks7dg1/xrgagmb.jpg)
-
I was corrected that I shall use dE* / CIELAB in PatchTool if comparing anything with the table from X-Rite PDF, not dE* / CIEDE2000 - so :
PatchTool COMPARE TOOL - COMPARE STATS REPORT
This file combines and compares the data of two files which have the same number of samples.
This report presents statistical data derived from the differences between the color values of these two data sets.
Date: "2015-11-18"
Time: "05:05:12 PM"
Version: "5.0.0 b397"
REFERENCE
- Name: "PatchTool Avg-1 (5 files)"
# The SOURCE data type is < spectrum >.
SAMPLE
- Name: "CC24 (ColorChecker Classic) = i1Pro2(SN#1044184, 2015-11-17, spectral, GMDI, AVG-5).cie (M0)"
# The SAMPLE data type is < spectrum >.
STATS-SETTINGS
Delta parameter: "E*"
Illuminant: "D50"
Observer: "2 deg."
Delta-E* formula: "CIELAB"
Absolute values: "NO"
Separate Neg./Pos. stats: "NO"
Negative samples: 0
Positive samples: 24
Number of samples: 24
AVERAGE
All samples: 0.17
Best 90%: 0.16
Worst 10%: 0.36
STANDARD-DEVIATION
All samples: 0.09
Best 90%: 0.07
Worst 10%: 0.04
MAXIMUM-ERROR
10th percentile: ± 0.08
Median (50th perc.): ± 0.14
90th percentile: ± 0.30
95th percentile: ± 0.32
Of all samples: ± 0.40
HISTOGRAM-DATA
Bin-size: 0.0500
Bin center 0.0250 0.0750 0.1250 0.1750 0.2250 0.2750 0.3250 0.3750
No patches 1 2 9 4 3 2 2 1
CUMULATIVE RELATIVE FREQUENCY (CRF) DATA
CRF of all-samples:
Max-error 0.0193 0.0643 0.0819 0.1005 0.1030 0.1076 0.1099 0.1200 0.1266 0.1331 0.1394 0.1538 0.1540 0.1883 0.2114 0.2325 0.2485 0.2724 0.3021 0.3237 0.3986
CRF 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
-
Consider for example i1Pro2 device... it has both XRGA and non XRGA modes.
That means that the device itself is operating either with GmB calibration standard/geometry or with X-Rite calibration standard/geometry and then whatever is implemented inside is then (can be) translated to XRGA (by driver, I'd assume, shall ask mr. Pascale - which I actually did this morning).
So which "blood" runs in i1Pro2 then ?
The i1Pro2 is native XRGA. X-Rite software uses a library (XRiteStandard.dll) to allow conversion between XRGA and legacy standards.
-
The i1Pro2 is native XRGA. X-Rite software uses a library (XRiteStandard.dll) to allow conversion between XRGA and legacy standards.
thank you so much for the input !
1) but to which legacy standard/calibration then "XRiteStandard.dll" 'd translate the spectrum (native hardware XRGA) when we run i1Pro2 in non-XRGA mode (using the relevant software like PatchTool for example... which btw does not have that .dll - ColorPort has it for example, but not PatchTool) ? to GmB legacy (as in Spectrolino, i1Pro, etc) or to X-Rite legacy (as in 5xx/9xx, etc) ?
2) Argyll utilities - do they then simply receive from i1Pro2 and output to user (display, file) native XRGA spectrum received straight from i1Pro2 w/o naming it as such (I mean we get XRGA data from Argyll in this case even Argyll does not use that word anywhere) ?
-
Hej Anders
Thank you very much for your work.
As a long analogue photographer, I'm quite new to the digital side (started two years ago with an a7). With buying an A7RII in august, I start having issues with colors in postprocessing. So I switched from LR to C1, then bought a CC24 and profiled with the x-rite plugin in LR, respectivly with the "C1-equivalent" camprof. It was better, but I still missed a good general profile. Then I stumbled over this thread and tried it out. I had some issues with installing it (and with the bias "technical content and foreign language" (I'm not a programmer and not very good in english)) at the beginning, but now it's quite easy and the first tries looks very promising.
My CC24 shot needs to be better, but it's not that easy too. :)
After two evenings playing in C1, I'm happy whith my profiles and everything works fine. My issues with saturation and clipping are gone.
I'm writing here, just to thank you, but maybe this is interesting for you: Thanks to your documentation I was looking for the solux bulbs and saw that they have a 5000k version now (besides the 4700k). I can't order them to switzerland, but I hope to find something equivalent.
Anyway, thanks again (and also to the others in this thread). Your program made a hobby photographer really happy. I will try to improve my profile and knowledge about it. :)
greetings, seb
-
Solux MR16 (DIY black painted back) = 24 degrees (as listed, but the box is labeled as 26 degrees though) x claimed as 5000K x 12V x 50W = regular 12v voltage supplied by WAC Lighting gooseneck lamp (model 214 or so), no overdrive :
(http://s9.postimg.org/tmx78oknj/solux.jpg)
argyll spotread -a -H ... -> averaged 20 measurements from i1Pro2 :
370 373 377 380 383 387 390 393 397 400 403 407 410 413 417 420 423 427 430 433 437 440 443 447 450 453 457 460 463 467 470 473 477 480 483 487 490 493 497 500 503 507 510 513 517 520 523 527 530 533 537 540 543 547 550 553 557 560 563 567 570 573 577 580 583 587 590 593 597 600 603 607 610 613 617 620 623 627 630 633 637 640 643 647 650 653 657 660 663 667 670 673 677 680 683 687 690 693 697 700 703 707 710 713 717 720 723 727 730
0.32088214 0.29784569 0.29401690 0.30621851 0.32705995 0.35354967 0.38184586 0.40966783 0.43894256 0.46920782 0.49828912 0.52510086 0.54895098 0.56796508 0.58454266 0.59962353 0.61439182 0.62851956 0.64198793 0.65451496 0.66938343 0.68756174 0.70638357 0.72585339 0.74626633 0.76402709 0.77853445 0.79261878 0.80619712 0.81536129 0.82198649 0.82940335 0.83821059 0.84743544 0.85308833 0.85858870 0.86881333 0.87986246 0.88918143 0.89832669 0.90668527 0.91587696 0.92567861 0.93001969 0.93304846 0.94441652 0.95679285 0.95984252 0.96268567 0.97147879 0.97858487 0.98159013 0.98626197 0.99194637 0.99484559 0.99671250 0.99953539 1.00000000 0.99659206 0.99356489 0.99129215 0.98641275 0.97891014 0.97201763 0.96555810 0.95630950 0.94471420 0.93509700 0.92885205 0.92353666 0.91714001 0.91085846 0.90600584 0.90300038 0.90110816 0.90006470 0.90038321 0.90001229 0.89849141 0.89736427 0.89803213 0.89918212 0.89828002 0.89503026 0.89028515 0.88461872 0.87602444 0.86522913 0.85448318 0.84379143 0.83145690 0.81608462 0.79810954 0.78074840 0.76474757 0.74917290 0.73181067 0.71200825 0.69176042 0.67355084 0.65715186 0.64078524 0.62236194 0.60378076 0.59156921 0.58853493 0.58886248 0.58654337 0.58395931
-
Wow! That looks really impressive! Thank you AlterEgo!
-
After two evenings playing in C1, I'm happy whith my profiles and everything works fine. My issues with saturation and clipping are gone.
Great to hear that you got it working and have use of the software! If something goes wrong during the calibration process strange things can happen.
I haven't touched the code since the last release. Maybe I'll just re-label the release to 1.0 at some point. I'll give it some more time in use first though.
-
...
almost forgot to update... CC24 ColorChecker Classic November 2014 edition = I averaged many measurements with argyll, with patchtool XRGA and patchtool non-XRGA modes... I still see consistent differences between all 3 (averaged) that are more than expected... even Graeme posted somthing that there shall be none.
attached are the data files
-
and so I averaged all i1Pro2 measurements (from the original series)
-
Great to hear that you got it working and have use of the software! If something goes wrong during the calibration process strange things can happen.
I haven't touched the code since the last release. Maybe I'll just re-label the release to 1.0 at some point. I'll give it some more time in use first though.
With all the documentation and the builds, it's quite easy to use the program and even to play with. One thing I would change although in the C1 easy way section: Create the CC24.tif and the curve.tif just together at the beginning. For all the non-tech-user that is less confusing. :)
The difficult thing is to make a good CC24 file. Maybe you can advise to make the first profile with an example (and give a link to some files), just to understand the process first. I'm still working on a "perfect" version from my own camera.
I'm in touch now with solux now. They can send me some with UPS. Do you think it's worth to order a 5000k instead of a 4700k version? Or does it not matter at all? Or should I wait until a weekend with a sunny day to make an outdoor profile? And it's a shame to ask that: Should the sun shine on the CC24 (of course outside the family of angles and not directed toward the sky)?
I'm wondering how the profiles will work with the new C1... As soon as I'm at home I will update and check.
seb
-
C1 9.0 previews strange colors when selecting a dcamprof 0.10.x profile in the ICC drop-down menu, and when clicked the menu reverts to "none". It takes a second attempt to have it to load, and then display the expected colors.
Curiously 0.9 profiles seem to behave correctly ?
-
The difficult thing is to make a good CC24 file. Maybe you can advise to make the first profile with an example (and give a link to some files), just to understand the process first.
the raw (for A7R2 for example) from imaging-resource, the one @ ISO50 (because it has a better exposure /sensor-saturation-wise/ than ISO100) is decent enough for experiments... you still don't have illumination spectrum and you don't have a flat fielding raw - but the illumination (across the target) is quite even versus what I-R used to post some time ago...
-
Do you think it's worth to order a 5000k instead of a 4700k version?
note that 5K comes _only_ in 24/26 degrees version... 4700K comes also in a wider coverage 36 degrees version... based on your setup you need to pay attention to how even your target will be illuminated (CC24 ColorChecker Classic is a big one for example) across the surface with one solux mr16 lamp... test illumination evenness by shooting a flat uniform colored surface in your setup with your target position/contour marked and see (suggest rawdigger) how even the light is... even with flatfielding option you still want to do your best here
-
the raw (for A7R2 for example) from imaging-resource, the one @ ISO50 (because it has a better exposure /sensor-saturation-wise/ than ISO100) is decent enough for experiments... you still don't have illumination spectrum and you don't have a flat fielding raw - but the illumination (across the target) is quite even versus what I-R used to post some time ago...
I use the ISO100 version, because the A7RII has no real 50. Good to know your additional infos about the quality of the ISO100, so I have to change that. Actually I'm using this CC24 and one shot of my own as my basic profiles. If the one from "imaging-resource" gives me strange colors (most time in the reds), I'll switch to my self maded. Often it looks good then.
If I read right between the lines (also about the difference between solux 4700k and 5000k), it's easier to make a good outdoor shot, than to get a good indoor setup. Especially if I only profile one or two cameras with it and not for colorcritical work.
-
I use the ISO100 version, because the A7RII has no real 50.
for profile making it does not matter in this case, here (when using I-R raws, not when you make your own shots) ISO50 shot will simply give you more properly exposed ISO100 raw = profit !
-
If I read right between the lines (also about the difference between solux 4700k and 5000k), it's easier to make a good outdoor shot, than to get a good indoor setup.
you can certainly try - some people make some kind of a "booth", like from a big cardboard box covered inside with black matte material to place the target inside to shot outdoors to reduce parasitic reflections from sides/bottom, etc...
-
C1 9.0 previews strange colors when selecting a dcamprof 0.10.x profile in the ICC drop-down menu, and when clicked the menu reverts to "none". It takes a second attempt to have it to load, and then display the expected colors.
Curiously 0.9 profiles seem to behave correctly ?
I have compared my profiles and I have some differences within 8.3.4 and 9.0.0 when exporting them to jpg.
generic: the C1 generic A7RII file (no differences)
profile: dcamprof profile from the imaging resource file (slight differences)
self: dcamprof profile from my own made CC24 shot (big differences in saturation)
As the tiff files are made out of C1 and phase one explicid says they improved the process engine, I think we have to re-export the tiffs and calibrate new profiles anyway. I will do that and compare it again. But this could take a while, because I don't have much spare time this week.
Edit: I don't have any differences if I look on the same Profile inside of the new version but with different engines (v8 engine vs. v9 engine). Did you upgrade your pic?
-
For now I'm seeing many weird things, even with P1 profiles (IQ daylight looks really low on saturation compared to the flash profile for instance), and lots of OS X beachballs of death...
Guess I'll wait it stabilizes a bit before building profiles again.
-
almost forgot to update... CC24 ColorChecker Classic November 2014 edition = I averaged many measurements with argyll, with patchtool XRGA and patchtool non-XRGA modes... I still see consistent differences between all 3 (averaged) that are more than expected... even Graeme posted somthing that there shall be none.
I said that the i1pro2 is native XRGA. That doesn't mean that there is no difference between XRGA and legacy Gretag mode, as X-Rites library is used to convert the other way too. I would expect little difference between patchtool XRGA and Argyll, and differences between patchtool GRMB and Argyll/XRGA for the i1pro2. But I'm not sure that taking multiple measurements and averaging them is sufficiently accurate to come to any conclusions.
The only way I got consistent enough measurements for this type of analysis (http://www.argyllcms.com/doc/i1proDriver.html) was to arrange it so the instrument wasn't moved on the color patch between comparisons, and making sure that the lamp had cooled down for a consistent time between readings.
[ This meant doing: calibrate using Argyll, change drivers, calibrate using the OEM driver, place on patch and measure, change drivers, measure using Argyll using -N. ]
-
That doesn't mean that there is no difference between XRGA and legacy Gretag mode, as X-Rites library is used to convert the other way too.
why would X-Rite go to legacy Gretag mode through software corrections of native XRGA data instead of going to legacy X-Rite mode ? it seems that X-Rite when introducing XRGA was clearly favoring their own legacy X-Rite instruments vs legacy Gretag - buy that logic in non XRGA mode they shall emulate their own legacy calibration...
-
Do I understand it correctly that the C1-9 issues may be due to stability problems with C1-9 and we should wait a while and see what happens? I know from experience that Capture One's first major releases are often a bit buggy to start with. I only have C1-7 myself and as I don't really use C1 regularly I did not plan to upgrade if I don't have to. Let me know if there are issues.
-
With all the documentation and the builds, it's quite easy to use the program and even to play with. One thing I would change although in the C1 easy way section: Create the CC24.tif and the curve.tif just together at the beginning. For all the non-tech-user that is less confusing. :)
The difficult thing is to make a good CC24 file. Maybe you can advise to make the first profile with an example (and give a link to some files), just to understand the process first. I'm still working on a "perfect" version from my own camera.
I'm in touch now with solux now. They can send me some with UPS. Do you think it's worth to order a 5000k instead of a 4700k version? Or does it not matter at all? Or should I wait until a weekend with a sunny day to make an outdoor profile? And it's a shame to ask that: Should the sun shine on the CC24 (of course outside the family of angles and not directed toward the sky)?
I'm wondering how the profiles will work with the new C1... As soon as I'm at home I will update and check.
seb
Thanks for the feedback, I'll look into the workflow.
If you have the ability to run the lamp on overdrive the 5000K is probably not adding any value, as you can push the 4700K lamp to that temperature anyway. With a fixed 12 volt lamp fixture I'd get the 5000K if I could. It's generally a bit easier to avoid glare in an indoor setup than outdoors, but with the CC24 and the glare compensation feature of DCamProf the outdoor result should be good too.
If you want a ~5000K shot the midday sun should shine on the CC24. If you want ~6500K shot the CC24 should be in the shadow. If you're like me located in Sweden, or even worse northern Sweden this time of year I don't think we ever get proper midday light so I'd prefer shooting indoor with a Solux.
-
Do I understand it correctly that the C1-9 issues may be due to stability problems with C1-9 and we should wait a while and see what happens? I know from experience that Capture One's first major releases are often a bit buggy to start with. I only have C1-7 myself and as I don't really use C1 regularly I did not plan to upgrade if I don't have to. Let me know if there are issues.
myself I did not see any issues (PC/Win8.1x64) - but then I do not have use catalogs... I only use one dummy session, as I use C1 a-la ACR.
PS: P1 dealer = https://captureintegration.com/capture-one-pro-9-0-tested-approved/ ... they tend to write if bugs happening with new releases, even they are P1 dealer.
-
I will repost here what David from P1 got us from a different LuLa topic for posterity, Esben was posting along those lines before, but never harms to have it again :
Please see below from Esben at Phase One.
Anyways, the thing is that we actually do most of the work in the native camera space. In fact, we can pretty much work in the native camera space all the way to the output file. The rule is that if you can export an image with an embedded profile, then we can work in the camera space all the way. In that case, the “colorimetric interpretation” is done by the program reading our output file (e.g. PhotoShop).
In fact, if you embed the camera profile and export as a tif, we embed tag 301 (TIFFTAG_TRANSFERFUNCTION) which you can use to get back to a colors-pace which is closely related to the data acquired by the sensor. Obviously, the data has been heavily processed, but the processing is controlled by the user. This feature is targeted at scientific applications and 3rd party software to build color profiles.
The gamut you can see in the camera profile does not limit the gamut used for internal processing in Capture One. While typically large, the gamut is mostly limited because it is intended to be converted to other profiles afterwards. Also, LUT-based ICC profiles which use Lab is the PCS, have a gamut limitation. However, the conversion from the camera space (the “colorimetric interpretation” if you will) is based on ICC profiles.
Now, there are a few exceptions where we are forced to convert into a standard color space:
1) Local color edits
2) Conversion to black&white
There are tradeoffs with this approach. While it allows us to extract just about everything provided in the file, it also means that we need a significant effort to get the best possible quality for a camera. However, it does mean we do not limit ourselves to standardize on a specific internal color space.
Kind regards,
Esben H-R Myosotis
Formerly “EsbenHR” on Luminous Landscape
-
Thanks for the feedback, I'll look into the workflow.
If you have the ability to run the lamp on overdrive the 5000K is probably not adding any value, as you can push the 4700K lamp to that temperature anyway. With a fixed 12 volt lamp fixture I'd get the 5000K if I could. It's generally a bit easier to avoid glare in an indoor setup than outdoors, but with the CC24 and the glare compensation feature of DCamProf the outdoor result should be good too.
If you want a ~5000K shot the midday sun should shine on the CC24. If you want ~6500K shot the CC24 should be in the shadow. If you're like me located in Sweden, or even worse northern Sweden this time of year I don't think we ever get proper midday light so I'd prefer shooting indoor with a Solux.
I can remember me on some days in November in Sweden (Borås)... But on the other hand, the light in the Swedish summer months is unique and lovely! :)
We have a nice day today and I'll try a shoot in the lunch break. Switzerland isn't the equator, but actually I'm shooting most of my pics here. So this should be allright for my work. On the other hand, I may order some Solux anyway. Thanks for your explenations.
I made some new profiles yesterday and as I suggested, the result ends up in an identical picture, as with the old profile on C1Pro 8. So it looks like DCamProf works fine with the new version too, but you have to make new profiles for the v9 version.
But at the moment, C1Pro 9 looks not very stable to me. I can't save a default profile. If I import a new pic, it shows me sometimes the generic one, sometimes old profiles, that were the default once (but are deleted a long time ago) and sometimes the new one, but with a strange string of numbers. Also, if I close C1 and reopens it, the made processing is gone sometimes. Same with the keywording, it shows strange combinations with cascaded keywords. And reorganising them, ends up in a mess.
I'm glad now, that I accidently bought a new licence (50€) instead of an upgrade (40€). So I can use C1Pro 8 until they get fixed all that. :)
-
I cleaned up my catalogue and C1 works now as it should. I don't have any issues any more. It looks like some wrong linked pics made the program unstable.
But back to the topic: I shot the CC24 yesterday in the sun on high noon. And made a C1Pro v9 ICC-profile with DCampProf v0.10.4 for the Sony A7RII. The profile looks great. It's almost identical with the shot from imaging resource. Mine has a slightly darker blue.
If you want to try or even use it by your own, you can find it in the attachement.
Thanks again, Anders, for this program.
-
Since the output from my strobes is not super consistent, and recording the illuminant for each shot not really convenient, I may give the solux route a try too.
What do you use to power those lamps ? And what are the reasons nobody favors a standard repro setup with 2 of them, and goes with only 1 lamp plus flat-fielding instead ?
Torger, could you please briefly let us know why white-balancing isn't necessary for the exported linear TIFFs ?
-- OT --
And finally, is there anyone here who could make measurements for some of my charts (CC24, CCSG, IT8, DT4) with an i1pro2 or similar ? Someone in the EU would probably be more convenient, we could ship there more quickly. Of course this would be a paid job (invoice needed).
-- OT --
Thanks.
-
What do you use to power those lamps ?
I posted what I use for Solux 5K few posts earlier in this topic
And what are the reasons nobody favors a standard repro setup with 2 of them, and goes with only 1 lamp plus flat-fielding instead ?
why do you think (or rather sure) that you will not need to flatfield 2 of them ?
one 24/26 degree lamp, Solux 5K 50W can easily illuminate a small target (for my purposes I tried many - but X-Rite Passport is hard to beat, even with that relatively shiny hard plastic shell that is has... I do not own mini CC24 unfortunately, missed the boat to buy when they were sold) in a regular room with <= 1.5-2.5% maximum (between min and max illuminated patches) unevenness (as calculated by collecting grid samples with raw digger over a regular surface where target is located with raw rgb channels)... and you further flatfield that.
-
And finally, is there anyone here who could make measurements for some of my charts (CC24, CCSG, IT8, DT4) with an i1pro2 or similar ? Someone in the EU would probably be more convenient, we could ship there more quickly. Of course this would be a paid job (invoice needed).
why don't you buy a used i1Pro (not more expensive i1Pro2) or EFI (rebranded i1Pro) on eBay ? $200-300 a piece.
-
I posted what I use for Solux 5K few posts earlier in this topic
Ok, thanks, missed that.
why do you think (or rather sure) that you will not need to flatfield 2 of them ?
I already flat-field charts shot with 2 strobes, I'd probably do the same with 2 solux. My question is why people seem to use only 1 lamp, and rely heavily on flat-fielding to get a balanced exposure.
why don't you buy a used i1Pro (not more expensive i1Pro2) or EFI (rebranded i1Pro) on eBay ? $200-300 a piece.
I already own one, but according to W. Faust my rev A version has issues with high densities (confirmed after some tests with a chart he sent me). Can't really justify buying a second one... I will probably use it again, and compare the results with your recent measurements for the colorchecker though.
-
I already flat-field charts shot with 2 strobes, I'd probably do the same with 2 solux. My question is why people seem to use only 1 lamp, and rely heavily on flat-fielding to get a balanced exposure.
and what is your unevenness with 2 Solux @ angle 'd be ? note that when you use 1 Solux - you can shine straight and aim camera @ angle... when you have 2 Solux you shine each @ angle - hence way more unevenness, albeit you hope that intersecting 2 circular fields will provide a more even field... and with Solux I think you need to keep them closer to the target versus strobes or halogen, powerwise - more unevenness (versus shining from a bigger distance)...
otherwise there is a school of thought (namely Iliah Borg as far as I recall what I read) that gives a reason for not using 2 lamps as the differences in WB / WB unevenness across the field... I never used 2 lamps, so I never tested that myself...
-
I'm in the next round of "Sebbe vs. DCamprof". I'm still looking for a double "Sebbe and DCamprof". But for that, I need to fight me some respect with it. :)
I made a mistake creating my last profile. Don't ask! :) But yesterday I was creating it correctly and now I have a question:
The new profile looks smoother, but has a light green color cast. So I added to my cc24 pic in C1 a 125/126 point on the green curve. After that, I saved my CC24.tif/curve.tif and profiled with them. It looks great now. But I wanted to know, if there is a better (or more sophisticated) way to do that? Like the lut-curve.json but for each color curve. You have the personal look section, but I don't get it. May you can give me an example?
(I did an update on my post above, so the corrected profile can be downloaded there)
-
I assume you're using a preset camera white balance when the green cast occurs?
There shouldn't be any green cast if the DCamProf white is matching the white balance setting of the camera, that is if you color pick the white patch, or for the CC24 I think it's the second neutral patch that's the most neutral (D02, when you run you can see which patch is picked by DCamProf, or you can steer it with the -b parameter).
However the camera's daylight preset may have some other multipliers that doesn't play well with a neutral profile. Cameras usually do, but if it doesn't one may need to make some manual look adjustments.
I'd look into the temperature/tint adjustment look operator first (SetTemperature) rather than curves. The look operator stuff is indeed a bit messy to get into, it really needs a GUI to be easy to work with, but at this point I'm not planning to develop any so it is what it is.
Try something like this, but keep temperature (yellow-blue axis) at 5000 and adjust the tint (green-magenta axis)
{ // Warmup
// Reference temperature (no change temperature) is always 5000K 0 tint.
"Operator": "SetTemperature", "TempTint": [ 5400, 5 ],
"BlendRGB": true, // RGB blending (instead of default JCh) used here as it gives better result for this operator
"Blend": [
{ // limiter to avoid DCP LUT hue discontinuity issues, and make a smoother rolloff to neutrals
"X": "HSV-Saturation",
"XRange": [ 0, 15 ],
"Curve": {
"CurveType": "RoundedStep",
"CurveHandles": [ [0,0], [1,1] ]
}
}
]
}
Extend the data-examples/ntro_conf.json with this look operator and play around a bit with it. See data-examples/ntro_lookop_conf.json to see exactly how you insert the look operator.
-
I was using with my CC24 shot in C1:
ICC profile: "Phase One Effects: No Color Correction", Select Curve: "Linear Response", "16 bit TIFF", "Embed camera profile". I didn't change WB (I think in engl. it's "as shot"), because you wrote in the documentation, that it would not affect the profile process.
Thanks for the "code" and the hints. I will try that, as soon as I get to it.
If I'm right, and it is just a WB issue with my current shot, I have three options:
- Just make a profile and correct the WB-green-magenta axis every time slightly to magenta,...
- use the code to adjust the green-magenta axis or ...
- use the -b operator to choose another WB-point and check, if that may give the same effect
Correct?
I will try all of them and compare them to my unsophisticated way and the as-is-profile. :)
-
Yes the WB in the chart shot doesn't matter as DCamProf rebalances it in any case unless you use the -B parameter, then the profile will also correct the white balance (only works for ICC, not DCP). In theory you might want to do that, but normally it's only applicable to reproduction work.
This is the first time I hear reports about white balance issues so I get a little bit suspicious about that there might be some other problem. What light are you using in your setup?
I was using with my CC24 shot in C1:
ICC profile: "Phase One Effects: No Color Correction", Select Curve: "Linear Response", "16 bit TIFF", "Embed camera profile". I didn't change WB (I think in engl. it's "as shot"), because you wrote in the documentation, that it would not affect the profile process.
Thanks for the "code" and the hints. I will try that, as soon as I get to it.
If I'm right, and it is just a WB issue with my current shot, I have three options:
- Just make a profile and correct the WB-green-magenta axis every time slightly to magenta,...
- use the code to adjust the green-magenta axis or ...
- use the -b operator to choose another WB-point and check, if that may give the same effect
Correct?
I will try all of them and compare them to my unsophisticated way and the as-is-profile. :)
-
the sun! :)
I never expect color shift due to the simply profiling with dcamprof. But there are so much variable light when shooting outdoor, that it would be a surprise, if there is no such issues. I did a shot at the same position but with a cloudy sky today and will compare it.
The lightsource is perfect, the setting is difficult. But if I can get rid of setting-issues with slight adjustements, I think, I will get a good profile anyway.
-
Hej Torger
Didn't had much time these days. But made some tests this morning. The subtle color shift seems to be from the shot setting itself. I tried the -b and the WB way to correct it, but both didn't end where I want to. The issue isn't a wrong balance, it's just a slight color shift in light (from reflections maybe?), while making the profile. So, the way with the dcamprof adjustment tool operator wouldn't the right too.
So I went back to my method: Adjusting all 3 curves (RGB) with a good reference. First I spread in C1 the curve tool over the whole display. The bigger the curve tool is, the finer the steps will be on a single point, when pressing the curser buttons. With that I could make for example a 127 to a 127.25.
I made just slight adjustments on 1 or 2 points within every of the three curves. Not more than +-0.75. These steps I copied to my CC24 shot and negate them (a +0.5 would be a -0.5). After that, I generated my tiffs (CC24 and curve) the normal way and profiled them with dcamprof. It looks like I have a profile now, that looks very good. It's for the A7RII and you can find it attached (if someone want to use it).
I will stop now with improving it and go on with shooting (yeah!) and reading more about theory and mechanic behind RAW files and how to process them. :)
Tack så mycket och god Jul!
sebbe
Edit: Today I had time (and weather) to shoot a new one, and it works without the RGB adjustments. So you can find both in the attachement section. I will use the new one now. :)
-
I'm looking for colorchecker shots for a few cameras from before imaging resource started posting raw samples:
Canon G9
Olympus SP-510UZ
-
Cross posted here...
___
Some people were interested in charts shot at ISO 50 with the A7rII, here a CCSG and IT8.7 :
https://www.dropbox.com/s/qitmp7aanzg72us/FH-A7_charts_ISO50.zip?dl=1
Shot with strobes around 5000K (.sp files included for each) + flat field pic. The folder includes the capture one v9 settings too.
CCSG is post Nov 2014, IT8 is from W. Faust (charge R131007).
Since I don't use my A7rII much, I have not bothered to create some ICC profiles yet.
-
Are Adobe standard profile colormetric with the tone curve deactivated? I considering options for my shots taken with cameras I no longer own or can find colorchecker shots for...
-
can find colorchecker shots for...
you can always try I-R website
-
Are Adobe standard profile colormetric with the tone curve deactivated?
Adobe has changed their profile design philosophy over the years, and possibly they have had different strategies for different cameras too. I have looked at quite many profiles for various reasons and seen that their design differs a lot, but I have not really tried to figure out any pattern so I can't say anything generic like "every profile before Lightroom version XX is like YY", I simply don't know.
Some of the older profiles might work in that way, but recent profiles for higher end cameras are surely not. Adobe seems to have decided to have their own "look" and design profiles to match that look, but still they make cameras look quite different, the desaturated look of the A7r-II and Pentax 645z for example is not matching the typically more saturated look. I don't really know what design strategy they have.
DCP's allow for a layered approach, matrix that is as colorimetric as possible linearly, a huesatmap that corrects further, and then a curve+looktable that adds a look on top. That would give you the property you're after, and that was how DCamProf made its profiles, and you can still make it that way if you want to, but it turns out that you get clipping issues with extreme colors this way, most typically demonstrated with deep blues.
So a modern Adobe profile (and indeed DCamProf with default settings) will instead make a matrix that compresses colors to fit within prophoto, and then stretches thing in place with the LUTs (including gamut compression). That is there's no colometric "layer" in the profile.
If Adobe would have designed the DCP format from scratch today it would probably be a bit different. The more or less subtle clipping issues is something that the early profiles did not care about, while the modern profiles are designed to work around limitations in the DCP format itself, and DCamProf's default action is to do exactly that.
-
you can always try I-R website
I did, no luck. :/
Some of the older profiles might work in that way, but recent profiles for higher end cameras are surely not.
Maybe I'm in luck then, the cameras I'm thinking of were included in the early "beta 2" batch IIRC. I can report back if I find anything interesting.
EDIT: Where can I find a good all-round tone curve for RawTherapees perceptual mode?
-
I did, no luck.
what is the camera in question ?
-
what is the camera in question ?
Canon G9 and Olympus SP-510UZ.
-
Where can I find a good all-round tone curve for RawTherapees perceptual mode?
Try this file: http://www.ludd.ltu.se/~torger/photography/files/acr.rtc
It's an (almost) exact copy of Adobe's default curve, and it's a fine curve.
-
Try this file: http://www.ludd.ltu.se/~torger/photography/files/acr.rtc
It's an (almost) exact copy of Adobe's default curve, and it's a fine curve.
It keeps more saturation in highlights in perceptual mode than the the built-in tone curve in Adobes profile(tone curve enabled under DCP). I can appericiate the difference on slightly overexposed portraits. Nice!
EDIT: I noticed some changed hues on saturated colors in my shots, I can't tell if the changed hues are more accurate on top of Adobes profile but so far nothing looks objectionable.
-
It keeps more saturation in highlights in perceptual mode than the the built-in tone curve in Adobes profile(tone curve enabled under DCP). I can appericiate the difference on slightly overexposed portraits. Nice!
EDIT: I noticed some changed hues on saturated colors in my shots, I can't tell if the changed hues are more accurate on top of Adobes profile but so far nothing looks objectionable.
The perceptual curve in RawTherapee is basically a copy of an older version of the DCamProf built-in tone reproduction operator. As it's an older version it does have some issues with clipping extreme colors, you can have very deep saturated blues turn magenta. At some point I'll update the perceptual curve in RT to come closer to the current DCamProf, but there's also a performance requirement when using it in RT so it's not feasible to bring over it unchanged, ie it's quite much work.
-
Sorry if this has already been covered. I tried making a quick profile for my Pentax *ist DS from Nikon D50 SSFs which uses the same sensor with these commands:
dcamprof make-target -c D50_istDS-ssf.json -p cc24 target.ti3
dcamprof make-profile -c D50_istDS-ssf.json target.ti3 profile.json
dcamprof make-dcp -n "Pentax *istDS" -d "D50 Pentax ist DS neutral" -t acr -g adobergb profile.json D50-Pentax-ist-DS-neutral.dcp
dcamprof make-dcp -n "Pentax *istDS" -d "D50 Pentax ist DS neutral+" -t acr -o neutral-plus.json profile.json D50-Pentax-ist-DS-neutral-plus.dcp
I'm pleased with what I'm seeing as i A/B with the Adobe standard profile, the only "fault" is a relatively high lightness for very saturated colors pushing them into clipping. Probably more of a subjective issue with the limited display gamut being at the root of the issue. I uploaded my profiles and a few example shots if you want to take a look: https://www.sendspace.com/filegroup/Gn8pVnV8JQ8SkaU7%2ByyPUKnu150DDgcQErkX04hq%2FdA
EDIT: I posted a general Adobe Standard vs. DCamProf Natural+ gallery: http://imgur.com/a/Dzqur/all
-
Some commercial profiles have a very strong gamut compression to avoid such issues. DCamProf has gamut compression too when you for example provide "-g adobergb", but it's quite weak. I may at some point look into making a stronger compression to make profiles that work more similar to the commercial when it comes to high saturation colors.
To make a very robust profile to work in all sorts of light conditions and colors you should make an unnaturally desaturated profile, which is a strategy Adobe is using on some cameras, like the A7r-II and Pentax 645z. This means that you don't get "finished" color to start with though.
My current ability to work with DCamProf is limited though so it's probably not happening much new things with it for a while.
That said I've looked at the example files you provided and to me it looks like the clear difference between DcamProf/Adobe for the D50 with your example pictures is that Adobe turns the high saturation reds towards orange and DcamProf stays red. The amount of clipping for AdobeRGB color space is about the same though, so to me it looks like it works as it should for those images. I'm a bit rusty on this, but if I remember correctly the saturated reds going orange is part of the "Adobe" designed subjective look, so you find that in many(most?/all?) of their profiles.
Note that some clipping is intended even with gamut compression. While you can easily make a "mathematically correct" gamut compression so there's never any clipping it doesn't make images that look good. If there's only one channel of RGB that's clipped the color usually works well anyway, and you get some more range to play with in terms of brightness. If you want to test and see how a gamut compression that doesn't clip looks you can set
"RGBChannelHighLimit": 1,
"RGBChannelLowLimit": 0,
in the gamut compression configuration. Read more in the example file data-examples/ntro_lookop_conf.json
-
I'll check that out. While I'm working on perfecting my SSF profile (dual-Illuminant, checking for optimal lut relaxation, WB shift fix) can you make a recommendation of a virtual target better than CC24, if such a thing can be said to exist?
-
I'll check that out. While I'm working on perfecting my SSF profile (dual-Illuminant, checking for optimal lut relaxation, WB shift fix) can you make a recommendation of a virtual target better than CC24, if such a thing can be said to exist?
While the software itself is rich in SSF features I haven't worked that much with them myself as most my focus has been on making profiles the traditional way when I've made own profiles.
You can play a little with the built-in targets, but to make an "ideal" target one would probably want to have a balanced set of not too many spectral measurements, so focus is put on "important" colors. I have no such ready-made target. The CC24 is a good starting point for sure. As long as the target has decent coverage, like the CC24 has, the target has less effect on the result than one may think. Why commercial bundled profiles look so differently is due to subjective design and how colors are modulated to match the film curve, the target used has less importance.
-
New Adobe Standard vs. DCamProf Neutral+ gallety: http://imgur.com/a/Dzqur/all
-
New Adobe Standard vs. DCamProf Neutral+ gallety: http://imgur.com/a/Dzqur/all
Subtle saturation and some color differences. But it's not clear which is the Adobe and which is the DCamProf version. Could you add that info?
Cheers,
Bart
-
Subtle saturation and some color differences. But it's not clear which is the Adobe and which is the DCamProf version. Could you add that info?
Cheers,
Bart
The first image is always the Adobe Standard version. Easy to tell by the darker shadow tone. Try the fullscreen link for easy A/B-ing.
EDIT: Differences that jump out at me: More salmon-red in the shadows, less cyan blues, much less blue purples, and orange-red less shifted to red.
-
The first image is always the Adobe Standard version. Try the fullscreen link for easy A/B-ing.
EDIT: Differences that jump out at me: More salmon-red in the shadows, less cyan blues, much less blue purples, and orange-red less shifted to red.
It's a quite big difference on the train, blue in Adobe, purple with DCamProf. Which profile is closer to the truth? I've seen that with some sensors, like the NEX6 it's been very hard to get the hue correct on deep blues, I'm not sure the CC24 target is ideal for that. With the NEX6 I made a manual target adjustment to get a good hue, it's described here how you do that: http://www.ludd.ltu.se/~torger/dcamprof.html#make_profile_deep_blue
The other examples looks as I would expect them to do, cyan skies is an "Adobe look", saturated reds also tends to be orange, it's warmup thing in their look. Seems like it's a slightly different contrast curve that the Adobe has darker shadows.
It's normal that differences are relatively small, especially for low to medium saturation colors.
-
Which profile is closer to the truth?
Definitely the DCamProf version. Some pretty accurately rendered shots [EDIT: As far as the purple hue goes]of the same type of train: http://www.postvagnen.com/forum/index.php?mode=thread&id=626184 http://postvagnen.com/forum/index.php?id=699902
-
Definitely the DCamProf version. Some pretty accurately rendered shots of the same type of train: http://www.postvagnen.com/forum/index.php?mode=thread&id=626184 http://postvagnen.com/forum/index.php?id=699902
Then your definition of accurate is not in line with what I've seen in nature for over 50 years first as a fine art painter and now color matching shots of colored objects taken near my display for A/B comparison without relying on memory of the scene.
But at least I get to see what others get with Adobe Standard that I don't get with my Pentax K100D vs a CC chart generated custom DNG profile.
-
Then your definition of accurate is not in line with what I've seen in nature for over 50 years first as a fine art painter and now color matching shots of colored objects taken near my display for A/B comparison without relying on memory of the scene.
But at least I get to see what others get with Adobe Standard that I don't get with my Pentax K100D vs a CC chart generated custom DNG profile.
If you think my statement wasn't limited to the question of the purple hue of trains then you are reading way to much into it.
-
If you think my statement wasn't limited to the question of the purple hue of trains then you are reading way to much into it.
Torger asked which of the image samples you linked to affected by Adobe Standard vs DcamProf was closer to the truth. You said the DcamProf was closer to the truth.
That's the statement I'm addressing according to how you define truth. Maybe you need to be more specific on how you define accurate to truth colors.
It boils down to how much work do you want to put into getting accurate colors. I'm seeing a thread that is disproportionately requiring way too much time over just fixing it in post with HSL and white balance.
-
If we're all going to be subjective on what colors should look like I can guarantee with certainty a profile whether canned, custom DNG or a DcamProf version is not going to fix subjective-ism.
One says a color should look a certain way because it's accurate according to the photographer's memory of the scene which adds another variable to color definitions and another disagrees. How can a profile possibly fix this or even come close to defining memory colors?
It's a waste of time IMO.
-
For your purposes it may be a waste of time. For me it helps my work flow. Having accurate colors from the start saves time on troubleshooting wrong colors when you dont have the convenience of having the object you are reproducing next to your monitor.
-
Having accurate colors from the start
how do you know that they are indeed accurate though ?
-
how do you know that they are indeed accurate though ?
I don't, by default smootness is prioritized over acuracy and lightness is uncorrected. Nevertheless The DCamProf generated profile is closer to my memory of the scene.
-
In this particular context it was two very different hues. Blue vs purple. You don't need very accurate color memory to remember if a color was purple or if it was blue. Unless it's a very strange light condition I expect a camera profile to at least be in the same ballpark concerning hue.
For true accuracy we need a controlled light condition and work reproduction-style, of course.
But if you just add some sense of proportionality you can also make judgments of general purpose profiles in varied conditions.
-
While waiting for better light (and weather) for shooting a charts I made a profile using EXIF matrix data from my Nexus 5 DNGs. Another gallery: http://imgur.com/a/ydILx/all
-
Hi. Does anyone have a A7R (original, I) profile made with DCamProf, preferably dual illuminant.
I was contemplating on a better place to ask this question, but failed.
-
Are these colorchecker shots useful? www.dpreview.com/reviews/image-comparison I notice some uneven lighting on the tungsten version.
-
Are these colorchecker shots useful? www.dpreview.com/reviews/image-comparison I notice some uneven lighting on the tungsten version.
I have never tried them, but you can probably make ok profiles from them despite the uneven light, just make sure that you don't try to match lightness too much as that axis is not to be trusted with such uneven light. Imaging Resource's test shots are much better so if the camera you're interested in has been tested there it's a better choice.
-
I have never tried them, but you can probably make ok profiles from them despite the uneven light, just make sure that you don't try to match lightness too much as that axis is not to be trusted with such uneven light. Imaging Resource's test shots are much better so if the camera you're interested in has been tested there it's a better choice.
Thanks, I went with the Imaging Resource shot (http://www.imaging-resource.com/PRODS/sony-rx100-ii/RX100IIhVFATS_F8.ARW.HTM) for profiling.
I found another example where DCamProf produces a subjectively less pleasing rendering or very saturated reds, if you have time to study the issue: http://filebin.net/0s1iarkd77
EDIT: One more: http://filebin.net/2zwxlq2l93
-
Thanks, I went with the Imaging Resource shot (http://www.imaging-resource.com/PRODS/sony-rx100-ii/RX100IIhVFATS_F8.ARW.HTM) for profiling.
I found another example where DCamProf produces a subjectively less pleasing rendering or very saturated reds, if you have time to study the issue: http://filebin.net/0s1iarkd77
EDIT: One more: http://filebin.net/2zwxlq2l93
Thanks very much for the examples. I've had a quick look and notice that it's artificial lights, likely narrow band, very high saturation. These colors are the most difficult as they excite the camera filters in unpredictable ways, the CC24 patches are not close to it. In general commercial profiles often have better robustness when it comes to this type of "extreme colors". This robustness seems to come at a price though, for example unrealistically low saturation in the normal range (to make a smoother transition into extreme colors).
I have done quite a bit of work to handle extreme colors better in DCamProf but I don't think I'm fully there yet, so I usually recommend to have another profile laying around to try when you have strange lighting conditions like this.
At some point I may have a look and try to do further work on these types of light sources, but it's surely not a "quick fix", and I'm not sure it's even possible to fix without certain sacrifice in the normal range (I suspect the only robust solution is a pretty strong gamut and wide compression). So I can't provide a fix for it at this time unfortunately.
With narrow band emissive colors like this one cannot really strive for correctness, as it will hurt performance of normal colors. So what one want to achieve is smooth gradients and avoid ugly clipping.
With all that said looking at your pictures DCamProf seems to do as intended. When a color gets very very bright and saturated there's a tradeoff to either clip or to desaturate. By desaturating you get better tonality (more visible shades) but with a less realistic desaturated color. I worked quite a bit with this to get a balance based on test images of red roses Bart sent to me (which I artificially converted to different colors to test highlight transitions for various colors).
In your particular image I too prefer the flatter rendering of the portrait profile provided, but in other cases, such as the roses or other detailed natural subjects, I prefer to have visible detail left rather than just flat color. The problem is that it's not possible to make one profile that makes the best tradeoff for all image material.
I'm very interested in hearing what you think about these particular images. I assume that you don't like that DCamProf desaturates the bright parts and prefer the more saturated rendering of the provided Portrait profile, but I'd like to hear so it's not some other issue I'm missing.
-
Added here crops to show what I was talking about above.
The portrait profile gives a smoother flatter rendering of the bright reds, while DCamProf makes tonality differences more visible in those areas. For this particular subject I prefer the portrait profile result, but as said when testing for various subjects I found it better overall to keep detail at the cost of saturation/uniformity.
If you prefer a different behavior this can be controlled using the neutral tone operator configuration, the "Curve" section. (See data-examples/ntro_conf.json). It's been a while since I was deep into the code so I'm myself a bit rusty on it but I think you should be able to get a similar result as "portrait" by changing KeepFactor to 1.0. From the documentation:
// The curve is always applied in the luminance channel, but the output luminanince can be derived from a
// RGB-HSV curve, meaning that the resulting lightness will then be the same as a pure RGB-HSV curve.
// For low saturation colors there's often a perceptual advantage of using the RGB-HSV curve, otherwise the shadow
// dip of an S-curve may look too dark and a bit dull. For high saturation colors the RGB-HSV curve causes tone
// compression (especially in the red range), ie poor tone separation and a less realistic look.
// Therefore we here control a transition from the RGB-HSV curve to a pure luminance curve based on chroma of
// input color.
-
I'm very interested in hearing what you think about these particular images. I assume that you don't like that DCamProf desaturates the bright parts and prefer the more saturated rendering of the provided Portrait profile, but I'd like to hear so it's not some other issue I'm missing.
Yes, the extreme reds appear desaturated and may shift earlier to orange and yellow. I've also seen the rose shots and I think you ended up with the best possible rendering for that particular subject. It's strange to realize that sometimes less color detail is preferable. I will experiment with KeepFactor.
EDIT:KeepFactor 1.0 did indeed make rendering closer to the portrait profile.
-
Hi guys,
I've been following this thread with interest and would like to thank you @Torger for making a profiling tool for C1 and @AlterEgo for providing a Windows compile. So far, I have had mixed results to be honest. The profile I get by following the ICC tutorial clips certain strong colours (as has been discussed here recently). Let me know if you would like to have a look at some examples.
As a passionate amateur I have no access to a studio, fine art printer or controlled light setup (most of my photos is taken outside, at night or in mixed lighting anyway) and have to rely on CC Passport and web resources. I bought CC SG but will be returning it as I understand the glare makes it useless outside.
I have downloaded RawDigger and would like to test how it performs with DCamProf as I like having a GUI and it has some export options. So far, I have not found any information regarding using the .txt file from RD in creating a profile. Could someone please help me out with setting this up and the commands for getting it to work? Optimal RD settings?
One last question for now: Any suggestions as to creating a robust profile for different lighting conditions with the above limitations in mind? Or do I need to create one for each standard temperature for example?
Thanks for any help!
Ole
-
I have downloaded RawDigger and would like to test how it performs with DCamProf as I like having a GUI and it has some export options. So far, I have not found any information regarding using the .txt file from RD in creating a profile. Could someone please help me out with setting this up and the commands for getting it to work? Optimal RD settings?
well, you need to read the manual and rawdigger website first... for exampe for my DCamProf usage (actually for the matlab script that approximates SSF/CMF to be used in DCamProf) when exporting the raw data ("selection grid" -> "save samples to file") I do not use WB or data scaling/gamma, I do use flat fielding naturally ... if you are using C1 then you can combine rawdigger and makeinputicc GUI frontend for argyll to stay fully GUI (not as flexible as DCamProf of course)... in that case rawdigger must use WB and data scaling/gamma (either around 1.8 for C1 or you can modify data yourself with actual tf curve data from C1 for your camera model, etc) when outputting CGATS for argyll.
-
So far, I have not found any information regarding using the .txt file from RD in creating a profile.
you need to combine the raw data exported from RD (after flatfielding, WB and scaling/gamma - whatever is necessary for your specific workflow...) with the target description (like cieXYZ/D50 numbers for patches) ... patch for patch matching
-
I want to report about a few shots I have where KeepFactor 1.0 made some extreme purples look too dark, whereas the default KeepFactor 0.0 looked more realistic to me. If you recall, for extreme reds we preferred KeepFactor 1.0. I hope everyone is having a happy easter!
-
Not to be conflated with the issue brought up in my previous post, I noticed extreme purples were rendered as "blobs":
(http://i.imgur.com/3d2SP1Q.jpg)
Top: DCamProf Neutral+, default settings
Bottom: Camera Portrait
-
Interesting artifact. If you can share raws I'd like to have a look and see what the reason is. It could be a unavoidable effect due to a trade-off, or maybe I need to make some tuning to the clip handling. I need to look at the raws to be able to figure out what it is.
-
Not to be conflated with the issue brought up in my previous post, I noticed extreme purples were rendered as "blobs":
I think with such light you really just go with matrix profiles - no LUTs...
-
I think with such light you really just go with matrix profiles - no LUTs...
Good idea.
Interesting artifact. If you can share raws I'd like to have a look and see what the reason is. It could be a unavoidable effect due to a trade-off, or maybe I need to make some tuning to the clip handling. I need to look at the raws to be able to figure out what it is.
PM sent.
-
Actually the normal matrix-only profiles will make it worse, the reason is that a matrix profile is optimized for "normal colors", meaning that ultra-saturated colors like this are pushed way out into negative and positive clipping.
You can however design a very low saturation matrix without any clipping, but the colors won't be that realistic. In fact if you open the file in RawTherapee, use the DCamProf profile and disable the LUTs (Base table and Look table checkboxes in RT) you have exactly that -- a low saturation no-clipping matrix.
When you then apply the base table the colors are stretched back into proper saturation, which causes a the blob issue with the magenta highlight. It will take quite some time for me to drill down to the exact reasons why the transition into clipping is not nicer, I'll have a closer look when I get some more time.
I had a similar issue with artificial blue light in a A7r-II image, it was a while ago so I don't remember the conclusion but it was something along the lines that in order to keep proper saturation and hue close to clipping it was impossible to make a really smooth transition into clipping. I'm not sure it's that which is happening here though...
For now I have to say that robustness of super-saturated clipped colors with high blue content is a known issue, and in those situations you may be better off using a different profile.
-
You can however design a very low saturation matrix without any clipping, but the colors won't be that realistic.
for that subject in such light do we really need realistic colors or rather no artefacts from improper LUT in that areas ?
-
for that subject in such light do we really need realistic colors or rather no artefacts from improper LUT in that areas ?
Indeed, there's little point in realistic colors as it's difficult in such light anyway. It should be possible to make it better than DCamProf does it currently though, not sure if it's possible to combine with the best color in good light though. I need to drill down deeper into the problem to figure that out.
-
I have now released version 0.10.5, you find it as usual here:
http://www.ludd.ltu.se/~torger/dcamprof.html
direct download here: http://www.ludd.ltu.se/~torger/files/dcamprof-0.10.5.tar.bz2
I've solved Markanini's disco light problem (http://forum.luminous-landscape.com/index.php?topic=100015.msg899463#msg899463) by improving the extreme value compression algorithm. Those that have no extreme value problems with their profiles don't need to regenerate them.
I've also included a reference file for the CC24 manufactured November 2014 and later, the data provided by AlterEgo in this post: http://forum.luminous-landscape.com/index.php?topic=100015.msg872313#msg872313
I also made the documentation web page a little bit friendlier for mobile devices.
The "extreme value compression" works like this:
The basis for the LUT is a linear matrix, but some of those values will be outside a valid gamut (negative Y etc), for cameras with high sensitive blue like many Sony cameras a significant range can be there. The LUT can't work well with invalid inputs, so there's a pre-compression done directly on the (white-balanced) raw RGB values. Saturation is reduced by using RGB-HSV desaturation until it fits in a valid gamut. To not hurt gradients the transition range is quite wide (it can be controlled using the new -k parameter to the make-profile command, but generally there's little reason to modify it) and the larger out-of-gamut range the camera has for a particular hue the larger transition range. RGB-HSV desaturation is chosen as it results in lighter colors, so instead of a more saturated color you get a lighter color and this way tonality is kept.
This pre-compression of out-of-gamut matrix values is applied before the normal gamut compression (if activated). It can be disabled completely by "-k 0", but that's a bad idea(tm), unless you're making a reproduction profile and want to be 100% sure that the compression doesn't disturb colorimetric accuracy within the valid gamut.
-
Macintosh build of version v0.10.5 available at the below indicated URL
-
0.10.5 build for Windows ( mingw = dcamprof.exe + libgomp_64-1.dll + both manual & tutorial / = copies of Torger's web pages / in 3 formats : IE archive .mht, Mozilla archive .maff and regular .pdf ) : https://app.box.com/DCamProf (https://app.box.com/DCamProf)
-
Thanks to all the three of you (Torger, howardm and AlterEgo)! - AlterEgo, how do you save a web page as .pdf??
-
Thanks to all the three of you (Torger, howardm and AlterEgo)!
+1
Cheers,
Bart
-
Compiles on the Mac with the following warning:
gcc -o observers.o -c -I. -Wall -std=c99 -g -O2 -D_XOPEN_SOURCE=600 -D__EXTENSIONS__ -DDCAMPROF_VERSION=\"0.10.5\" observers.c
observers.c:107:21: warning: unused variable 'cmf_2006_2_390_830_5'
[-Wunused-const-variable]
static const double cmf_2006_2_390_830_5[][4] = {
^
1 warning generated.
No idea whether that is significant or not.
-
Nope. He's got -Wall which is 'complain about everything' and in this case, he doesn't actually use that (constant) variable. (how's that for an oxymoron?)
Also note that in my compile/build, it's completely 'static' meaning there are no runtime dynamic library dependencies which can bite you (or someone else you give the binary to) in the butt. Just an FYI in case you were not aware of that.
-
Please note that my previous build of v0.10.5 had a small error in it.
The problem would show up if you tried to use glare matching.
I have rebuilt it and re-uploaded it.
If you downloaded it before 10:50PM EST 5/7, please throw
it away and re-download it.
Sorry for the inconvenience.
-
No need to worry about that warning. I haven't look into the observer stuff for quite a while. In the beginning of the project I had an idea to use the "best" observer functions, that is those that gave the best matching results in experiments. The 1931 observer is not really the best. However the CIE standards organization only change their standards if something comes along and provides significant improvement, otherwise they keep the old standard, and that's why the 1931 color matching function have survived for 85 years.
I was thinking that I could use some of the experimental matching functions though, but there's a major problem -- all the other color spaces are defined with the 1931 observer, so if I just change observer those are thrown out of balance. So in order to use a different observer one have to make some sort of transform from the new observer XYZ to a matching XYZ in the 1931, and I haven't really found any good way to do that.
The color matching functions does not say what color we see, just that spectrum A will have the same color as spectrum B if the resulting XYZ is the same, that is the colors match, hence the name "color matching functions". If we just exchange the color matching functions in the test target spectrum matching but keep the rest the same we will be in error, as the resulting XYZ coordinates will be transformed eventually to an RGB value on your screen and that transform is tuned for 1931 observer.
Anyway I don't think it's a big issue for this type of application to use the old 1931, from what I've read the matching errors seems to be largest for narrow band lights like some types of LED lights, where you may have an error of like 10 DE, but this is also the type of light conditions that cameras will naturally be in high error. For more sane colors the matching should be good with the old standard, and there the cameras have a better chance to perform well too.
-
RGB-HSV desaturation is chosen as it results in lighter colors, so instead of a more saturated color you get a lighter color and this way tonality is kept.
A further comment about this. I often "complain" about the RGB HSV and HSL color spaces for not being psychovisually accurate, so why am I using it myself in this case? The RGB spaces have some advantages, they are very good at maintaining smooth gradients, and you don't need a valid XYZ coordinate to work with it. The camera registers colors in an RGB space, and when you transform that to an psychovisually advanced space like CIECAM02 you sort of mess up the camera's gradients. This is generally fine when you're some distance from clipping, but when you want to make a smooth transition into clipping it's hard to do that if not doing it in concert with the camera's RGB channels. This is represented in the DCamProf's neutral tone reproduction operator which blends in RGB space stuff close to clipping.
However in this compression case the reason is different, here we're working in a range where we don't have any valid XYZ coordinate and thus we can't transform it further into CIECAM02 or any other advanced color space which all start with a valid XYZ coordinate. So what we do is that we compress the camera's "extreme" RGB value into a new RGB value that can be transformed into a valid XYZ coordinate. Not being able to work in any advanced color space falling back to a raw RGB space is what we have to do. In this case RGB-HSV desaturation proved to show a visually pleasing effect as it lightens the color simultaneously which makes it look like it's closer to "film clipping".
Remember Bart's red tulips I used in refining high saturation highlight rendering? Those are unaffected by this change, as their coordinates where still in valid XYZ range, however the compression have a much similar effect in look as on those tulips where I also chose an approach to lighten the colors near clipping to keep tonality.
This new compression is primarily for the usual suspect -- Sony's high sensitive blue range. There's probably more cameras with this high sensitivity, but in my tests there's always been Sony sensors that have triggered issues. Not all Sony sensors though, I recently had a look on Phase One's IQ3 100MP which is using Sony's new 645 full-frame sensor, and it doesn't have the same "over-sensitive" blue range, possibly through a custom CFA, or perhaps more likely simply by dampening the blue range somewhat with the sensor cover glass. I don't know why Sony choose to have such high sensitive blues, it's probably some high ISO thing, but say Canon doesn't have poor high ISO either and their blue response is still not as challenging. Or they just like to optimize for indoor low temperature lights, then the high sensitive blue is an advantage as low temperature lights have very little blue content.
-
how do you save a web page as .pdf??
I don't know how AlterEgo did, but you can do it directly with Adobe Acrobat DC: File->Create->PDF From Web Page
-
In the Mac world, we'd simpy tell it to print and then Save As PDF. Isn't there something similar in the Windows print dialog box (ie. a PDF pseudo-printer) ?
-
In the Mac world, we'd simpy tell it to print and then Save As PDF. Isn't there something similar in the Windows print dialog box (ie. a PDF pseudo-printer) ?
Yes, but because Windows is not as intimate as Apple / Adobe are, it requires a driver / program to do that. There are some free ones. and others are paid versions.
Cheers,
Bart
-
Everytime I have to delve into that world, I know my choice of OS was correct. ;)
-
Isn't there something similar in the Windows print dialog box (ie. a PDF pseudo-printer) ?
W10 has "Microsoft Print to PDF" 'printer', but I always use Adobe Acrobat.
-
In fact I can save this single LuLa discussion thread page in PDF in OS 10.6.8 and it will embed the original URL so if years later I want to hunt down the entire 60 page thread I won't have a problem finding it on the web.
I have a lot of forum discussion pages not just here but from other website forums in my Documents folder. Very convenient over hunting down bookmarks in my browser or doing a Google search online. And if by chance the site page is removed by administrators or just cleared from servers to make more room, I still have a copy. It's better than a screengrab.
-
Thanks to all who responded to my question on how to save a web space as .pdf!
Since I, too, am on a Mac, the easiest way seems to be to open the print dialogue and then Save As pdf. It also records the URL. And the disk space is only 40% of the web archive. What annoys me about Apples web archive is that it pretends to save, but if you re-open the page after a while, the images are gone, so they are just cached. But Apple doesn't tell you that.
Well, I will not digress into off-topic any further. Thanks again!
-
These are good changes. Thanks for this update!
Seb
-
In the Mac world, we'd simpy tell it to print and then Save As PDF. Isn't there something similar in the Windows print dialog box (ie. a PDF pseudo-printer) ?
In Windows is the same (and no need for additional drivers, at least since windows 7), Acrobat has just more options to customize the output. You can even generate a pdf file with an entire multilevel website at once
-
I've considered the option of "rebuilding" existing DCPs with DCamProf. The usage case would be profiles that already are colorimetric accurate profile but lack a desired tone curve or look operators BLE, black subtraction etc. Another could be for cross checking.
I'm assuming this would require a script to read the source DCPs data and re-format it into the DCamProfs native .json format?
-
I've considered the option of "rebuilding" existing DCPs with DCamProf. The usage case would be profiles that already are colorimetric accurate profile but lack a desired tone curve or look operators BLE, black subtraction etc. Another could be for cross checking.
I'm assuming this would require a script to read the source DCPs data and re-format it into the DCamProfs native .json format?
If I understand you correctly you want to take any DCP (not necessarily made by DCamProf) which has a good colorimetric base but then has the standard Adobe curve slapped on top without caring about the color appearance distortions it makes, and then make a new profile which uses that colorimetric base but uses DCamProf's netural tone reproduction operator to add the curve (and possibly add look operators too).
To support all cases you would need to make something that transforms DCPs to the native .json format, which for LUTs would be quite complicated and a 100% 1:1 match is not always possible.
However in some cases you can do it with some hand-editing. The key is that the original DCP is structured such that the colorimetric base is in the HueSatMap and the LookTable is not used (or if used applies a crap look you want to strip away). Then you make a DCamProf profile, for any camera, with the desired look. All tone reproduction and look operator stuff will be stored in the LookTable + tone curve.
Then you copy the LookTable and Curve elements into the original profile, but keep the HueSatMap and matrices. By using dcp2json and json2dcp you can edit them as text files.
Some/many DCP profiles are however not made in that way, but there is no HueSatMap but instead only a LookTable. You could then just rename LookTable to HueSatMap and merge as above, but keep in mind that the HueSatMap is applied before exposure adjustment and LookTable after but it in most cases that will probably not matter.
(Using the Looktable instead of the HueSatMap is often used to mimic an ICC profile pipeline, as ICCs are generally applied after exposure adjustment rather than before. Adobe Lightroom has this design for most of their "brand-specific" profiles which I think is made to mimic in-camera JPEG looks, not sure though as I have never compared them myself).
-
Went perfectly thanks to your clear instructions. Have a nice weekend.
-
You could then just rename LookTable to HueSatMap and merge as above, but keep in mind that the HueSatMap is applied before exposure adjustment and LookTable after but it in most cases that will probably not matter.
dcptool will move LookTable table(s) into HueSatMap table(s)
-
dcptool will move LookTable table(s) into HueSatMap table(s)
Would I do that by using dcptool to make a profile untwisted?
EDIT:Turns out the answer is to make the profile invariate rather than untwisting. I cant see any difference when basing on a untwisted profile from an invariate. Perhaps it depends on the profile.
-
A comment about "hue twists" -- this is often presented as something bad and distorting, and if you want constant color appearance regardless of exposure you must remove them.
This ignores the fact that general-purpose profiles have tone curves, and if you want constant color appearance, like DCamProf's neutral tone reproduction operator strives for, there's all sorts of non-linear tricks you need to do in the LookTable LUT to counteract the effects of the curve, which will show up as "hue twists". The LookTable is applied after exposure and the curve directly after that so they follow each-other, that is the LookTable corrections will always match the curve.
That is "untwisting" a profile only makes sense if you intend to use the profile without tone curve. One use case for that would be to prepare files for HDR merging, but then it's better to merge in raw (like in my own Lumariver HDR), or if you don't have software for that use a pure matrix profile as even an untwisted LUT can have other nonlinearities left.
Profiles like Adobe's own also add subjective elements to it, probably stuff like cooling down shadows and warming up highlights (a common adjustment) which is a "true" hue twist, but as long as it's in the LookTable and thus applied after exposure it makes sense.
Another aspect is that if your profiler works with a different colorspace like say CIECAM02 JCh it will not perfectly match up with RGB-HSV, so you get some minor "hue twists" in the HSV space despite that the color is kept perceptually stable in the other space. DCamProf uses CIECAM02 quite extensively.
-
I wonder if my results are what is expected.
Basically I'm doing:
dcp2json profile.dcp profile.json
dcamprof make-dcp profile.json new-profile.dcp
I've tried using profile.json in different states
As is
ProfileLookTableDims changed to ProfileHueSatMapDims and ProfileLookTable cahnged to ProfileHueSatMap1
Made invariate by dcptool
Made untwisted by dcptool
Made untwisted by dcptool +ProfileLookTableDims changed to ProfileHueSatMapDims and ProfileLookTable cahnged to ProfileHueSatMap1
All of the above look identical to me and the historgram. The look is mostly simmilar to a matrix only profile.
-
Make-dcp expects a native profile. Use json2dcp instead to convert from dcp in json format to dcp in binary. Make-dcp probably only reads the forwardmatrix, I had not realised that it was even possible to feed it with a dcp json file.
-
So my final steps were:
make the source dcp invariate with dcptool
convert the new dcp to json
copy "ProfileLookTableDims" and "ProfileLookTableEncoding" "ProfileLookTable" and "ProfileToneCurve" from an existing DCP/json
convert the edited json to dcp
Now I have the original look but reds are no longer oversaturated etc
Mission accomplished!
EDIT:
A more detailed report on my results if anyone's interested.
On a colorimetric profile it was easy to add the tone curve and neutral+ look operators on top and the results looked excellent.
An early batch Adobe Standard profile required making the the profile invariate first with dcptool which converts looktables to huesatmap tables so that you can add on the tone curve and neutral+ look operators. Also excellent results to my eyes.
On a camera based profile I lost smoothness in skin tones from making it invariate with dcptool. Making invariate looked better than simply renaming looktables to huesatmap tables. Untwisting first gave me hue shifts. In the end the profile in it's original state looked best, even with oversaturated colors so in this case it was a lost effort.
-
I have now released version 1.0.0. It's basically just a re-branded 0.10.5 (no new features) plus a much needed cleanup of the docs.
DCamProf was first released about a year ago, then followed a intensive feature upgrade and bug fix period for six months, and after that it's been resting and tested by users all around. Now I think it's mature enough to be promoted into version 1.0, which is saying that it's ready for a broader audience.
http://www.ludd.ltu.se/~torger/dcamprof.html
-
On a camera based profile...
I don't know how Adobe makes the camera based profiles, but I suspect that they shoot a target and render a profile from the in-camera JPEG. This cannot be split into a colorimetric base + look, but will only be a LookTable (+ curve) which contains a combination of both. It will be just as non-linear as the in-camera rendering is, and thus very hard to reverse into a linear curve profile an adding another tone reproduction operator on top.
It would be the same problem to reverse DCamProf's profiles if all was put into the LookTable, however the DCP's DCamProf makes split them in two parts, colorimetric base (matrix + HSM) + tone reproduction on top (LookTable + Curve).
-
Macintosh build of dcamprof-v1.0.0 now available at below URL
-
1.0.0 build for Windows ( mingw = dcamprof.exe + libgomp_64-1.dll + manual, tutorial, pre-release history log / = copies of Torger's web pages / in 3 formats : IE archive .mht, Mozilla archive .maff and regular .pdf ) : https://app.box.com/DCamProf (https://app.box.com/DCamProf)
brief instructions how to build DCamProf yourself on Windows platform :
download and install TDM GCC from http://tdm-gcc.tdragon.net/download ( tdm64-gcc-5.1.0-2.exe at the moment ), don't forget to check 'openmp' option during installation.
download and install MSYS ("...MSYS is a collection of GNU utilities such as bash, make, gawk and grep to allow building of applications and programs which depend on traditionally UNIX tools to be present..." ) from https://sourceforge.net/projects/mingw/files/MSYS/Base/msys-core/msys-1.0.11/MSYS-1.0.11.exe/download .
download LCMS by Marti Maria from https://github.com/mm2/Little-CMS or from https://sourceforge.net/projects/lcms ( http://www.littlecms.com ) and build it = run msys.bat, then go to your LCMS root folder and do './configure', then do 'mingw32-make' -> lcms library will be 'src\.libs\liblcms2.a' .
download LibTIFF from http://www.libtiff.org and build it = run msys.bat, then go to your LibTIFF root folder and do './configure', then do 'mingw32-make' -> libtiff library will be 'libtiff\.libs\libtiff.a' .
download DCamProf from http://www.ludd.ltu.se/~torger/dcamprof.html#download , copy liblcms2.a + lcms2.h from LCMS and libtiff.a + tiff.h + tiffconf.h + tiffio.h + tiffvers.h from LibTIFF to the source code folder, make sure that 'makefile' has the proper library names (replace '-llcms2 -ltiff' with 'liblcms2.a libtiff.a') and add -D_POSIX_THREAD_SAFE_FUNCTIONS to CFLAGS in 'makefile' to account for 'localtime_r' and 'asctime_r' with TDM GCC and build it = run msys.bat, then go to your DCamProf root folder and do 'mingw32-make'... copy dcamprof.exe and libgomp_64-1.dll from TDM GCC to where you want to keep the binaries.
-
I have now released version 1.0.0. It's basically just a re-branded 0.10.5 (no new features) plus a much needed cleanup of the docs.
DCamProf was first released about a year ago, then followed a intensive feature upgrade and bug fix period for six months, and after that it's been resting and tested by users all around. Now I think it's mature enough to be promoted into version 1.0, which is saying that it's ready for a broader audience.
http://www.ludd.ltu.se/~torger/dcamprof.html
Thanks for your good work and the helping people around. I'm very happy with my profiles made by your program.
-
Thanks for your good work and the helping people around. I'm very happy with my profiles made by your program.
I totally agree. With the Neutral+ operators I get a excellent refined look for general photography and a good starting point for further processing. I realise DCamProf has become an indispensable tool for me where previously included lightroom profiles and JPEG colors would leave me frustrated.
-
I have now released version 0.10.5, ...
I've also included a reference file for the CC24 manufactured November 2014 and later, the data provided by AlterEgo
In case it helps, Xrite have published data here http://xritephoto.com/ph_product_overview.aspx?ID=1192&Action=Support&SupportID=5884&catid=28
-
In case it helps, Xrite have published data here http://xritephoto.com/ph_product_overview.aspx?ID=1192&Action=Support&SupportID=5884&catid=28
yes, was reported here on Oct 30, 2015 = http://forum.luminous-landscape.com/index.php?topic=100015.msg864549#msg864549
the issue was/is - X-Rite does not publish ___spectral___ data...
-
Macintosh build of dcamprof-v1.0.0 now available at below URL
Hi Howard,
thanks again for your work! - Strangely, this one won't download. If I click the Download button, it says 'Your download has started', but it hasn't, at least I can't see it in the Downloads folder. I have no firewall active, and app downloads are allowed from everywhere in the Safari prefs. Would you have any clue to what could be wrong? I'm on OS X 10.10.5.
Thanks.
-
Not yet but I do see some weirdness. I'll try contacting them tmrw unless it auto-fixes itself by then.
-
Are there any fuller instructions around on how to extract the target image correctly for use with dcamprof.
If I extract it from the raw linear TIFF using Photoshop, for example, is Photoshop going to apply a profile (unless I say it should be unmanaged)?
I would use RawTherapee, as suggested, but the interface is fairly non-standard. Before I dive into the manual, is there a quick recipe for getting it to do the necessary, if that is preferable to using DCRaw?
Why when running the scanin command do I not use -G 1.0 to tell it that it is a linear file? Or -p to compensate for the inevitable for perspective distortion?
Thanks.
-
Are there any fuller instructions around on how to extract the target image correctly for use with dcamprof.
If I extract it from the raw linear TIFF using Photoshop, for example, is Photoshop going to apply a profile (unless I say it should be unmanaged)?
I would use RawTherapee, as suggested, but the interface is fairly non-standard. Before I dive into the manual, is there a quick recipe for getting it to do the necessary, if that is preferable to using DCRaw?
[...]
Thanks.
Open the raw file in Raw Therapee, go to Editor > Color > Color Management. There is a command 'Save Reference Image for Profiling'. The tool tip says something like 'Save the linear TIFF image before the input profile is applied. The result can be used for calibration purposes and generation of a camera profile.' This is the one. It looks like you still will need to zero various other parameters in the Editor panel. The GUI may be unusual, but it's quite systematic/logic/intuitive. Even I as a Mac user can quite easily find my way around just looking at all the tool tips.
Good light!
-
Are there any fuller instructions around on how to extract the target image correctly for use with dcamprof.
Rawdigger PE does a good job...
-
Thanks. Yes, it does, but it seems costly for the limited task that I would use it for.
-
I still dont have an answer/reason it wont download but if someone wants/needs the Mac version, send me a private message and include your email address and I'll send it to you directly (assuming you can receive a 10MB email)
-
Thanks. Yes, it does, but it seems costly for the limited task that I would use it for.
for a limited task the trial shall work ....
-
Well, it appears as though the issue is that box.com has a monthly bandwidth cap of 10GB. I have a very hard time believing that I busted that cap but that what they are telling me. So, the box.com link is dead until June 1.
I've copied the Macintosh dcamprof stuff over to OneDrive for now............
https://onedrive.live.com/redir?resid=C3203EEC29F80668!260&authkey=!AD4b0aQht3zfw8g&ithint=folder%2cpng
-
@howardm
Yes this one downloaded without problems. Thank you for making this possible.
-
for a limited task the trial shall work ....
Thanks. I'll give it a go. Looking at the instructions on your site, I see that you recommend output gamma of 1.8. Is that right for dcamprof?
-
Thanks. I'll give it a go. Looking at the instructions on your site, I see that you recommend output gamma of 1.8. Is that right for dcamprof?
not exactly... if you are looking to compensate C1 'transfer function' more precisely then you can extract it from .tiff and then apply it according to your profile building workflow... alternatively you certainly can use gamma around ~1.8* (tastewise, may be you like something between 1.7* and 1.9*)... your call... but 'transfer function' != gamma
that is if you are talking about C1 raw converter of course
-
@Hening
You're welcome. It's at least some small attempt to 'give back' to the community here that has taught me so much.
I had no idea BOX had a monthly BW cap. Will have to examine alternatives.
-
not exactly... if you are looking to compensate C1 'transfer function' more precisely then you can extract it from .tiff and then apply it according to your profile building workflow... alternatively you certainly can use gamma around ~1.8* (tastewise, may be you like something between 1.7* and 1.9*)... your call... but 'transfer function' != gamma
that is if you are talking about C1 raw converter of course
I'm talking about Adobe / DCP.
The scanin tool find that my pattern match is not good enough, so I'm a bit stuck.
scanin -v -p -G 1.0 -dipnvh 20160521-122518_L1010315-cut.tif ColorCheckerPassport.cht cc24_ref-new.cie
TIFFFetchNormalTag: Warning, Incompatible type for "RichTIFFIPTC"; tag ignored.
Input file '20160521-122518_L1010315-cut.tif': w=1000, h=668, d = 3, bpp = 8
Data input file 'cc24_ref-new.cie'
Data output file '20160521-122518_L1010315-cut.ti3'
Chart reference file 'ColorCheckerPassport.cht'
Creating diagnostic tiff file 'diag.tif'
About to allocate scanrd_ object
Verbosity = 2, flags = 0x62a07
About to read input tiff file and discover groups
adivval = 1.000000
About to calculate edge lines
198 useful edges out of 314
About to calculate perspective correction
Perspective correction factors = 0.000009 -0.000001 500.000000 334.000000
About to calculate rotation
Mean angle = 0.170600
Standard deviation = 0.803220
Robust mean angle = 0.145905 from 163 lines
About to calculate feature information
About to read reference feature information
Read of chart reference file succeeded
About to match features
Checking xx
Checking yy
Checking xy
Checking yx
Checking xix
Checking yiy
Checking xiy
Checking yix
Axis matches for each possible orientation:
0: xx = 0.200645, yy = 0.056684, xx.sc = 0.269088, yy.sc = 0.207938
90: xiy = 0.096582, yx = 0.115018, xiy.sc = 0.393503, yx.sc = 0.125325
180: xix = 0.211312, yiy = 0.058271, xix.sc = 0.269088, yiy.sc = 0.207334
270: xy = 0.111208, yix = 0.120400, xy.sc = 0.277110, yix.sc = 0.124871
r0 = 0.161118, r90 = 0.047834, r180 = 0.168894, r270 = 0.073857
There are 0 candidate rotations:
About to write diag file
Argyll 'V1.8.3' Build 'OS X 64 bit' System 'Darwin Darwin Kernel Version 15.5.0: Tue Apr 19 18:36:36 PDT 2016; root:xnu-3248.50.21~8/RELEASE_X86_64 15.5.0 x86_64'
scanin: Error - Scanin failed with code 0x3, Pattern match wasn't good enough
-
Have a look in your "diag.tif", it usually gives a lead of what's wrong in the picture. Argyll's scanin is very sensitive, it needs a perfectly flat projection, no rotation, and without any surroundings.
-
Thanks. I've tried it with and without the corner markers (although the documentation says that the latter should be included).
Most lines are detected (see attached).
-
The scanin tool find that my pattern match is not good enough, so I'm a bit stuck.
Hard to know why without more information, but if it's normal orientation, try running with the -a flag added, and see what happens. Check the diag output.
-
Thanks. The previous 2 posts give those diagnostics.
-
The shot orientation looks perfectly okay. Maybe there's something wrong with the colors is the converted file? I note in the log output that the input file is only 8 bits, it should be 16 bit if you converted in any of the ways suggested in the docs.
I assume you're making a file for DCP? It should look greenish, like this:
http://www.ludd.ltu.se/~torger/photography/camera-profiling.html#step4
The easiest way to do it with free tools is using RawTherapee (as you can crop/rotate there), described here:
http://50.87.144.65/~rt/w/index.php?title=Color_Management#Save_Reference_Image_for_Profiling
If you're going to use the profile in Adobe Lightroom it's safest to first make a DNG via Adobe DNG Converter and make the reference image from that, as raw decoding might differ. If you don't have a too exotic camera it shouldn't differ though and you could use the native raw also in RawTherapee/DCRaw.
-
I had no idea BOX had a monthly BW cap. Will have to examine alternatives.
If you have Gmail, Google Drive is a very good alternative and its free.
-
Thanks but I try to avoid most Google products these days.
-
Thanks but I try to avoid most Google products these days.
In which case there is still Dropbox (in case you want to avoid Microsoft's OneDrive as well).
Cheers,
Bart
-
The shot orientation looks perfectly okay. Maybe there's something wrong with the colors is the converted file? I note in the log output that the input file is only 8 bits, it should be 16 bit if you converted in any of the ways suggested in the docs.
I assume you're making a file for DCP? It should look greenish, like this:
http://www.ludd.ltu.se/~torger/photography/camera-profiling.html#step4
The easiest way to do it with free tools is using RawTherapee (as you can crop/rotate there), described here:
http://50.87.144.65/~rt/w/index.php?title=Color_Management#Save_Reference_Image_for_Profiling
If you're going to use the profile in Adobe Lightroom it's safest to first make a DNG via Adobe DNG Converter and make the reference image from that, as raw decoding might differ. If you don't have a too exotic camera it shouldn't differ though and you could use the native raw also in RawTherapee/DCRaw.
Thanks. This is for the Leica SL, which generates DNGs natively.
I converted to TIFF copying and pasting the dcraw command line that you have on the tutorial page, I think, so not sure why it's not 16-bit.
I have then taken the resulting TIFF into Photoshop (unmanaged) and used the warp tool to correct perspective, cropped and run scanin as above.
-
Oops, the 8 bit thing is an error in my docs, forgot the -6, it should be:
dcraw -v -r 1 1 1 1 -o 0 -H 0 -T -6 -W -g 1 1 <rawfile>
8 bit or not should not be a difference between success or failure though as far as I know, so it's a mystery. I've always used RawTherapee myself so I haven't run that particular workflow but I don't see why it wouldn't work...
Thanks. This is for the Leica SL, which generates DNGs natively.
I converted to TIFF copying and pasting the dcraw command line that you have on the tutorial page, I think, so not sure why it's not 16-bit.
I have then taken the resulting TIFF into Photoshop (unmanaged) and used the warp tool to correct perspective, cropped and run scanin as above.
-
Do you need to resize the file?
The scanin docs say that it doesn't like more than 1200 on the longest side, which seems a bit small. I think that your workflow gives a large file, unless you film the target from a great distance. (Neither small nor large sizes worked for me.)
-
I have never needed to resize from what I remember. If you make the file available I can scan it for you.
-
Thanks Anders. I have shared a dropbox folder with you.
-
Thanks. The previous 2 posts give those diagnostics.
The previous posts do not have diagnostics for a run using the -a flag, and the suggestion was for your own benefit - i.e. given it is not working normally, don't just blindly use the output even if output is produced using the -a flag.
-
I've tried your file and here's a scan (attached).
However I also had much difficulties scanning it, and I can't guarantee that the attached file is 100% perfect as the matches sits on the borders of the patches. Even if the image looks square and alright to the eye after a perspective correction some tiny residual scale differences between patches throw scanin out of balance and it can't match. It's extremely sensitive, I hadn't really realized how much as I've always used straight perpendicular shots and not needed to perspective correct.
I shall update the manual to stress the importance of perpendicular shots for scanin, as perspective corrected shots rarely seems to be good enough for scanin. This tool is originally designed to read patch sets from scanners (where perspective distortion is zero and rotation is zero) so it does not give much margin for error for camera test target shots.
If you need more forgiving patch scan software you need to turn to one of the commercial softwares like RawDigger.
-
I shall update the manual to stress the importance of perpendicular shots for scanin, as perspective corrected shots rarely seems to be good enough for scanin. This tool is originally designed to read patch sets from scanners (where perspective distortion is zero and rotation is zero) so it does not give much margin for error for camera test target shots.
Hi Anders,
I understand the compatibility with scanning, but I find it a bit strange that perspective corrected shots cause 'scanin' to trip over the input. I do not understand why it has to be so sensitive to geometry (if that's the issue). Afterall, there is a '-p' flag to adjust for perspective correction. Maybe there is something that Graeme could redesign in his utility?
There is quite a number of situations when not shooting perpendicular to the target can be a common scenario, and sometimes even preferable (to avoid reflections). One can always square the image before feeding it to 'scanin', and/or use the -p flag, but then it should be accepted.
I do understand that resampling changes the grain/noise structure, but since we are averaging over a larger patch area, maybe the changes in noise weigh in too much?
Cheers,
Bart
-
Hi Anders,
I understand the compatibility with scanning, but I find it a bit strange that perspective corrected shots cause 'scanin' to trip over the input. I do not understand why it has to be so sensitive to geometry (if that's the issue). Afterall, there is a '-p' flag to adjust for perspective correction. Maybe there is something that Graeme could redesign in his utility?
There is quite a number of situations when not shooting perpendicular to the target can be a common scenario, and sometimes even preferable (to avoid reflections). One can always square the image before feeding it to 'scanin', and/or use the -p flag, but then it should be accepted.
I do understand that resampling changes the grain/noise structure, but since we are averaging over a larger patch area, maybe the changes in noise weigh in too much?
I haven't had much success with the -p parameter. You'd have to ask Graeme what the limits are supposed to be. From the tests I just did all I can conclude is that if your shot aren't perpendicular you're in for some trouble. I did small small adjustments of perspective correction in jrp's image and suddenly it succeeded matching, but changing a tiny bit (into something that looked even more regular to my eye) it failed again, so it appears pretty random. For perpendicular shots when the only correction required if any is rotation I haven't had issues though. You could still have issues if there's significant distortion in the lens.
I agree that for outdoor shots it can be an advantage to shoot at an angle to minimize reflections, but then it seems like scanin is almost impossible to use.
-
The versions of the files that I tried used the Photoshop Warp tool to transform the target into a rectangular rectangle.
Perhaps it's an exposure rather than a geometry issue?
I tried it with and without -a, but it made no difference.
I did fiddle with RawTherapee but did not pursue it much further as there are, apparently, settings that you have to switch off manually before exporting.
Ideally, you want to mechanical stuff in the workflow to happen mechanically, so that you can concentrate on tuning.
This sensitivity to alignment does seem to be a forbidding weak link in the workflow for dcamprof.
-
I agree that for outdoor shots it can be an advantage to shoot at an angle to minimize reflections, but then it seems like scanin is almost impossible to use.
Yes, and even indoors in a partially bright surrounding. It's no fun (although the most robust profiling workflow) having to shield the entire ambiance with black, including the T-shirt and jeans of the photographer, to avoid potential reflections on the surface of the target.
I hope Graeme will chime in on this potential showstopper. Is it geometry, is it noise (what if we add blur or defocus), or is it ... ?
Cheers,
Bart
-
I've thought about making an own scan tool, it's an interesting programming challenge with pattern recognition. The most robust thing would be to make a GUI where you have a patch grid wireframe that you can manually stretch and bend in place, but then DCamProf would get lots of GUI dependencies. It would be nicer with a command that could search and find a target in a picture and automatically map without any GUI, but it's no easy task.
-
what about opencv or a number of other similar packages?
-
what about opencv or a number of other similar packages?
-
I've thought about making an own scan tool, it's an interesting programming challenge with pattern recognition. The most robust thing would be to make a GUI where you have a patch grid wireframe that you can manually stretch and bend in place, but then DCamProf would get lots of GUI dependencies. It would be nicer with a command that could search and find a target in a picture and automatically map without any GUI, but it's no easy task.
http://campus.udayton.edu/~ISSL/index.php/research/ccfind/ -> http://campus.udayton.edu/~ISSL/index.php/software/
-
I've thought about making an own scan tool, it's an interesting programming challenge with pattern recognition.
Yes, assuming that the issue with 'scanin' is geometry based.
There are different degrees of sophistication possible, but one could start as scanin does, by assuming a more or less properly squared input. It doesn't have to be perfectly distortion free either, as long as one can get enough of the centers of the patches for a statistically significant number of samples to average.
Strictly speaking, one doesn't even have to use perspective corrected images, just the locations of each of the 4 patch corners from which to take the non-resampled pixel values, and a zone within those coordinates to avoid along the patch edges (to avoid shadows and/or edge blur). It can also avoid the need for resampling of the source image, just sample and average the pixels within the patch inner boundaries.
A simplification would be to use full pixel row shifts (or only offsets) to align vertical edges a bit better, and a full pixel column shift to align horizontal edges a bit better. Such a trapezoidal de-skewing avoids potential resampling issues (by using only full pixel shifts), and produces somewhat more rectangular ROIs in case a target is rotated.
The most robust thing would be to make a GUI where you have a patch grid wireframe that you can manually stretch and bend in place, but then DCamProf would get lots of GUI dependencies. It would be nicer with a command that could search and find a target in a picture and automatically map without any GUI, but it's no easy task.
That's more complex (although nice), and not trivial to do for multiple OS environments. But let's see if Graeme can (and is willing to) improve the usability of scanin, by changing whatever is causing the refusal of certain target inputs.
EDIT: Maybe AlterEgo's link gives some more ideas, I haven't read it yet.
Cheers,
Bart
-
I've tried your file and here's a scan (attached).
Many thanks, Anders. I have now tried to generate Natural and Natural+ profiles.
The result is indeed natural looking in Lightroom. The result is quite desaturated and lower contrast, relative to the regular Xrite or Adobe-generated profiles (and indeed Adobe Standard). The biggest difference is in the Blue and the Red patch. This is not necessarily a bad thing; I need to test out some more images.
../dcamprof make-dcp -n "LEICA SL (Typ 601)" -d "Natural+" -t acr -g srgb -o neutral-plus.json my-profile.json natural-plus.dcp
Generating 2.5D HueSatMap with 90x30 = 2700 entries...done!
The tone curve's contrast value is 1.30 (=> auto chroma scaling value 1.121)
Generating 3D LookTable with 90x30x30 = 81000 entries for the neutral tone reproduction operator...
0%..10%..20%..30%..40%..50%..60%..70%..80%..90%..100%
Writing output to "natural-plus.dcp"...
Complete!
PS: gnuplot on OS X does not like -background gray.
sage: gnuplot [OPTION] ... [FILE]
-V, --version
-h, --help
-p --persist
-d --default-settings
-c scriptfile ARG1 ARG2 ...
-e "command1; command2; ..."
gnuplot 5.0 patchlevel 3
-
PS: gnuplot on OS X does not like -background gray.
Yeah, it's a mess I've recently found out that it's only working on the X11 terminal :/ as far as I know there's no good standard way to change background color in the others. I've seen hacks like plotting a gray rectangle in the background and stuff like that.
-
Even if the image looks square and alright to the eye after a perspective correction some tiny residual scale differences between patches throw scanin out of balance and it can't match. It's extremely sensitive, I hadn't really realized how much as I've always used straight perpendicular shots and not needed to perspective correct.
No, it isn't intended to be this way - quite the contrary.
If someone would like to make the image available to me, then I will look into it.
-
The most robust thing would be to make a GUI where you have a patch grid wireframe that you can manually stretch and bend in place, but then DCamProf would get lots of GUI dependencies.
The scanin -F parameters let it be used as a back end to extract patch data from a manually aligned chart.
-
No, it isn't intended to be this way - quite the contrary.
If someone would like to make the image available to me, then I will look into it.
I hope jrp will share it with you. I got the file but as it's his photo I'd like him to share it.
-
Sure. I've made it available to Graeme.
-
I haven't had much success with the -p parameter.
From the images made available to me, it seems that the problem is that the wrong recognition reference file is being used. A Classic ColorChecker is not the same dimensions as half of a ColorChecker Passport! (the relative patch size is different).
If I take the original Passport shot, crop it, rotate it and reduce the dimensions to 1024, scanin -p works perfectly. The shot is probably useless because it is so unevenly exposed, but the patches are captured fine.
-
Oh, didn't know about the dimension difference. I have now updated the documentation accordingly.
Anyone who knows if the CC24 part of the color checker matches in color with the full-size card, or is that different too?
(Uneven light is not too big of a problem for DCamProf as lightness correction is disabled per default, if you enable it you should have even light or correct with flatfield correction though. Most commercial software also ignores the lightness axis)
-
Strange, I used only the CC24 part of my Colorchecker at least 15 times for building profiles for different cameras and I never had any difficulty with the scanin command. I always used Raw Therapee and cropped the CC24 patches quite closely.
-
Strange, I used only the CC24 part of my Colorchecker at least 15 times for building profiles for different cameras and I never had any difficulty with the scanin command. I always used Raw Therapee and cropped the CC24 patches quite closely.
Same here, it does work but as the fit isn't perfect you get much less margin concerning perspective error. If the shot is perpendicular and nice scanin can match it with the ColorChecker.cht layout.
-
Thanks all.
I'm not quite sure what the conclusion from Graeme's investigation is.
Is there a new configuration file required?
Do we need to use both sides of the passport and rotate it?
Is the resizing key? (Does resizing lose information or is the averaging helpful?)
-
Thanks all.
I'm not quite sure what the conclusion from Graeme's investigation is.
Is there a new configuration file required?
Do we need to use both sides of the passport and rotate it?
Is the resizing key? (Does resizing lose information or is the averaging helpful?)
Resizing is not key as far as I understand, but the matching will run quicker. I usually don't resize. Resizing is not harmful though, it's just averaging which scanin will do anyway.
ColorCheckerPassport.cht should be used instead of ColorChecker.cht. The passport should be fully open and the shot should cover both pages, and it should be oriented so the logo text is red normally left-to-right, seen in Grame's shot above. You still use the cc24-ref-new.cie data file as before (at least my current assumption is that the spectra matches with the larger colorchecker), that is only the CC24 part of the passport will be used. The other patches doesn't add any significant anyway. You could of course spectro-measure all the patches and provide an own complete .cie file but it won't make any real difference. In any case it's much better to use the cc24 ref file with spectra than a complete ref file without spectra (which is bundled with Argyll).
Updated here:
http://www.ludd.ltu.se/~torger/photography/camera-profiling.html#the_easy_way
-
I'm not quite sure what the conclusion from Graeme's investigation is.
Using the Classic ColorChecker recognition file with half a Passport is not reliable, because the geometry doesn't match.
Using the Passport recognition file with the full Passport works much more reliably.
[ In theory I could create a "Right Half Passport" recognition file, if that would be any use. ]
-
Thanks, Graeme. I was using ColorCheckerPassport.cht, but with only half the Passpport. That must have been my misstep. No harm in creating the half-passport .cht, as that will be clearer to new users.
-
Anyone who knows if the CC24 part of the color checker matches in color with the full-size card, or is that different too?
X-Rite (not that we can believe them 100%) says the following = http://xritephoto.com/ph_product_overview.aspx?ID=824&Action=Support&SupportID=5159
Date Created: 1/22/2010 Date Modified: 4/20/2012
The chart below lists colorimetric values for the ColorCheker Classic, ColorChecker Mini, and ColorChecker Passport targets.
but R. Mayers stated that there were still some spectral differences between CC24 Classic and Passport = http://www.rmimaging.com/information/ColorChecker_Passport_Technical_Report.pdf
but then who knows how many time X-Rite (or GmB) actually changed CC24 Classic formulations before, etc... R. Mayers says in his PDF file that CC24 Classic was from 2007 and Passport was from 2009 (?)
but then we know that CC24 Classic colors formulation were recently changed (11/2014)... X-Rite is silent whether they changed Passport... probably they shall be asked persistently (like they were bothered recently till they finally published the note about 11/2014 formulation change a year after the change was done).
-
Well I assume from
http://xritephoto.com/ph_product_overview.aspx?ID=938&Action=Support&SupportID=5884
that they are intended to be the same. If no one noticed that they changed the formulation perhaps the variations are less important than other factors in producing good profiles.
-
Well I assume from
http://xritephoto.com/ph_product_overview.aspx?ID=938&Action=Support&SupportID=5884
that they are intended to be the same. If no one noticed that they changed the formulation perhaps the variations are less important than other factors in producing good profiles.
the link talks about CC24 Classic and ColorChecker SG - nothing about the Passport or CC24 Classic Mini (which is, I guess, revived, I think I saw a press release that it will be relaunched/was relaunched by now (???)/, at least in UK = http://www.photographyblog.com/news/x_rite_colorchecker_classic_mini_now_available)
-
Well the software for the ColourChecker says
Notes: ColorChecker Camera Calibration software is fully compatible with the industry standard ColorChecker 24 patch classic target from X-Rite. You can use this software with ColorChecker Passport and ColorChecker Classic (standard and mini sizes) products.
So they must surely use the same colour values as the software will not be able to tell which target you used.
-
Well the software for the ColourChecker says
Notes: ColorChecker Camera Calibration software is fully compatible with the industry standard ColorChecker 24 patch classic target from X-Rite. You can use this software with ColorChecker Passport and ColorChecker Classic (standard and mini sizes) products.
"compatible" means it will produce the output from the shot of the target that it will recognize as having patches arranged in a certain manner... you can print yourself something sufficiently resembling CC24 Classic or Passport and that software will take it in... that does not mean it knows which target you are actually feeding it in and what are the target specifications exactly...
So they must surely use the same colour values as the software will not be able to tell which target you used.
look, CC24 Classic formulations were changed in November 2014... the same software will work with both pre and post 11/2014 targets... so try to ask yourself, what is in the code... how does that software distinguish pre and post 11/2014 targets and accounts for the difference... ;D ... the answer is simple - X-Rite created this software as a quick and dirty mean to create profiles aiming @ audience who does not care about which targets they are using in fact and who most probably make so much errors in shooting the target that difference between targets will simply be obliterated by those errors... just like Adobe DNG PE.
-
Wo ho. That's fightin' talk.
In my experience, these commercial people are not complete idiots. They may take shortcuts, but they are calculated shortcuts.
I like the results of Anders' approach better, despite it being GUI-less, and so a PitA to use, even for someone like me, who remembers the DOS world.
The RawDigger approach may be better, but the documentation is quite limited (e.g., the how to on creating camera profiles starts somewhere in the middle and does not explain what sort of profiles you need to generate, for what purpose). In this case, what may be a technically superior solution to XRite's will remain an undiscovered country to those not prepared to do more than follow a simple recipe.
-
I've tried your file and here's a scan (attached).
Anders, the result is more or less as advertised. The Natural+ results are pleasing, subject to the following observations:
- the yellow/greens are perhaps a bit hot (oversaturated)
- it is quite hard to keep grey skies gray -- they are either a touch over blue or over pink
- highlights tend to be a bit detail-less and the shadows over-compressed. I'd prefer darker highlights and more open shadows.
Most of this can be dealt with using the Lightroom Basic panel controls, but the settings required seem to be more extreme than would be ideal.
(All this with the Leica SL and various cityscape shots in either overcast or near sunny light.)
Many thanks for this software, which is leading to much more pleasing pictures than the out-of-box solutions.
-
Wo ho. That's fightin' talk.
no, that's just a simplest explanation... when both X-Rite and Adobe were designing their software they rightfully assumed that the difference in some pigments formulation will not be specifically detected/noticed (and attributed to) in profiles (produced that software) by the audience... can't blame them
-
So which target would you recommend?
-
You can adjust those aspects. The curve use in the default workflow is Adobe's default curve which many find pleasing, but you can design an own lower contrast one to open up shadows and compress highlights less. The neutral+ has a slight saturation increase over neutral, you can find that in its configuration file and disable it. In fact for those that always do color adjustments manually having a slightly desaturated profile can be an advantage, which indeed is an approach Adobe has chosen lately for many cameras.
The neutral+ profile has some desaturation of neutrals, but also keeps highlight color higher up than a typical Adobe profile which could explain your experience of the skies. This too can be adjusted.
Making adjustments is the hardest part, but as long as you're able to see what needs to be adjusted and have some patience you can do it.
I'd suggest using RawTherapee for the curve design, here's a description of how: http://www.ludd.ltu.se/~torger/photography/camera-profiling.html#step8
Personally I don't use the ACR default curve in my profiles, but I've instead matched it with the camera's native rendering and then fine-adjusted to taste.
Anders, the result is more or less as advertised. The Natural+ results are pleasing, subject to the following observations:
- the yellow/greens are perhaps a bit hot (oversaturated)
- it is quite hard to keep grey skies gray -- they are either a touch over blue or over pink
- highlights tend to be a bit detail-less and the shadows over-compressed. I'd prefer darker highlights and more open shadows.
Most of this can be dealt with using the Lightroom Basic panel controls, but the settings required seem to be more extreme than would be ideal.
(All this with the Leica SL and various cityscape shots in either overcast or near sunny light.)
Many thanks for this software, which is leading to much more pleasing pictures than the out-of-box solutions.
-
I'd love to have a GUI myself, and I might put some effort into it in the future, but it wouldn't be for the scanning part (which I think works satisfactory using scanin), but for the look operator design stuff which is really messy and slow to do without GUI. That GUI I would use myself.
To make a broad appeal product the right thing would be to make a good GUI for the scanning part though, and just skip the look operator stuff at all as it's too complicated for casual users anyway. At this point I have no intention making a commercial product though, as the market is just too narrow. I've done doomed-to-lose-money-products before though, so we'll see...
despite it being GUI-less
-
So which target would you recommend?
naturally the one that you 1) can measure yourself carefully, accounting for/correcting by reheating the possible spectrometer drift, averaging, etc and 2) can shoot yourself carefully avoiding improper reflections (like the difference between SG anc Classic - one semigloss and one quite matte)
or you can decide that differences does not matter vs other factors for you and just ignore the #1... a lot of people do ignore both #1 and even #2 and "we" like their pictures still :)
-
To make a broad appeal product the right thing would be to make a good GUI for the scanning part though
or there might be some 3rd party with a proper "ruki.sys" like the author of DisplayCAL who will build the GUI front-end
-
You can adjust those aspects. The curve use in the default workflow is Adobe's default curve which many find pleasing, but you can design an own lower contrast one to open up shadows and compress highlights less. The neutral+ has a slight saturation increase over neutral, you can find that in its configuration file and disable it. In fact for those that always do color adjustments manually having a slightly desaturated profile can be an advantage, which indeed is an approach Adobe has chosen lately for many cameras.
The neutral+ profile has some desaturation of neutrals, but also keeps highlight color higher up than a typical Adobe profile which could explain your experience of the skies. This too can be adjusted.
[:]
Personally I don't use the ACR default curve in my profiles, but I've instead matched it with the camera's native rendering and then fine-adjusted to taste.
Thanks. I've processed a few more images with a neutral+ profile with the ACR adjustment that is meant to avoid the need to adjust color balance when changing profiles.
Generally, it is an improvement over the packaged Adobe Standard Leica SL profile. The oranges are not unpleasant and it works as advertised: foliage looks as if the sun has hit it, close to neutrals neutralised (this is a tad strong, for my taste, unless you are going to use the Margulis PPW to whack up the colours later), skies are blue, not white, etc.
The camera also produces a profile embedded in each DNG (which I assume is a fairly crude one, as it seems to over-emphasise the reds, for example). I don't know what that profile contains or whether it would provide a short-cut to getting a dcamprof-rolled profile that has better colours (eg, spectral data, curves, etc)?
-
The camera also produces a profile embedded in each DNG (which I assume is a fairly crude one, as it seems to over-emphasise the reds, for example). I don't know what that profile contains or whether it would provide a short-cut to getting a dcamprof-rolled profile that has better colours (eg, spectral data, curves, etc)?
You can extract the profile to JSON text format with
dcamprof dcp2json rawfile.dng
the profile looks like this:
{
"UniqueCameraModel": "LEICA SL (Typ 601)",
"CalibrationIlluminant1": "StdA",
"CalibrationIlluminant2": "D50",
"ColorMatrix1": [
[ 1.258700, -0.523200, -0.149600 ],
[ -0.361000, 1.084100, 0.027700 ],
[ -0.091100, 0.167400, 0.207000 ]
],
"ColorMatrix2": [
[ 0.838900, -0.319800, -0.101900 ],
[ -0.383400, 1.022200, 0.066100 ],
[ -0.112200, 0.190000, 0.341500 ]
]
}
That is a dual-illuminant matrix-only profile without separate forward matrices, which is typical for embedded profiles. As no curve is embedded ACR will add the default curve to this on top rather than presenting it as a linear profile.
Indeed this is very crude. You could make a new full-featured DCamProf profile based on those matrices. If the matrices are designed for a good scene-referred match the result could be quite good I guess. You would derive the forward matrices using CAT and then use the neutral tone reproduction operator when applying a curve. Any decent shot of a CC24 is a better start though...
-
Thanks.
I've enjoyed trying this out and even getting a good profile out of it. Better than either the XRite or the Adobe DNG Profiler which, surprisingly, starting from the same image produce completely different profiles.
What are your plans for DCampProf? To me, the most potentially productive next step would be to provide facilities for tweaking the profile in ways that go beyond having to edit a text file, regenerate profiles and restart lightroom, and repeat.
I quite like the Natural+ profile; it does what you say it is designed to do. But I would sometimes prefer different emphasis (no lime green foliage, no neutralising of near neutrals, but better separation of colours, eg). Having something that would allow me to adjust visually would be ideal.
-
Thanks.
I've enjoyed trying this out and even getting a good profile out of it. Better than either the XRite or the Adobe DNG Profiler which, surprisingly, starting from the same image produce completely different profiles.
What are your plans for DCampProf? To me, the most potentially productive next step would be to provide facilities for tweaking the profile in ways that go beyond having to edit a text file, regenerate profiles and restart lightroom, and repeat.
I quite like the Natural+ profile; it does what you say it is designed to do. But I would sometimes prefer different emphasis (no lime green foliage, no neutralising of near neutrals, but better separation of colours, eg). Having something that would allow me to adjust visually would be ideal.
A GUI for making subjective look adjustments is what I'd like to have the most too, the trial-error process with editing the text file is really tedious. I would put effort into that before making a GUI for say patch matching. However I think that would be a six month project at least and the older I get the tougher it it becomes to both have a full-time job and make additional coding on top.
There's also some strong reasons not to do it, while it would make custom design much easier it's something you would do once for every camera you buy or even less. It would be a pretty advanced GUI for something you use extremely rarely.
I really want it myself though, and having a powerful GUI would most likely give some further insights into how camera colors work in practice, so we'll see.
Other core aspects I'm more curious about investigating further is precision of high saturation colors, and work more with gamut compression. I'm also curious about looking into making a custom color space adapted specifically for general-purpose camera profiling, I have a few ideas there I'd like to test to see if they're feasible. The things that interest me the most is the unmeasurable "psychovisual" aspects of color which plays a central role when you make profiles for general-purpose photography rather than reproduction.
-
A GUI for making subjective look adjustments is what I'd like to have the most too, the trial-error process with editing the text file is really tedious. I would put effort into that before making a GUI for say patch matching. However I think that would be a six month project at least and the older I get the tougher it it becomes to both have a full-time job and make additional coding on top.
There's also some strong reasons not to do it, while it would make custom design much easier it's something you would do once for every camera you buy or even less. It would be a pretty advanced GUI for something you use extremely rarely.
I really want it myself though, and having a powerful GUI would most likely give some further insights into how camera colors work in practice, so we'll see.
Other core aspects I'm more curious about investigating further is precision of high saturation colors, and work more with gamut compression. I'm also curious about looking into making a custom color space adapted specifically for general-purpose camera profiling, I have a few ideas there I'd like to test to see if they're feasible. The things that interest me the most is the unmeasurable "psychovisual" aspects of color which plays a central role when you make profiles for general-purpose photography rather than reproduction.
I'd say that for may a mass (which is of course a strange thing for a command line tool, but still) user some specific command line options/keys to generate profiles aimed for either @ "landscape" (properly "green" grass and properly "blue" skies + proper constructing LUTs in the areas of such tones) or @ "portraiture" (properly "colored" skin + proper constructing LUTs in the areas of such tones)... granted you can achieve that with a lot of existing options, but I mean specifically a "hoi polloi" options like -aim-skin or -aim-landscape type of... $0.02
-
I'd say that for may a mass (which is of course a strange thing for a command line tool, but still) user some specific command line options/keys to generate profiles aimed for either @ "landscape" (properly "green" grass and properly "blue" skies + proper constructing LUTs in the areas of such tones) or @ "portraiture" (properly "colored" skin + proper constructing LUTs in the areas of such tones)... granted you can achieve that with a lot of existing options, but I mean specifically a "hoi polloi" options like -aim-skin or -aim-landscape type of... $0.02
This would be great. The last weeks I tried to figure out the -a paramater on make-target without much success. When I set "AdjustJCh" for one field, this has an impact also on the other field. That's logic to me. But I really don't get, what values I have to set. And with about 5 fields (e.g all skin color related fields), I'm completely lost. Let's say I have A1 (3.05 DE LCh +2.05 -2.08 0.90) and A2 (3.90 DE LCh +2.55 +1.78 2.36), what would be the "AdjustJCh" for these two fields to get them close to 0?
The idea of AlterEgo would be a good workaround for most situations: "for landscape", "for portrait", "for general use".
Or if I can priorise the fields with rankings. Or the hue values are more important, than lightness and chroma.
And of course all within a GUI, where I can the impact directly (after calculations), instead of using "test-profile" each time. :)
-
There appear to be two conflicting objectives:
= make the GUI usable by the masses for intuitive profile refinement, a functional, results-oriented tool
= make the GUI powerful, giving full direct access to the capabilities of dcamprof, for research purposes
It may be that you need to build the latter in order to get to the former.
I don't think that profiling would be something that you would just use once (we change cameras every 1-2 years, and we want a range of different "look"s for different types of shooting).
-
I really want it myself though, and having a powerful GUI would most likely give some further insights into how camera colors work in practice, so we'll see.
If you are thinking about going the GUI way and perhaps even cross platform, I'd recommend QT library. A couple of years ago had to do this myself (some cross platform tools for Kodak ProBacks) and doing it to work simultaneously on Mac and Windows was a really easy exercise. Also ported a few Photoshop plugins using QT from Windows to Mac and it was relatively easy task. The learning curve (if you have not used it) is also not that steep...
-
This would be great. The last weeks I tried to figure out the -a paramater on make-target without much success. When I set "AdjustJCh" for one field, this has an impact also on the other field. That's logic to me. But I really don't get, what values I have to set. And with about 5 fields (e.g all skin color related fields), I'm completely lost. Let's say I have A1 (3.05 DE LCh +2.05 -2.08 0.90) and A2 (3.90 DE LCh +2.55 +1.78 2.36), what would be the "AdjustJCh" for these two fields to get them close to 0?
I guess you mean -a on make-profile, that is provide a target adjustment file. This file is used to change the values in the target to the "wrong ones", but in a subjective way. The main purpose is as a side effect to control the result of the matrix optimizer. A typical thing would be to say that the deep blue CC24 patch is lighter than it actually is if you want to avoid to strong blue subtraction in the matrix (but usually -y parameter can solve that problem).
A problem with matrix optimization is that the problem is "unsolvable", if you make one patch match better another match worse and an optimizer tends to pull towards certain solutions regardless of weights (because they're still the least bad ones despite other weights), so to really change the matrix result more directly one have to adjust the target, and that's where the -a comes in. But even when adjusting the target the results can seem somewhat "random" at times, simply because the linear matrix can only fit the data that good.
An advanced technique which I sometimes use is to develop the matrix first, using target adjustment, and then store the matrix and do another run with the LUT against a target without adjustment. Unlike the matrix optimizer the LUT responds very well to weighting, -w and -l, and the LUT can pull errors to zero if you want that (unless you have several patches on the same chromaticity), but some relax is recommended, it's only a target we're matching after all.
Anyway, before jumping into -a you should probably play around with the LUT and -l.
-
If you are thinking about going the GUI way and perhaps even cross platform, I'd recommend QT library. A couple of years ago had to do this myself (some cross platform tools for Kodak ProBacks) and doing it to work simultaneously on Mac and Windows was a really easy exercise. Also ported a few Photoshop plugins using QT from Windows to Mac and it was relatively easy task. The learning curve (if you have not used it) is also not that steep...
I'm more used to GTK via other projects (like RawTherapee), but still I would probably do it in QT as it's better maintained.
-
There appear to be two conflicting objectives:
= make the GUI usable by the masses for intuitive profile refinement, a functional, results-oriented tool
= make the GUI powerful, giving full direct access to the capabilities of dcamprof, for research purposes
It may be that you need to build the latter in order to get to the former.
Yes I agree, I think I need to do the later to be able to do the former in the best way. That is by making a powerful GUI one can learn more about "what matters" and put that into a simplified GUI. And for myself I rather have a powerful complicated GUI that almost only I can use and learn a lot from that, than only a simple GUI that doesn't contribute to developing my understanding of camera color.
-
I'm more used to GTK via other projects (like RawTherapee), but still I would probably do it in QT as it's better maintained.
Personally I just find QT a lot easier from portability prospective - GTK after all was not designed to be crossplatform from scratch...
-
Personally I just find QT a lot easier from portability prospective - GTK after all was not designed to be crossplatform from scratch...
QT vs GTK is much like a Mac vs PC or Nikon vs Canon thing in the open-source community, but usually it's politics rather than features that is discussed. QT is the better designed toolkit, but the ownership jumping around between different companies with unclear directions have not been good for the credibility. As free community-developed open-source the GTK project has better credibility, but it's development/maintenance has not been satisfactory. For the moment it seems to have gained a bit of speed though.
Anyway, I'm not starting this project tomorrow...
-
Well I made a couple of further profiles and the result is, to me, quite interesting. If I use the -m option to copy the white balance out of the Adobe profile in many pictures there is almost no change between the Adobe profile and the DCamProf ones (natural, natural+).
This is with the latest version of Lightroom; I don't know whether they updated the profile for the Leica SL. Yes, there are differences in very dark, saturated blues, violets and oranges, but these tend not to be critical, in the sense that if they are off, no one will complain, in general. So, assuming that Adobe have created their profiles using some expensive equipment, DCamProf has more or less matched it and the tweaks in the + profile can be attractive. The DCamProf profile darkens blue skies a bit, for example.
It is striking that these profiles are almost invariable better than ones created with the XRite ColorChecker software, or even Adobe's profile editor. I should be able to use the latter to identify where the various profiles differ, but haven't got around to figuring it out yet.
There are a couple of things that stand out, however. Grey, overcast clouds seem to render much bluer than they appear in real life; setting them grey, with white balance, makes everything else look far too warm. This is with both Adobe and DCampProf DCP profiles. I don't know whether this has something to do with the UV filters that I use, rather than a profile issue.
-
There are a couple of things that stand out, however. Grey, overcast clouds seem to render much bluer than they appear in real life; setting them grey, with white balance, makes everything else look far too warm. This is with both Adobe and DCampProf DCP profiles. I don't know whether this has something to do with the UV filters that I use, rather than a profile issue.
They are never grey. Pollution filters light and short waves are more affected than longer. That's why anything tend to blue in a certain distance. Our brain does adapt this. So we "think to see", that they are grey.
But beside that, did you check your profile with test-profile? You can look at the errors.tif to see the differences visually. How about to white balance with the preset "cloudy" or a grey card (instead of white balance on the clouds).
-
Well I made a couple of further profiles and the result is, to me, quite interesting. If I use the -m option to copy the white balance out of the Adobe profile in many pictures there is almost no change between the Adobe profile and the DCamProf ones (natural, natural+).
That is good, it simply means that all those profiles have the intention to produce natural and realistic colors. Profiles that are designed for that will all look very similar, it's only when you study details you discover differences. Probably all profiles are using the ACR default curve too which helps in the similar look as the contrast is the same.
Adobe's profiles are not all designed in the same way or even for the same look, so there's great variation depending on camera on what you get. Some profiles are good, some are not so...
-
Anyway, before jumping into -a you should probably play around with the LUT and -l.
I'm using both now (-a and -l) and made three profiles for different purposes (good general match, great general match (except A1, which is too bright and not that saturated) and one that reduces very saturated orange to magenta).
It's great to have all these possibilities to tweak a profile. :)
-
A question about including gamut compression in the generated profile.
With Lightroom is it preferable to
(a) include prophotoRGB gamut compression and then get Lightroom to compress down further to sRGB or your printer's RGB profile when exporting/printing; or
(b) include sRGB gamut compression in the profile itself?
-
A question about including gamut compression in the generated profile.
With Lightroom is it preferable to
(a) include prophotoRGB gamut compression and then get Lightroom to compress down further to sRGB or your printer's RGB profile when exporting/printing; or
(b) include sRGB gamut compression in the profile itself?
Given how large ProPhoto is, it would hardly be a "compression" :)
The default seems to be compression to AdobeRGB, and this is what I use with my profiles (for Raw Therapee). I also compared it to the same profiles without compression, the differences are visible e.g. for very saturated reds (flowers etc.), which look a bit tamer and less saturated with Adobe RGB compression.
-
Ok. Thanks. But do they look better if you use prophoto / no gamut compression and use Lightroom to produce AdobeRGB output?
-
Ok. Thanks. But do they look better if you use prophoto / no gamut compression and use Lightroom to produce AdobeRGB output?
I guess it depends what you mean by better ;) Also, I don't have Lightroom, I only use Raw Therapee, but I guess the trade-offs with using compression are similar in both programs.
Usually I stick to the gamut compression with AdobeRGB because it is better at avoiding oversaturation and blowing out of colors. On the other hand, I also had some images which relied on very saturated, wild colors, these looked better with compression disabled. I guess it's good to have both options and use them based on the artistic results you want to achieve. For a general purpose profile, I'd probably go with the "safer" AdobeRGB gamut compression.
-
It depends on your hardware, workflow and output. If you only work on a sRGB-capable monitor and process for an sRGB-jpg at the end, it may be good to start with a profile that fits. On the other hand, when you use an expensive wide gamut display and do prints, why compressing?
-
To be clearer:
Suppose that I am aiming for sRGB for the final JPEG.
I have a choice of compressing the gamut early (in the DCP) or late (as part of the final generation of the JPEG).
is one route clearly better than the other?
-
...
Suppose that I am aiming for sRGB for the final JPEG.
...
If you never ever print and only go for sRGB, there is no reason to wait until the end.
(We're a little off topic btw)
-
To be clearer:
Suppose that I am aiming for sRGB for the final JPEG.
I have a choice of compressing the gamut early (in the DCP) or late (as part of the final generation of the JPEG).
is one route clearly better than the other?
If the raw converter has a good gamut compression algorithm it would be best if the DCP didn't compress at all. However most raw converters expects that the DCP has some compression in them, as their own compression isn't too good at handling large compression ranges. Targeting the profile for AdobeRGB and outputting sRGB is usually fine, I'd recommend that for LR.
Note that the gamut compression in DCamProf is not "mathematically exact", in it's standard configuration it allows quite some clipping as it looks better in practice, a sunset is a good test scenario. If compressed mathematically exact it doesn't become as bright as it should but flat and dull.
-
Thanks, Anders. That's helpful.
-
I finally got around to make my own Windows and Mac OS X builds, now available for download:
http://www.ludd.ltu.se/~torger/dcamprof.html#download
It's a new version too, version 1.0.1, but the code changes are really minimal so if you have 1.0.0 already running there's no hurry to upgrade.
I'd like to thank Howard and AlterEgo for making builds up to now. I plan to do my own builds from now on for Windows 64 bit and Mac OS X, so there's no need to make third-party builds for those platforms, but it's allowed of course as the software is free.
-
First off, you're welcome :) I'm glad to help the community where possible.
Second, I've been building mine fully static & without OMP. I see your new binary................
HJM-MBA:~/Desktop/dcamprof-1.0.1-macosx] howardm% otool -L dcamprof
dcamprof:
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.5)
/usr/local/opt/llvm/lib/libomp.dylib (compatibility version 5.0.0, current version 5.0.0)
and the dreaded dylib linker error:
HJM-MBA:~/Downloads] howardm% ../Desktop/dcamprof-1.0.1-macosx/dcamprof
dyld: Library not loaded: /usr/local/opt/llvm/lib/libomp.dylib
Referenced from: /Users/howardm/Downloads/../Desktop/dcamprof-1.0.1-macosx/dcamprof
Reason: image not found
Trace/BPT trap
I'm not sure its reasonable for 99.9% users to have to install entire 'port' or 'fink' systems just to have libomp.
Is there enough of a runtime performance 'win' to really care about using OMP?
I suppose the libomp.dylib could be distributed w/ dcamprof.
I haven't looked into the whether libomp can be built as a old-time .a library for inclusion during build.
-
Oh, my mistake in building, thanks for reporting. I'll look into it. I'll pull the Mac OS X build for the moment until I've come up with a suitable solution.
-
Is there enough of a runtime performance 'win' to really care about using OMP?
a "hoi polloi person" 'd assume that if there is matrix optimization -s used then it is, no ?
PS: does Win64 version include OMP statically linked inside binary executable ?
-
a "hoi polloi person" 'd assume that if there is matrix optimization -s used then it is, no ?
PS: does Win64 version include OMP statically linked inside binary executable ?
Yes, unless I've made some mistake there too, the OpenMP should be statically linked in that case. I'm generally working in Linux so I'm a bit of a beginner messing around on Windows and Mac linking/building.
OpenMP is not strictly necessary no, but some aspects of the software run significantly faster. The standard Clang has had OpenMP support for a while, but unfortunately Apple have rolled their own Clang version which doesn't have OpenMP, so it's a bit of a mess to get working there.
-
Honestly, I have yet to actually use the tool to generate a profile but if the common usages of it run N seconds longer w/o using OMP, at what value of N does the pain become significant (esp considering the tool itself isn't run that often)??
To the users of my earlier builds, does the runtime seem excessive?
-
When you start generating spectra for SSF simulations it can be nice with a 8-fold speedup, as runtime can be minutes. Otherwise the speedup is in most cases fairly cosmetic as it's fast enough anyway. I as a developer sure wants my parallel coding to be used though when I've done it, so when I make an own build I want OpenMP to be in. So far it does seem like one have to distribute libomp.dylib with the package though if to stay with standard procedures, that's how others do it.
Apple has their own "GCD" for parallel coding, I've coded a bit of that too, but I think it's a pain to use different proprietary technologies to make parallel coding, OpenMP is the best standard way out there, which unfortunately Apple has been slow to adapt despite that the standard Clang package has it.
-
I would leave it w/ OMP.
Assuming there isn't a sane way of creating libomp.a...............
Perhaps:
1. distribute libomp.dylib and have user install it (or provide script) in /usr/local/lib
2. compile w/ -R or -runpath to force the linker to look there w/o env. vars.
-
Yes, unless I've made some mistake there too, the OpenMP should be statically linked in that case.
how do you manage that ? I am not a software developer at all and when I tried to search "THE INTERNET" I always saw using libgomp***.dll (means non static) - if you don't mind to comment on this (what was the toolchain and the makefile used to build the static version on windows) ? I am sorry if it is a stupid question
-
I've put back out a mac build, hopefully it works as it should now. There is that extra file libomp.dylib in the package, I figured if you can copy "dcamprof" file to "/usr/local/bin", one should manage to copy "libomp.dylib" to "/usr/local/lib". If one can't get past that, DCamProf is probably too complicated to use anyway ;)
It's technically possible to make a libomp.a and link statically, but then you need to make it yourself from source, which is a mess. The reason there typically is no static version of the OpenMP lib in most distributions is because if more than one library in your application depend on it, there will be performance issues and possibly errors, as there can only be one OpenMP instance in a binary. It would be no problem in this case, but I'm not in charge of the Clang toolchain... hopefully Apple will eventually include OpenMP in their own Clang and then libomp.dylib will most likely become a standard library, just as in Linux.
That it can be static in Windows is because I'm using the MSYS2 distribution and they have made an exception to this rule and provided a static option also for OpenMP. I'm using MSYS2/MinGW64.
-
how do you manage that ? I am not a software developer at all and when I tried to search "THE INTERNET" I always saw using libgomp***.dll (means non static) - if you don't mind to comment on this (what was the toolchain and the makefile used to build the static version on windows) ? I am sorry if it is a stupid question
Certainly not stupid question, I've also been through all THE INTERNET and it's a mess with all different compiler versions and distributions out there. I was actually surprised that it worked statically with MSYS2/MinGW64, it was not something I actively searched for, it just worked. My plan was to distribute the DLL together with the binary just as I do with OS X now, but it turned out to not be required as there's a libgomp.a with the MSYS2 distribution.
Makefile is the same as the distributed Linux makefile, but with the added "-static" option to LDFLAGS and added "-ljpeg -llzma -lz" on the libraries (DCAMPROF_LIBS), then built inside a mingw64_shell that comes with MSYS2 (if you install it using the package management pacman). I don't know exactly which packages I have installed as the machine used for compiling is used for other projects too, and MSYS2 was not installed for this.
-
Certainly not stupid question, I've also been through all THE INTERNET and it's a mess with all different compiler versions and distributions out there. I was actually surprised that it worked statically with MSYS2/MinGW64, it was not something I actively searched for, it just worked. My plan was to distribute the DLL together with the binary just as I do with OS X now, but it turned out to not be required as there's a libgomp.a with the MSYS2 distribution.
Makefile is the same as the distributed Linux makefile, but with the added "-static" option to LDFLAGS and added "-ljpeg -llzma -lz" on the libraries (DCAMPROF_LIBS), then built inside a mingw64_shell that comes with MSYS2 (if you install it using the package management pacman). I don't know exactly which packages I have installed as the machine used for compiling is used for other projects too, and MSYS2 was not installed for this.
I shall experiment w/ that tonight, thank you !
-
Yes, unless I've made some mistake there too, the OpenMP should be statically linked in that case. I'm generally working in Linux so I'm a bit of a beginner messing around on Windows and Mac linking/building.
OpenMP itself does not recommend static linking since a few things can go wrong. Depending how you compiled it for the mac, you may be able to statically link it as well - I did this before for gcc 4.7 build with support for OMP for my own QT based projects but abandoned the approach in favour of bundled shared OMP library.
-
OpenMP itself does not recommend static linking since a few things can go wrong. Depending how you compiled it for the mac, you may be able to statically link it as well - I did this before for gcc 4.7 build with support for OMP for my own QT based projects but abandoned the approach in favour of bundled shared OMP library.
Yes I know, and that recommendation has caused most distributions to not include a static OpenMP lib. There's a million things in programming that can go wrong though, this is just one thing. If you know that none of your dependencies are using OpenMP you can link statically, and that's not too hard to find out, so I think it's a bit unfortunate that they are so stiff about it. I agree that eventually it should be dynamic, the real problem is that Microsoft and Apple are slow to adopt portable standards (as they like proprietary solutions better to lock in developers in their own platforms) so they don't provide any OpenMP lib in their platforms, but as soon as they do (if ever) dynamic is the right thing. Meanwhile being stiff about not making static libs just causes additional irritation for us developers.
Oh well, to be fair Microsoft actually have OpenMP support in their compilers. It's very difficult to make portable software that builds with Microsoft compilers in general though (especially if you have advanced stuff like inline asm and such), (both Linux and Mac OS X are Unix-based and Clang/GCC has very similar behavior, while Windows and their compilers are all different) that's why MinGW is used which brings Windows much closer to the other two.
Anyway, using GCC has become much more cumbersome in later OS X versions (don't remember the exact reason, but there's something about linking I think), so I'm using Clang. I'm awaiting that Apple should include OpenMP in their own Clang distro so I can just run with the default Apple toolchain, but so far one have to use a third-party standard Clang to get OpenMP. I've installed via "brew", so it's easy to get going there's no longer need to compile your own compiler (which I had to do a few years ago on the Mac).
-
Anyway, using GCC has become much more cumbersome in later OS X versions (don't remember the exact reason, but there's something about linking I think), so I'm using Clang. I'm awaiting that Apple should include OpenMP in their own Clang distro so I can just run with the default Apple toolchain, but so far one have to use a third-party standard Clang to get OpenMP. I've installed via "brew", so it's easy to get going there's no longer need to compile your own compiler (which I had to do a few years ago on the Mac).
Same here. I have not used brew - actually gotten the Clang officially build one for OS X and made it into a toolchain so it plugs nicely to XCode. With that I am able to use it just like standard bundled Apple version of clang with all the bells and whistles...
-
I'm working on an update, which this time actually adds a new feature :-). Over the past year I've got some user feedback of various kinds which all I can translate to that gamut compression is not as strong as people are used to from the bundled commercial profiles. The current effect of DCamProf's gamut compression is indeed quite mild, which cause some scenes to clip quite badly compared to what they do with a typical commercial profile. I've always thought that it's bad design to have static gamut compression in the profile, it should instead be dynamic and handled by the raw converter, but well, that's not the way it is so I just have to deal with it.
So I have studied how to make a really strong compression that you see for example in Capture One's profiles. Quite early on I found a good way to strongly compress colors, but making the effect gradual towards the gamut edges without hurting gradients or bleeding too much into low saturation colors turned out to be very difficult.
I've come about 80% to the goal though. I think. Sometimes when you start testing more broadly you discover that the approach doesn't hold up and you need to get back to the drawing board. We'll see. If all goes well I should be able to make a new release within a couple of weeks.
-
I'm working on an update, which this time actually adds a new feature :-). Over the past year I've got some user feedback of various kinds which all I can translate to that gamut compression is not as strong as people are used to from the bundled commercial profiles. The current effect of DCamProf's gamut compression is indeed quite mild, which cause some scenes to clip quite badly compared to what they do with a typical commercial profile. I've always thought that it's bad design to have static gamut compression in the profile, it should instead be dynamic and handled by the raw converter, but well, that's not the way it is so I just have to deal with it.
So I have studied how to make a really strong compression that you see for example in Capture One's profiles. Quite early on I found a good way to strongly compress colors, but making the effect gradual towards the gamut edges without hurting gradients or bleeding too much into low saturation colors turned out to be very difficult.
I've come about 80% to the goal though. I think. Sometimes when you start testing more broadly you discover that the approach doesn't hold up and you need to get back to the drawing board. We'll see. If all goes well I should be able to make a new release within a couple of weeks.
may be a stupid question - is it a good idea (or possible usability-wise) to have a parameter controlling which "colors" (shall be some notation allowing to indicate which colors do you mean) to compress "more" or "less" ?
-
I'm working on an update, which this time actually adds a new feature :-). Over the past year I've got some user feedback of various kinds which all I can translate to that gamut compression is not as strong as people are used to from the bundled commercial profiles. The current effect of DCamProf's gamut compression is indeed quite mild, which cause some scenes to clip quite badly compared to what they do with a typical commercial profile. I've always thought that it's bad design to have static gamut compression in the profile, it should instead be dynamic and handled by the raw converter, but well, that's not the way it is so I just have to deal with it.
So I have studied how to make a really strong compression that you see for example in Capture One's profiles. Quite early on I found a good way to strongly compress colors, but making the effect gradual towards the gamut edges without hurting gradients or bleeding too much into low saturation colors turned out to be very difficult.
I've come about 80% to the goal though. I think. Sometimes when you start testing more broadly you discover that the approach doesn't hold up and you need to get back to the drawing board. We'll see. If all goes well I should be able to make a new release within a couple of weeks.
Until it gets to the RAW converter how would it even know what the target colorspace gamut is? What kind of compression does C1 do and what assumptions re gamut does it make?
-
What kind of compression does C1 do
icc/icm camera profiles used by C1 naturally guide the color transfrom (somewhere at the end of the internal pipeline) from camera's digital numbers to coordinates in a proper PCS (cie***/D50, *** = Lab or XYZ, OEM profiles are Lab, but C1 will digest profiles with *** = XYZ too) - so he certainly can see what LUTs inside icc/icm container do.... and then it is from that PCS to the selected colorspace for the output /unless we are doing output for profiling/... no ?
-
Until it gets to the RAW converter how would it even know what the target colorspace gamut is? What kind of compression does C1 do and what assumptions re gamut does it make?
Exactly, that's why I don't think it's particularly good design to make it in the profile, but that's what's "everybody" does, and raw converters have as a result limited control of large gamuts and thus makes it harder to work with profiles that doesn't compress a lot, when you shoot high saturation subjects.
The assumption seems to be that target color space is something similar to AdobeRGB, but there's not really a fixed target gamut, it's more like "with proper handling of clipping it should look good in AdobeRGB sized gamuts". Compressing camera gamuts is quite different from screen-to-printer gamut mappings, as with the camera you don't really know what you're going to get. Few "real" colors trigger a raw response of 0% R, 100% G and 0%B for example, so the outer limit of what the camera will deliver is sort of unknown. It's popular to say "camera's don't have a gamut", and this is one of the reasons.
Another aspect is that cameras can deliver very saturated colors, far outside AdobeRGB. The amount of compression required is typically much larger than regular gamut-mapping tasks, so you need to apply different much more subjective methods rather than just using colorimetric formulas.
I can't describe exactly how Capture One does its gamut compression, as I don't know. I haven't really studied it in detail, I try to come up with the best method I can do based on own research. I do look at Capture One and Hasselblad etc for sanity checking and compare performance. One thing I can say though is that to achieve higher compression you don't only work along the saturation axis, but also lightness, darken saturated colors to fit in gamut, and lighten saturated shadows (as colors can clip to zero too, not just max value). Evenso you must still consider scenarios when the camera does clip, like sunsets. "Mathematically accurate" gamut compressions make sunsets and other scenarios involving clipping look dull.
Making pleasing gamut compression for cameras is a little bit like making pleasing skin tones. It's not so much science, but more trial and error and judging the result by eye. It's not about putting the camera into a fixed gamut range, but to dampen the outer range so it behaves a bit "nicer", so when you shoot that deep purple flower you don't immediately need to pull lots of sliders in the raw converter to reduce clipping into reasonable levels, but instead it provides a pleasing output without adjustments.
As I've mentioned many times, camera profiles are still today intended to work as a "film roll", it should provide a well-behaved look within a reasonable gamut without requiring further adjustments of the raw converter, automatic or manual. It's not the way I'd like it to be, but it's legacy from the analog days and it's not going away soon.
-
may be a stupid question - is it a good idea (or possible usability-wise) to have a parameter controlling which "colors" (shall be some notation allowing to indicate which colors do you mean) to compress "more" or "less" ?
Yes it's probably going to be some hue-control, although few will change the default setting. I've tried to used equal compression over the whole hue range, but it seems now that's not going to yield the best results. For example it seems that yellows cannot be compressed as much as reds for the most pleasing result. I'm investigating these aspects just now.
-
Hi Anders, I hope the old way stays available as an option? The tech stuff is over my head, but I have more confidence in you than in what "everybody" does just for historic reasons, that is without good reason. - I highly appreciate the way you document your programs.
Best regards - Hening.
-
I don't profess to understand the pipeline either, but I do find that on the Sony A7r II and Leica SL, I need to add saturation to get a pleasing picture (which I am happy to do, rather than reducing saturation after if it comes overdone).
-
Hi Anders, I hope the old way stays available as an option? The tech stuff is over my head, but I have more confidence in you than in to what "everybody" does just for historic reasons, that is without good reason. - I highly appreciate the way you document your programs.
Best regards - Hening.
Yes the old way will be there, you will still be able to make profiles entirely without gamut compression, with the old milder gamut compression, and then a new option to make this new stronger compression.
-
I don't profess to understand the pipeline either, but I do find that on the Sony A7r II and Leica SL, I need to add saturation to get a pleasing picture (which I am happy to do, rather than reducing saturation after if it comes overdone).
Indeed this is often the case. Gamut compression I'm talking about here is in the "extreme color" range, colors you find in say deep red flowers, artificial colored lights, artificial colored toys etc. In most "normal" scenes you don't come across colors that needs compression, so it's a quite narrow case. In my own photography I have really never had problems with over-saturated colors. I would post examples to make it clearer, but now my test images are mostly "problem images" that users have sent me which I can't post. Maybe when I'm a bit more finished with the stuff I can post some before/after examples.
-
Exactly, that's why I don't think it's particularly good design to make it in the profile, but that's what's "everybody" does, and raw converters have as a result limited control of large gamuts and thus makes it harder to work with profiles that doesn't compress a lot, when you shoot high saturation subjects.
If I understand you correctly, this compression is an attribute only for LUT profiles. In this case I think it probably meant to have less errors as LUT attempt to "fix" some of the colour problems and have smoother interpolation. My understanding is that matrix profiles do not compress so I would not really attribute this to all raw converters as default behavior.
-
If I understand you correctly, this compression is an attribute only for LUT profiles. In this case I think it probably meant to have less errors as LUT attempt to "fix" some of the colour problems and have smoother interpolation. My understanding is that matrix profiles do not compress so I would not really attribute this to all raw converters as default behavior.
Yes it's only applicable to LUT profiles. By nature matrix profiles are 100% linear and thus put responsibility on the raw converter concerning tone reproduction if you apply a curve, and gamut compression if you need that. The big name raw converters like LR and C1 are not designed for that, but expect the profile to apply a tone curve, make subtle look adjustments, and then some gamut compression, and for all that a LUT is required. They will of accept matrix profiles, and yes then you are correct that no such adjustments will then be applied. Personally I think the rendering then falls short, so I don't think matrix-only profiles are a very good to use in either LR or C1, but it's a matter of taste.
-
Personally I think the rendering then falls short, so I don't think matrix-only profiles are a very good to use in either LR or C1, but it's a matter of taste.
Depends on a camera. Works perfectly with Kodak SLR/n and Proback. I personally find LUT profiles less predictable and of questionnable quality so always try to build matrix ones.
-
Here's an example. I've borrowed one of the test images I've got from a user, cropped out a detail so I hope it's okay to show it here.
There are three images, first without gamut compression (no-gamut), then with the weaker old compression (adobergb-old) and then with the upcoming newer stronger compression (adobergb-strong). The compression is configurable so one can tune it, but I expect most to use my presets either "adobergb" if you don't need to strong compression, or the new "adobergb-strong" if you want stronger compression, more similar to what you find in bundled profiles.
I'm doing some final testing and hope to get a release out within the next few days.
The new strong compression is just adding a pre-processing step to the old compression. In the pre-processing the (Prophoto) RGB max and min values are compressed (=HSV Value and Saturation), so you see a darkening effect of highlights and lightening of shadows of the saturated colors. The strength of compression is controlled by how far the inner gamut (AdobeRGB in this case) is from the outer gamut (ProPhoto) for each hue, plus some configurable parameters. The hard part was finding suitable amount of compression, making smooth compression curves and blending it in smoothly to not hurt gradients.
For this particular image I've compared with Capture One's compression, and the "adobergb-strong" compression is in the same ballpark concerning strength.
Note that this is not pure gamut mapping, there's some loseness and clipping required as one doesn't actually know what the outer gamut limit will be, and one need to leave some clipping to avoid dullness, so I've tested through a bunch of images to find a "suitable amount" of compression.
The attached images are sRGB, and indeed even if the output is going to be sRGB compressing towards AdobeRGB is usually the best idea, and let even more saturated colors clip. If you need even stronger compression there are sRGB presets though.
-
I've now released v1.0.2, with the new stronger gamut compression, which is optional. The old is still in and works the same.
What's new:
http://www.ludd.ltu.se/~torger/dcamprof.html#news
Downloads (win, mac, source):
http://www.ludd.ltu.se/~torger/dcamprof.html#download
-
A v1.0.2 fully static binary for Macintosh is available below (from box.com, the MS OneDrive has not)
-
Its working great on my personal red-blowout shots. :D
-
I may have to actually use dcamprof. I have a Fuji XT2 and took some images of British Redcoats (and they are VERY red) and they really can use some help.
-
This thread certainly got more interesting.
From editing the AdobeStrong version of the red pants image in ACR 6.7 I can't help but think there's got to be some issues caused by the display profile (or video hardware) mucking up a camera profile's ability to properly visually map luminance/saturation to the display that may have nothing to do with color gamut spaces.
Since the red pants image is in sRGB (edited on an sRGB-ish display) I still could make the highlights appear brighter but noticed each slider in ACR 6.7 raw converter acted upon the preview in different ways where even point curve edits created a wack-a-mole response between increasing luminance vs saturation. I can see how a LUT based profile can cause issues as well. The point curve edits required a raised flat, mesa shaped (not arched) curve segment for brightening the red pants highlights in order to retain definition (override compression effect) and yet it's in low gamut sRGB.
And the Lab L* channel gives the same highlight readouts on all three versions posted. Clearly there's also a disconnect between how the numbers define a human perceptual appearance model and reality.
Good work, Anders, I really learned a quite a bit in this thread.
-
Just released v1.0.3. I had accidentally broke curve parsing in v1.0.2 so this is a pure bugfix release. Without it it's not possible to complete a full Capture One workflow with custom curves, so it's an important fix.
http://www.ludd.ltu.se/~torger/dcamprof.html#download
-
I seems canons picture style format is well on it's way to getting hacked http://magiclantern.fm/forum/index.php?topic=16299.125 I would be awesome if you could use DCamProf generated profiles directly in camera.
-
I seems canons picture style format is well on it's way to getting hacked http://magiclantern.fm/forum/index.php?topic=16299.125 I would be awesome if you could use DCamProf generated profiles directly in camera.
If the "LUT3D" field really provides a LUT and it's not only about simple adjustments on top of fixed base profiles it could be possible to load profiles there. However I wouldn't put any massive effort into it, but say if the magic latern project provides a way to load straight ICC profiles into that LUT so it's simple to generate a profile for it I could do it.
It sure would be interesting and it's really how cameras should work -- it's a mystery how closed cameras still are. That you really can't load third-party profiles into the camera is just odd, it should have been possible since cameras became digital but the camera industry is just so incredibly conservative.
-
Outstanding stuff Anders. And your explanations are both clear and show a deep understanding of color science. It's also very useful for those of us that need to do repro work from time to time.
-
Just released v1.0.3. I had accidentally broke curve parsing in v1.0.2 so this is a pure bugfix release. Without it it's not possible to complete a full Capture One workflow with custom curves, so it's an important fix.
http://www.ludd.ltu.se/~torger/dcamprof.html#download
I have a question... I did not make pure matrix profiles for some (long) time and it seems I forgot/miss something obvious - why I can't get rid of "LookTable" in dcp profile unless I put '-h 1,1,1' as a parameter for 'dcamprof make-dcp' ? I think I remember that it used to be that '-L' parameter alone was enough, no ?
Thank you !
-
If the "LUT3D" field really provides a LUT and it's not only about simple adjustments on top of fixed base profiles it could be possible to load profiles there. However I wouldn't put any massive effort into it, but say if the magic latern project provides a way to load straight ICC profiles into that LUT so it's simple to generate a profile for it I could do it.
It sure would be interesting and it's really how cameras should work -- it's a mystery how closed cameras still are. That you really can't load third-party profiles into the camera is just odd, it should have been possible since cameras became digital but the camera industry is just so incredibly conservative.
I agree cameras are excessively closed down. I think these efforts could open a lot of doors for different workflows concerning repro work. RAW is still king but if you can give it up ML's silent shutter can be used to minimize shutter wear for timelapses and astro or increase the speed of motion detection. In conjunction with a DCamProf generated profile it make a real killer feature. Then there's video.
-
I have a question... I did not make pure matrix profiles for some (long) time and it seems I forgot/miss something obvious - why I can't get rid of "LookTable" in dcp profile unless I put '-h 1,1,1' as a parameter for 'dcamprof make-dcp' ? I think I remember that it used to be that '-L' parameter alone was enough, no ?
Thank you !
Yes it should only be '-L', but that parameter has apparently become broken. I'll fix to next release. Meanwhile you can get the original behavior by adding '-L -o standard'.
-
Anders, thanks for your great work.
In my oppinion the handling of the brights and darks with a DCamprof C1-Profile was already very well. The new function may be useful in some situations. I will use it, when the other profile has issues.
Here are some comparisons.
(note: the (linear curved) C1-profile was made with an IT8/7.2 target in sunlight (d50, dEmax5.16, dEavg1.65)). Maybe there are differences in the gamut compression to other targets)
-
and two more...
I pushed the saturation a little bit on 3strong.jpg to be comparable to 3normal.jpg (except in the reds/yellows).
-
Thanks for the example images Sebbe. For anyone that wants to compare I recommend to download the images and make a A/B swap in place, it's always hard to see smaller differences side by side.
What can be seen is that the main effect of the stronger gamut compression is darkening of certain high saturation colors, unless the color is clipped in raw, then there's not much darkening as the color is already clipped and there's nothing to recover. This can be seen in the out of focus green blob for example, it keeps the brightness while the yellow unclipped blobs are darkened.
The gamut compression also have a shadow lightening effect, which is most notable in the first example images with the deep dark blues, but if you look carefully you can see it in the sewing spools image especially on the purple threads.
One interesting effect I hadn't thought about is also seen in the purple spools -- the stronger gamut compression actually increases saturation in them slightly. The reason for this is that the HSV preprocessing step darkens/lightens the color so the follow-on CIECAM02 compression gets more room to operate on and needs to compress less. That is you get more of lightness compression than a chroma compression.
I'm pretty pleased myself with the result, but something tells me that it's not the last time I'll look into gamut compression algorithms.
-
Certainly not stupid question, I've also been through all THE INTERNET and it's a mess with all different compiler versions and distributions out there. I was actually surprised that it worked statically with MSYS2/MinGW64, it was not something I actively searched for, it just worked. My plan was to distribute the DLL together with the binary just as I do with OS X now, but it turned out to not be required as there's a libgomp.a with the MSYS2 distribution.
Makefile is the same as the distributed Linux makefile, but with the added "-static" option to LDFLAGS and added "-ljpeg -llzma -lz" on the libraries (DCAMPROF_LIBS), then built inside a mingw64_shell that comes with MSYS2 (if you install it using the package management pacman). I don't know exactly which packages I have installed as the machine used for compiling is used for other projects too, and MSYS2 was not installed for this.
btw static build for windows worked - I had only to mod the makefile from your distribution to account for the library names
$(CC) -o $@ $(LDFLAGS) $(DCAMPROF_OBJS) liblcms2.a libtiff.a libgomp.a -lm
why didn't I try before the static .a library that I had all the time in the tdm-gcc distribution just sitting near .dll beats me... well, that is what separates a practising software developer from somebody who was /at least tried to be/ once a long time ago ;D
-
I'll make a new release tomorrow (I hope) which contains new matrix optimization refinement feature; it will make it much easier than before to control tradeoffs in patch matching, which may be desirable especially for matrix-only profiles. As a LUT when relaxed closes in on the matrix it may be useful also for LUT profiles, for the perfectionist.
I've attached an example for the common CC24 target. First the normal matrix, where DCamProf without any refinement parameters and thus generates a matrix as before, trying to make total sum of errors as small as possible. As you can see the D02 patch has 0 error, as D02 is the most neutral patch in a CC24 and is assigned "D50 white", and as DCamProf's matrices are always "white point preserving" it will always have 0 error, that cannot be changed. DNG profiles require it by definition, and it's a good idea in any case so it can't be turned off.
Then I've added manual refinements using the new feature, to make the following adjustments: exact match on the light skin-tone patch, make sure that blues are not darkened for more stable behavior, and make sure colors are rather pulled away from purples than towards it (reds rather orange than purple, blues rather cyan than purple). It's specified like this:
dcamprof make-profile -L -v A02 0 -v C01 0,2,-1,1,-2,0 -v C03 -3,3,-3,3,0,3 -r dump1 cc24.ti3 test-profile.json
A02 0: exact match, DE 0.
C01 0,2 lightness range (up to 2DE lighter, but no darker), chroma range -1,1 and hue rather counter-clockwise (towards cyan) and absolutely not towards purple, same for red.
The tricky thing with matrices is that everything is interconnected, so if you make one patch match better, someone else must become worse. Quite easily you refine too hard so there's no solution possible, and then DCamProf will fail with an error, so it's a trial and error process. A quite simple one though.
-
I'll make a new release tomorrow (I hope) which contains new matrix optimization refinement feature; it will make it much easier than before to control tradeoffs in patch matching, which may be desirable especially for matrix-only profiles. As a LUT when relaxed closes in on the matrix it may be useful also for LUT profiles, for the perfectionist.
Excellent! Like you were reading my mind!
-
v1.0.4 is now released. The key new feature is better manual control of the matrix optimizer, especially useful for those making matrix-only profiles, but can also be useful to the perfectionist making LUT profiles as the matrix is the linear base the LUT is drawn to when relaxed, so it's always good to have the matrix as close as possible to the desired end result. As before -- for casual use the default automatic mode is good enough, so the manual controls are for advanced users.
News:
http://www.ludd.ltu.se/~torger/dcamprof.html#news
Downloads:
http://www.ludd.ltu.se/~torger/dcamprof.html#download
-
v1.0.4 is now released. The key new feature is better manual control of the matrix optimizer, especially useful for those making matrix-only profiles, but can also be useful to the perfectionist making LUT profiles as the matrix is the linear base the LUT is drawn to when relaxed, so it's always good to have the matrix as close as possible to the desired end result. As before -- for casual use the default automatic mode is good enough, so the manual controls are for advanced users.
News:
http://www.ludd.ltu.se/~torger/dcamprof.html#news
Downloads:
http://www.ludd.ltu.se/~torger/dcamprof.html#download
thank you and, this was, I believe, already asked - does it make any sense (I think it does, but) to allow a command line parameter that will make dcamprof to store the whole set of parameters (from that same command line + what was stored on the prev. step in case sequence like make-profile + make-dcp) inside some tags for make-profile (that shall be easy as it is your own json), make-dcp (may be in "ProfileCopyright" tag ?), make-icc ? so that later you can (if you wish) peek how did you create this profile of yours in the first place...
-
thank you and, this was, I believe, already asked - does it make any sense (I think it does, but) to allow a command line parameter that will make dcamprof to store the whole set of parameters (from that same command line + what was stored on the prev. step in case sequence like make-profile + make-dcp) inside some tags for make-profile (that shall be easy as it is your own json), make-dcp (may be in "ProfileCopyright" tag ?), make-icc ? so that later you can (if you wish) peek how did you create this profile of yours in the first place...
Yes that sounds like a good idea, I'll consider it.
-
v1.0.4 is now released. The key new feature is better manual control of the matrix optimizer, especially useful for those making matrix-only profiles, but can also be useful to the perfectionist making LUT profiles as the matrix is the linear base the LUT is drawn to when relaxed, so it's always good to have the matrix as close as possible to the desired end result. As before -- for casual use the default automatic mode is good enough, so the manual controls are for advanced users.
News:
http://www.ludd.ltu.se/~torger/dcamprof.html#news
Downloads:
http://www.ludd.ltu.se/~torger/dcamprof.html#download
Anders, can you shed some light on the following :
1) assume we have some SSF (ssf.json file) - it does not matter whether it is a for a real camera or we just invented some curves resembling some good or so-so data posted wherever at this moment
2) let is make a very simple synthetic (as we have SSF) target (with further pure matrix profile in mind, hence CC24 - we don't really need tons of patches for a pure matrix profile) : dcamprof make-target -c ssf.json -i spectrum -C -p cc24 ssf.ti3 , where spectrum will be D65, then D50, then StdA... see below why.
3) let us make dcamprof profile (with further pure matrix profile in mind, hence -L for example) : dcamprof make-profile -c ssf.json -i spectrum -C -B -L -r .\report ssf.ti3 profile.json
now the question - why in the world when I move from D65 to D50 to StdA I am getting the results significantly worse and worse with each "lower-K" spectrum used for forward matrix (as reported by dcamprof) ? is it how is should be (like because StdA shape is very deficient energy-wise in "blue" part of the spectrum, but why so noticeable difference between D50 and D65 ?) or is it some bug or is it because of a particular shape of SSF curves or am I doing something stupid here ?
-
let is make a very simple synthetic (as we have SSF) target (with further pure matrix profile in mind, hence CC24 - we don't really need tons of patches for a pure matrix profile)
From personal experience building pure matrix profiles from real camera SSFs, CC24 is not enough for a good pure matrix profile. I managed to generate a very good matrix profile with CC SG set plus some from Munsell set with slow matrix optimisation on (-s).
-
From personal experience building pure matrix profiles from real camera SSFs, CC24 is not enough for a good pure matrix profile.
OK, but the core question is why we are getting significantly worse results for forward matrix (as reported by DCamProf) when we are moving illumination from D65 to D50 to StdA with each step ?
I managed to generate a very good matrix profile with CC SG set plus some from Munsell set with slow matrix optimisation on (-s).
just because there were more patches or more patches with specific hue/sat that were driving DCamProf into certain direction while making 3x3 matrix transform ? I did not see any reasonable improvement with "-s" with CC24 spectral data embedded in DCamProf code... and how you will define "good" in terms of numbers (d*-whatever = average, mean, max) in your example ?
attached dcamprof report (left w/o "-s", right "-s") - I'd not call "-s" beneficial in this particular case
PS: replaced .tif with .jpg
-
and ssf.json used attached
-
just because there were more patches or more patches with specific hue/sat that were driving DCamProf into certain direction while making 3x3 matrix transform ? I did not see any reasonable improvement with "-s" with CC24 spectral data embedded in DCamProf code... and how you will define "good" in terms of numbers (d*-whatever = average, mean, max) in your example ?
I tried -s with CC24 as well - it generally had better results. In that particular case I had a target ideal matgrix profile to compare it to (which was built by someone else and fine tuned through substantial database of photos) - the one I built from CC SG matched it almost perfectly (if comparing gamut). Every single one I built from CC24 patches alone had problems in violets/blues at least (I have not investigated all other differences in detail).
-
In that particular case I had a target ideal matrix profile to compare it to (which was built by someone else and fine tuned through substantial database of photos)
one can assume that you were trying to match IB's matrix profile in RPP for your kodak back then (why 'd otherwise you invest time & effort in that exercise ?)... but then one can assume that IB did profile himself from (some copy of) CCSG target raws from a similar back and you built monochromator to measure CFA SSFs and then you reasonably shall get close to IB's matrix profile if you are using similar target data but with measured SSFs, no ?
-
one can assume that you were trying to match IB's matrix profile in RPP for your kodak back then (why 'd otherwise you invest time & effort in that exercise ?)... but then one can assume that IB did profile himself from (some copy of) CCSG target raws from a similar back and you built monochromator to measure CFA SSFs and then you reasonably shall get close to IB's matrix profile if you are using similar target data but with measured SSFs, no ?
Not in this particular case - the profile was for Kodak SLR/n.
To answer "why" - to see how easily I can build good quality profiles from SSFs. Iliah's profile in that case was built from CC24 + quite a few iterations rebuilding it, colour correcting the set of images (from what I understood).
I did a few other builds for different SSFs I took - with pre-canned Kodak profiles, and without comparing it to any profiles but testing the outcome on a set of photos and was getting reasonably close with the patchsets substantially larger than CC24. Have not built satisfactory profiles from CC24 yet. I only use matrix for camera profiles (not really interested in LUTs) hence my interest in getting the best matrix I can from the SSFs.
-
Did anyone experiment with virtual targets other than CC24 for LUT profiles?
-
That StdA results get worse than D50 is expected. I don't have all reasons for that fully analyzed but I think one part is that camera SSFs are less suited to separate colors with low temperature illuminants, but I suspect that simply because the spectrum is so slanted colors get harder to match in general.
That D50 is worse than D65 I find surprising, but I need to see how much worse worse is. If it's 0.5 DE here and there I'd say it's about that particular cameras SSFs, if it's huge differences on specific colors I'd suspect some sort of error in the process or bug. I'm travelling for the moment so I don't have that good ability to do testing on my own.
Anders, can you shed some light on the following :
1) assume we have some SSF (ssf.json file) - it does not matter whether it is a for a real camera or we just invented some curves resembling some good or so-so data posted wherever at this moment
2) let is make a very simple synthetic (as we have SSF) target (with further pure matrix profile in mind, hence CC24 - we don't really need tons of patches for a pure matrix profile) : dcamprof make-target -c ssf.json -i spectrum -C -p cc24 ssf.ti3 , where spectrum will be D65, then D50, then StdA... see below why.
3) let us make dcamprof profile (with further pure matrix profile in mind, hence -L for example) : dcamprof make-profile -c ssf.json -i spectrum -C -B -L -r .\report ssf.ti3 profile.json
now the question - why in the world when I move from D65 to D50 to StdA I am getting the results significantly worse and worse with each "lower-K" spectrum used for forward matrix (as reported by dcamprof) ? is it how is should be (like because StdA shape is very deficient energy-wise in "blue" part of the spectrum, but why so noticeable difference between D50 and D65 ?) or is it some bug or is it because of a particular shape of SSF curves or am I doing something stupid here ?
-
I tried -s with CC24 as well - it generally had better results. In that particular case I had a target ideal matgrix profile to compare it to (which was built by someone else and fine tuned through substantial database of photos) - the one I built from CC SG matched it almost perfectly (if comparing gamut). Every single one I built from CC24 patches alone had problems in violets/blues at least (I have not investigated all other differences in detail).
"-s" is makes a brute force optimizer run, it's intended as a sanity checker or debug if one suspects that the normal matrix optimizer does some mistake. As it's brute force it searches the whole space and is thus extremely slow. I have yet not come across any case when the normal optimizer does any big mistake, but the brute force finder will likely reach at a slightly better match, mathematically speaking, but personally I don't think the difference is worth the wait. After the brute force run the normal optimizer is still used for refinement steps, where white point preservation is applied. Refinement means only small changes so it isn't any problem.
Matrix profiles often have problems with violet/blues due to clipping, limiting matrix negative factors with the -y parameter can help that. Possibly by using CCSG you get a similar balancing effect thanks to higher saturation colors, I would expect a bit worse matching of lower saturation colors though.
-
Did anyone experiment with virtual targets other than CC24 for LUT profiles?
Yes I've worked with many target types and spectral databases, but it is a while ago as the recent features I've worked with haven't been related to SSF.
In general my view on targets is that for reproduction photography it's probably great to have special-made targets that have patches from the artwork you should copy, but when it comes to making a profile for general-purpose photography I see little need to have large special targets. If you have SSF and a rich spectral database experimenting is free though, there's no harm.
General-purpose photography has so large variability in illuminants, cameras are relatively limited in matching capability, and there's always a need to consider smoothness (that is not over-correct), so larger targets isn't a recipe for "better" profiles. With a LUT profile much of the look sits in the curve and tone operator, gamut compression, and subjective look operators if any.
When I've got feedback about hue "errors", there's almost always been huge errors (5+ DE), and it's not really about some target mismatch or something, but that the person giving feedback has a subjective idea what the color should be, which is not matching the colorimetric fact. So it's not been a target issue. DCamProf supports subjective adjustments too, but it makes it much more difficult to make a profile.
There's an exception range, and that's extreme saturation colors, where cameras get difficulties matching and all sort of technical issues happen in the color pipeline. If one has matching needs there one may need to use special targets and do hand-tuning etc.
This brief target simulation testing can shed some light on what large tartgets contribute:
http://www.ludd.ltu.se/~torger/photography/camera-profiling.html#target_eval
-
camera SSFs are less suited to separate colors with low temperature illuminants
interesting to see what SSFs imitating something like some CIE 1931/2 Observer will yield in this case
-
I've now come far enough in this project to know that it's not going to be vaporware so I thought I could tell about it, although I can't mention a release date. Hopefully Q1 2017, depending how swamped I'll be with other projects.
The project is "Lumariver Digital Camera Profiler", which basically is DCamProf with a GUI. It will be a commercial product, but won't affect DCamProf command line tool which will continue to be free and open-source. With a GUI I hope that DCamProf can reach a broader audience. It's designed to be simple for the basic use, just place the grid on the image and press "Render" and you have a profile ready to use, just as easy as any other profile maker. However there's also lots of tuning possibilities for the advanced user.
Developed with QT it will run on Mac and Windows, and Linux too if anybody except me wants it. The first version will be DNG Profile and CC24 only. If interest proves to be good, I'll probably follow up with a version that supports ICC and custom targets. Development time is substantially shortened with those two limitations though. Originally I didn't intend to make a GUI for look operators, but there will be that too. You can't make all things you can in the DCamProf JSON text format, but most of it.
Here's a feature list (may change!)
- Profile maker for single or dual-illuminant DNG profiles using the CC24 target
- Well-picked defaults to make it easy for casual use
- Built-in reference spectra, or load your own for target and illuminant
- Glare compensation, flatfield correction
- Visual manual tuning of reference colors
- Visual manual tuning of matrix optimization
- Visual manual tuning of LUT optimization
- Visual manual tuning of tone curve
- Choose tone reproduction operator, gamut compression, base look (built-in or load DCamProf look format)
- Visual look adjustment editor, like a "color editor on steroids"
- Own project file format to store your profile projects, export to DNG profile as needed
- Profile comparison mode to make A/B testing on images of your choice
-
If that one produces anything I can use with C1, I’m your customer.
-
If that one produces anything I can use with C1, I’m your customer.
It probably won't be C1 in the first release, as C1 requires a special form of ICC support, a separate rendering pipeline etc. However DCamProf command line tool can do C1 so it's not unlikely that it will appear later, just like custom target support and maybe some SSF features. First release's first though, much is done, but still much work to do.
-
It will be a commercial product
how much $ ?
-
Hi Anders,
You have a buyer here!
/Erik
I've now come far enough in this project to know that it's not going to be vaporware so I thought I could tell about it, although I can't mention a release date. Hopefully Q1 2017, depending how swamped I'll be with other projects.
The project is "Lumariver Digital Camera Profiler", which basically is DCamProf with a GUI. It will be a commercial product, but won't affect DCamProf command line tool which will continue to be free and open-source. With a GUI I hope that DCamProf can reach a broader audience. It's designed to be simple for the basic use, just place the grid on the image and press "Render" and you have a profile ready to use, just as easy as any other profile maker. However there's also lots of tuning possibilities for the advanced user.
Developed with QT it will run on Mac and Windows, and Linux too if anybody except me wants it. The first version will be DNG Profile and CC24 only. If interest proves to be good, I'll probably follow up with a version that supports ICC and custom targets. Development time is substantially shortened with those two limitations though. Originally I didn't intend to make a GUI for look operators, but there will be that too. You can't make all things you can in the DCamProf JSON text format, but most of it.
Here's a feature list (may change!)
- Profile maker for single or dual-illuminant DNG profiles using the CC24 target
- Well-picked defaults to make it easy for casual use
- Built-in reference spectra, or load your own for target and illuminant
- Glare compensation, flatfield correction
- Visual manual tuning of reference colors
- Visual manual tuning of matrix optimization
- Visual manual tuning of LUT optimization
- Visual manual tuning of tone curve
- Choose tone reproduction operator, gamut compression, base look (built-in or load DCamProf look format)
- Visual look adjustment editor, like a "color editor on steroids"
- Own project file format to store your profile projects, export to DNG profile as needed
- Profile comparison mode to make A/B testing on images of your choice
-
how much $ ?
More than Adobe DNG Profile Editor, less than BasICColor Input ;-). Haven't really decided on price yet, but I want to make it reasonable.
-
Anders, this looks very promising.
A 1-click profile generator with well-picked defaults would be a very good start.
The technical manual tuning would be nice to have, but to make this more than a single-use tool (once you have built a profile for your camera you never use it again) it would be good to make it something that builds a community around it -- eg, people can contribute look operators -- and that doesn't require a deep understanding of illuminants and gamut compression.
The Adobe DNG editor has some useful features, but you can't see what difference your tweaks make and so the workflow with Lightroom / ACR is cumbersome. Finding a clever visual way of making comparisons between profiles or individual adjustments is the key to success. For example, being able to tune the DCampProf Natural or Natural+ profile generated profile (relative to the Adobe Standard, say) would be a good start. (So being able to pick up the profiles installed in your system without copying them and then using them to tune your own profiles would be a good start. Put another way, concentrate on easy workflow, rather than raw capability would be good.
My 1/2p. Happy to offer to B test at an appropriate time.
-
More than Adobe DNG Profile Editor, less than BasICColor Input ;-). Haven't really decided on price yet, but I want to make it reasonable.
consider to make a deal libraw people to bundle that with rawdigger profile edition ...
-
This sounds great! Very exciting, Anders. I like the look of the feature list, especially the notion of using my own readings of my CC Passport target. Good luck with getting it up & available soon.
And it's good to hear you may not need to charge as much as BasICColor. ;-)
John
-
As a business case, this type of niche software is not so good. It's high complexity in development (requires lot of time to do), and even if you sell well the volumes is not going to be huge. That I make the GUI a commercial product is not really to make it a bread-winner for my company, but to allow me to put in a few extra daytime hours and not just spare time. I like to work, but I no longer have the capacity I had when I was 25.
I think BasICColor has about the only niche that makes sense commercially -- professional reproduction photography. For a pro photographer €500 for an efficient workflow tool is not unreasonable (just think about the cost of all hardware you need, and salaries), and for reproduction you really need it. It's low volume market, so the high price is necessary, and software engineering in Germany is not cheap. I don't think it's particularly good at making general-purpose profiles though, which instead is the focus of my product.
There's a reason Adobe doesn't care to sell their profile editor, that X-Rite's profile maker is not actively developed, and that QPCard, which is actually a Swedish company like my own, from the public numbers does not seem profitable enough to even finance one software developer. Camera profile making is not a great niche if you want to make a bunch of money. It is a great niche if you want an enjoyable software development challenge though :-)
Knowing it won't be a great financial success gives me some freedom though, I can make the software the way I like it personally and not put effort into making it sell as much as possible. In other words I won't make the software in a way that I personally don't like even if I think it's a good idea commercially. BasICColor has done the right thing(tm) in terms of design for selling as much as possible -- their software has almost no settings, and that's what the general user wants, something than looks clean and easy and something that just works. While my software will do great profiles with "one click" too, there will still be massive amount of settings and it will look quite technical and that will scare some people away regardless of how simple it will be to make a profile with default settings.
-
Thanks, you have some good ideas. It's not going to be a raw converter, I've excluded the capability to open all kinds of raw formats as it's a mess to maintain, and if you're a ACR user you need to make a DNG anyways to make sure conversion becomes the same as ACR. If/when ICC support is added I only need to add TIFF support as the ICC workflows go via that format rather than raw directly.
My vision is that it should be a camera profile designer that has good enough capabilities to make truly excellent general-purpose camera profiles -- with subjective tunings as there's no agreement of what the ultimate color is. In other words the goal is to break the monopoly the camera manufacturers and raw converter makers have today on making great general-purpose profiles. I think it's unfortunate that people choose camera based on what profile that comes built-in into the default workflow, but that is the situation when there is no good camera profile design tool available to the public.
To the average user I think it's hard to not be a "single use" program as you say, but that's not too bad I think. Perhaps commercially it would be better for regular use, but if the software does what most need in a few clicks I'm happy. To the advanced user I think it will be a "periodic use" software, that is you don't really use it regularly but at times you put some effort into getting a profile juuust right for your camera and your taste, and perhaps you go back to re-adjust a profile you made before, or use the comparison mode to check out some other profiles.
Profile makers often make up the idea that you should have a colorchecker in each of your scenes and then make scene-specific profiles, which gives you regular use. That may make sense for reproduction photography, but I don't think it makes much sense for general-purpose photography, and I can't really promote a use that I don't believe in, so single use to periodic use it is... :-)
Anders, this looks very promising.
A 1-click profile generator with well-picked defaults would be a very good start.
The technical manual tuning would be nice to have, but to make this more than a single-use tool (once you have built a profile for your camera you never use it again) it would be good to make it something that builds a community around it -- eg, people can contribute look operators -- and that doesn't require a deep understanding of illuminants and gamut compression.
The Adobe DNG editor has some useful features, but you can't see what difference your tweaks make and so the workflow with Lightroom / ACR is cumbersome. Finding a clever visual way of making comparisons between profiles or individual adjustments is the key to success. For example, being able to tune the DCampProf Natural or Natural+ profile generated profile (relative to the Adobe Standard, say) would be a good start. (So being able to pick up the profiles installed in your system without copying them and then using them to tune your own profiles would be a good start. Put another way, concentrate on easy workflow, rather than raw capability would be good.
My 1/2p. Happy to offer to B test at an appropriate time.
-
My vision is that it should be a camera profile designer
but one can load existing dcp or icc profile and tune it from there, using whatever picture you one wants to use (to select colors with some color picker and watch the changes how they affect picture - something like C1 color editor that can then save your changes as a new "camera profile"), no ?
-
BasICColor has done the right thing(tm) in terms of design for selling as much as possible -- their software has almost no settings, and that's what the general user wants, something than looks clean and easy and something that just works. While my software will do great profiles with "one click" too, there will still be massive amount of settings and it will look quite technical and that will scare some people away regardless of how simple it will be to make a profile with default settings.
a lot GUI frontends for command line tools that actually do a decent jobs - DisplayCAL is a prime example (or smallish makeinputicc for camera profiles)
-
but one can load existing dcp or icc profile and tune it from there, using whatever picture you one wants to use (to select colors with some color picker and watch the changes how they affect picture - something like C1 color editor that can then save your changes as a new "camera profile"), no ?
No, like DCamProf it has no profile import, it can only make and edit its own profiles, which then is *exported* to a DCP.
DCamProf starts with calculating a colorimetric base, and then adds a tone reproduction operator including gamut compression on top, and then one adds look adjustments on top of that. If you import a finished ICC or DCP you just have the end result, you can't split it into those mentioned parts, so it doesn't fit into the profile design workflow.
It wouldn't be too hard though to just make an edit tool just for that specific purpose, but it's not something I plan for the first release and it's not something I would use myself so people would have to nag about it :-). Also, Adobe's DNG Profile Editor already has this for DCPs, and C1 has it for ICCs. If your profile was designed by my software from the start it makes much more sense to edit that project than adjusting the resulting DCP, but if you want to I suppose you can by using Adobe's DNG Profile Editor.
It's not *unthinkable* that I would grow the software into becoming a sort of RawDigger for ICC and DNG profiles, eg analyzing and doing stuff with existing profiles made by others, but the primary focus now is to make a profile design software where you built one from the ground and up.
-
Also, Adobe's DNG Profile Editor already has this for DCPs, and C1 has it for ICCs.
yes and QPCard software has plugin for example, but... you can do it better !
PS: and not only that - the value of the tool is when the author is available (to get somebody from Adobe or P1 is difficult)
-
RawDigger for ICC and DNG profiles
(http://www.gotvectors.com/images/preview/thumb/stock-icon-hungry-emoticon-vector.jpg)
-
Looking forward to it Torger. Imho there is a lot of value in
1) eliminating the back and forth to Argyll and its finicky handling of cc24 photos
2) easily help develop and fine tune one's own 'look' in real time.
Jack
-
eliminating the back and forth to Argyll and its finicky handling of cc24 photos
you can use rawdigger instead
-
Congrats, Anders, on the new front end GUI design. You've got to be the hardest working programmer I've ever seen.
Something I've been wanting to ask is how do edits applied with sliders and curve adjusts in several different Raw converters using your profiles act on the preview?
The profiles do a very good job of making a properly and similarly exposed image based on the CC24 target look correct, needing none or very minor edits, but out in the field at least in my case I have to underexpose to preserve highlights in a lot of high contrast scenes where with some I have to severely move to the left the contrast slider in ACR/LR and/or increase Shadow or Fill slider and apply notch curves with the point curve.
Have you seen any artifacts in the previews as a result from applying extreme edits using your profiles?
-
Looking forward to it Torger. Imho there is a lot of value in
1) eliminating the back and forth to Argyll and its finicky handling of cc24 photos
2) easily help develop and fine tune one's own 'look' in real time.
Jack
Yes this will provide that, although "real-time" is more like semi-realtime. DCamProf's algorithms are really suited for realtime processing, and while optimizing them heavily could make it considerably better it would be a massive amount of work and one would still not reach the speed of say LR sliders. The thing with raw converters is that they in many ways are optimized for speed rather than using the perceptually best possible color spaces, while DCamProf of course is the opposite. I'm very impressed with the speed of LR by the way, it's hard to get a raw software that responsive.
I've done some optimization from the command line tool, but it will still be a bit slow. It will be like ~1 second for refresh on the look adjustments on a decent computer, and for other edits it can be multiple seconds and you manually press a button to render after a change. I've adapted the GUI and workflow to be able to work as efficiently as possible despite having limited realtime capability. For example after you made a change and new render, you can peek on the previous result with a quick A/B swap, to see if you want to revert back and re-adjust. The look adjustments and the curve edit is what can be made with automatic "realtime" update. I think it's ok way to work, and it wouldn't make sense to compromise the algorithms just to get speed, for that you have a raw converter. We'll see what people think about it :-).
-
1) eliminating the back and forth to Argyll and its finicky handling of cc24 photos
Are you just referring to the fact that it's a command line process, or
something else ? (i.e. , what precisely do you find "finicky").
Thanks.
-
Are you just referring to the fact that it's a command line process, or
something else ? (i.e. , what precisely do you find "finicky").
Hi Graeme, I did not mean to appear to be disparaging your otherwise excellent collection, which in fact I find extremely useful and professional. Command line does not phase me (in fact I consider it a feature). My comment refers to my (limited) experience getting scanin to properly find and read the patches in a ColorChecker Passport with the following command:
scanin -v -p -dipn input.tif recogin.cht valin.cie
I am pretty sure I have the correct recogin and valin files. input.tif is in linear gamma and by choice, my workflow does not include pre-rotation or downsampling. It is however always taken in landscape orientation and not too far from the correct angle. A typical image is shown below, after final rendering (not linear gamma). There was no cropping that would make it work. The only way I was able to get proper results was to open the raw file in RawDigger, get the coordinates of the four corners of the cc24 target and feed them to scanin via the -F switch.
(http://i.imgur.com/VCCb3GC.jpg)
Any suggestions on how to setup scanin to work automatically in such a situation, without downscaling and pre-rotating the tif?
Thanks for your help,
Jack
-
A common usage error with Argyll is assuming that the ColorChecker Passport has the same layout as the larger CC24 target. The patches are the same, but the shape of them is slightly different. This means that it becomes hard to match the target if the CC24 layout file is used (ColorChecker.cht), if the image is perfectly straight on it usually works anyway, but otherwise one needs to Passport layout file (ColorCheckerPassport.cht). Note that it requires that you shoot the whole target, that is also the second page.
With a properly matched cht file Argyll is quite good at auto-matching, and using the -p parameter it can correct for some perspective distortion too.
I have myself been guilty of using the ColorChecker.cht file for the colorchecker passport target and thought that Argyll was a bit bad at matching, but once I started using the correct file it became much better.
-
Hi Jack,
torger has correctly diagnosed the problem - the ColorChecker recognition layout is not identical to that of the Passport. I have hacked together a half Passport .cht file that may work better if only the ColorChecker 24 side of the chart has been shot, and it is here (http://www.argyllcms.com/ColorCheckerHalfPassport.cht).
-
Hi Anders and Graeme,
I believe that I have been using the correct Passport .cht file. Things that I thought might be confusing scanin (based on the relative doc page):
1) non-target related detail in the image;
2) the fact that I normally like to take a slightly out of focus image of the target;
3) the target is made up of too many pixels;
4) the image is in linear gamma;
Would using the -G switch help? I'll try using the half Passport .cht file. Thanks for your help.
Jack
-
I believe that I have been using the correct Passport .cht file.
I don't think that's possible, since the full Passport .cht file won't work on half of it
-
I don't think that's possible, since the full Passport .cht file won't work on half of it
Ah, I did not realize that it expects the full Passport to be visible. When I have a little time I will try your half-passport.cht.
Jack
-
I've now come far enough in this project to know that it's not going to be vaporware so I thought I could tell about it, although I can't mention a release date. Hopefully Q1 2017, depending how swamped I'll be with other projects
Great news! Although, I can handle dcamprof, I will buy it. Just to donate your work on dcamprof itself. :)
-
Great news! Although, I can handle dcamprof, I will buy it. Just to donate your work on dcamprof itself. :)
Me too ;)
-
Although, I can handle dcamprof, I will buy it.
Well, being 'command-line challenged', I can't handle dcamprof as it is right now, so am eagerly awaiting the opportunity to buy your version with the GUI! ;D
John
-
Apologies if this was discussed already but would making my own profiles allow me to match two cameras in regard to colors?
I use a Canon 6D and 7D2 in Capture One and the colors from each don't look quite the same. I see that it's possible to make C1 camera profiles now so I wonder if it's worth trying out.
-
Apologies if this was discussed already but would making my own profiles allow me to match two cameras in regard to colors?
I use a Canon 6D and 7D2 in Capture One and the colors from each don't look quite the same. I see that it's possible to make C1 camera profiles now so I wonder if it's worth trying out.
in general a noticeable (for hoi polloi eyes) color difference is either because of sufficiently different CFA or sufficiently different profiles... in C1 you can select 7D2 profiles for 6D raws and vice versa... if colors are way different after you try that then naturally the reason is quite different CFAs... then to try to come up with the more or less the same color (for your eyes) from them you really need LUT profiles and then from practical stand point you want either a real target with many patches (which I bet you don't have) or deal with synthetic targets... granted you can always live with whatever results you receive, they still might be a better match than canned profiles from C1 for your subjective taste - may be you really only need to match greens ? or reds ? or blues and only approximately... may be C1 color editor will be the better option then
-
Apologies if this was discussed already but would making my own profiles allow me to match two cameras in regard to colors?
I use a Canon 6D and 7D2 in Capture One and the colors from each don't look quite the same. I see that it's possible to make C1 camera profiles now so I wonder if it's worth trying out.
Yes two cameras should look quite close indeed if you make two profiles from identical targets in identical lighting situations. I doubt that the color filters differ that much between 6D and 7D2, the most probably differ some, but the main design principles are probably the same as it's both fairly recent Canon cameras.
That they do not look the same in in C1 could be that Phase One have had different design goals for the profiles. It's quite common that raw converter makers make cameras look more different than they need to be. I don't really know why, if it's just a side-effect of their workflow, or if they have some marketing reason to differentiate cameras. Sure their medium format looks are unique for an obvious reason, but I don't see why one would want to differentiate two Canon cameras.
Even if the SSFs (ie color filters) are quite drastically different cameras should still look very much alike, at least in normal light conditions for normal color saturations. Most modern cameras can match a target like a CC24 quite accurately with just a matrix (that was not the case for say 10-15 years ago) so a LUT on top doesn't need to make much stretching job to make them match. One of the key arguments why I am myself always using my own profiles is because I want my cameras, even from different manufacturers, to produce the same color and a neutral color. Simply so I get a well-defined starting point in my regular post-processing.
If you make profiles for the cameras and do quick A/B swaps on the same colorful shot you will still quite easily detect some differences, especially in how light colors are rendered, but hue and saturation should match well, which means that if you see the images a bit apart they will be very hard to differ.
-
I just made a hotfix 1.0.5 release as I became aware that the exclude patch and glare parameters for the make-profile command hasn't had any effect since 1.0.1. Probably not many as me have noticed earlier since if you use a split workflow with make-target command as I describe in the tutorial you're not affected.
There will be a bit larger update later on, but I'm only doing critical fixes now on DCamProf until the commercial "Lumariver Profile Designer" is out. Talking about that there will be some more time until I can release that as I've expanded the feature set to include ICC profiling and more, so I'm currently guessing it will slip out of Q1 into early Q2, very much dependent on pressure from other simultaneous projects.
-
I just made a hotfix 1.0.5 release as I became aware that the exclude patch and glare parameters for the make-profile command hasn't had any effect since 1.0.1. Probably not many as me have noticed earlier since if you use a split workflow with make-target command as I describe in the tutorial you're not affected.
There will be a bit larger update later on, but I'm only doing critical fixes now on DCamProf until the commercial "Lumariver Profile Designer" is out. Talking about that there will be some more time until I can release that as I've expanded the feature set to include ICC profiling and more, so I'm currently guessing it will slip out of Q1 into early Q2, very much dependent on pressure from other simultaneous projects.
Hi Anders,
As a Capture One user, now you've got my attention ... ;)
Cheers,
Bart
-
As a Capture One user, now you've got my attention ... ;)
Yep there will be C1 support, which is the most hellish of all of course ;-), due to their own idea of how an ICC profile should be applied. There's no less than three curves involved. There's their close-to-but-not-quite 1.8 encoding gamma, their user-selectable in-software curve, with or without white-level adjustments, and a small modifier curve embedded into the ICC profile's Lab LUT. It was hard to not bloat the GUI just to support C1's own idea of ICC pipeline (there are other ICC-using converters too, don't want it to make a C1-specific software), but I think it came out quite okay.
The software will still for sure require a manual though, as it won't be self-evident unless you know a little about how the pipeline works. The DNG workflow will be a bit more straightforward, but it also have its own quirks -- that DNG LUTs can't scale the neutral axis may lead to some confusion and I'll try to minimize that if I can. The nice thing with ICC is that it's really a plain LUT, 3 values in 3 values out, no fuzz, while DNG LUTs have their special constraints that means that there's some things you can't do.
I'm currently looking into including a 3D correction LUT for reproduction and scanner use cases, and DNG is giving me a headache there as you can't correct the neutral axis. By skipping the HSM lut and putting it all into the looktable I can use the tonecurve to at least linearize the neutrals, which probably will be the way to go for the reproduction/scanner use case for DNG profile users. Not sure I will include a 3D LUT for the first release though. The main focus is still general-purpose profiles and there 3D correction LUTs doesn't make sense as the exposure is variable, then the current 2.5D is the way to go.
-
Hi!
I have successfully built ICC profiles with DCamProf. Now I wanted to try the combo profile based on the Imaging Resource target of a CC24 and a CCSG. But while
./dcamprof-v0.10.4 make-profile -n HB_a7r2_CC24 -g -i D50 -B -S ImaRes_CC24.ti3 HB_a7r2_CC24.json
results in the .json profile,
./dcamprof-v0.10.4 make-profile -n HB_a7r2_combo -g -i D50 -B -S combo.ti3 HB_a7r2_combo.json
only displays the manpage. What am I missing? (The .ti3 is successfully created).
Thank you for your help - Hening.
-
Hi!
I have successfully built ICC profiles with DCamProf. Now I wanted to try the combo profile based on the Imaging Resource target of a CC24 and a CCSG. But while
./dcamprof-v0.10.4 make-profile -n HB_a7r2_CC24 -g -i D50 -B -S ImaRes_CC24.ti3 HB_a7r2_CC24.json
results in the .json profile,
./dcamprof-v0.10.4 make-profile -n HB_a7r2_combo -g -i D50 -B -S combo.ti3 HB_a7r2_combo.json
only displays the manpage. What am I missing? (The .ti3 is successfully created).
Thank you for your help - Hening.
why the old version and then post the .ti3 file just in case
-
Hi scyth,
why the old version? Well it worked last year, and I have so far seen no need to update. Anyway, here is the .ti3 file if you care to have a look at it.
-
Hi scyth,
why the old version? Well it worked last year, and I have so far seen no need to update. Anyway, here is the .ti3 file if you care to have a look at it.
you are using -g for make-profile :
-g <target-layout.json> provide target layout for glare matching and/or flatfield correction.
where is the json file for it ?
-
Hi!
I have successfully built ICC profiles with DCamProf. Now I wanted to try the combo profile based on the Imaging Resource target of a CC24 and a CCSG. But while
./dcamprof-v0.10.4 make-profile -n HB_a7r2_CC24 -g -i D50 -B -S ImaRes_CC24.ti3 HB_a7r2_CC24.json
results in the .json profile,
./dcamprof-v0.10.4 make-profile -n HB_a7r2_combo -g -i D50 -B -S combo.ti3 HB_a7r2_combo.json
only displays the manpage. What am I missing? (The .ti3 is successfully created).
Thank you for your help - Hening.
There should be a layout description file after the -g parameter, unless it was different back at that version, I don't remember. You can skip the parameter though, for the combo target it won't work anyway. I'd recommend to get the latest version, I have binaries directly on the web nowadays. It's easier to provide help if you use the latest version as I quickly forget what was different with the older ones :-).
When it shows the manpage, if you scroll all the way back to the top you usually get an error message of what went wrong. I should probably split the manpage per command as it's becoming veeeery long so noone sees the error message at the top...
-
Many thanks to the both of you! I have now downloaded version 1.0.4, and I see there are new things I have to study before I go on.
Good light! - Hening
-
Hi again!
I have now done a little reading and have a few questions:
1-In the section 'DCamProf's neutral tone reproduction operator', you (Anders) write:
'However, most raw converters are designed for profiles that have a fixed curve applied.'
How does RawTherapee do in this regard?
In the command reference to make-profile:
2- -p, -f, -e, -m, pre-generated matrices if you want to skip the matrix finder steps.
I don't understand this. What are the matrix finder steps, and what are these flags?
3-
-t <linear | none | acr | custom.json>, embed a tone-curve in the output DCP or ICC, and apply the default tone reproduction operator.
What is the difference between 'linear' and 'none'? I assume/hope, that one of them means 'Apply the neutral tone reprodution operator, but no other curve'; and that the other means 'keep the profile entirely linear'. But which is which? (I want to try both).
Thank you for your help!
-
RawTherapee has the "perceptual tone curve" which is a simplified realtime version of DCamProf's neutral tone operator, so using that you could provide a profile with only colorimetric correction and no curve (linear curve). Most new bundled RawTherapee profiles are generated with DCamProf and have a curve embedded though.
The pre-generated matrices flags can be used if you want to render matrices in one run and the LUT in another. This is not a common workflow, and there are generally two reasons for it. 1) You're testing lots of LUT tuning flags over and over again in serveral runs and don't want to wait for matrix optimizer for each run, 2) you're using different target for the matrix and the LUT, usually a smaller target for the matrix. The rationale being that super-saturated colors found in larger targets may be interesting to correct with the LUT, but is more disturbance than gain when it comes to the linear matrix optimization -- cameras are designed to linearly match "normal range" colors well, so it's better to use normal range colors when making the the matrix. That said the matrix optimizer usually comes to about the same conclusion also when using the larger target. So for me it's most often the first reason, to save time when tuning LUT parameters.
Oh, if you haven't the difference between matrix and LUT -- matrix is the base 100% linear correction with 100% perfect gradients. On top of that you can make non-linear corrections to better match colors using the LUT, short for Lookup Table. DCamProf's native LUT is not a traditional LUT but a mathematical model with anchors, which is sampled into a traditional LUT when you make an ICC or a DCP.
The difference between "linear" and "none" is only applicable to DCP (in the ICC docs it should say only "none"). The resulting matrix/LUT will be the same with linear and none, that is adapted for no curve (linear curve), but if "none" is chosen the curve element is left out from the DCP, otherwise a linear curve is put there. When the DCP lacks a curve raw converters have different ways of handling it. ACR/Lightroom will apply the standard ACR curve, while others may treat it as a linear curve. Thus I recommend to provide "linear" so it's clearly defined what you want.
-
Hi Anders,
thank you for your fast reply.
Sorry I have to ask some follow-up questions.
1-Concerning your first paragraph:
Does this mean, if I convert with RT using a linear matrix or LUT profile, RT will under the table use a simplified version of your ntro?
(I'm aware of that a LUT profile by definition can not be strictly linear; but 'with no tone curve')
(How) can I achieve that RT doesn't add anything on it's own? I mean: that it puts out a linear TIF, if I choose 'linear'; and a TIF with your ntro (full version) but no other curve, if I choose that?
2-Concerning the last paragraph in your answer:
Now I understand the difference between 'none' and 'linear'; but I still don't understand what I have to enter if I want 'the ntro, but no other curve'. (The command reference says 'Embed a tone curve [...] AND apply the default tone reproduction operator'.)
Or is it like this:
-t linear : will embed ntro, but no other curve.
no -t flag in the command: no ntro, no other curve.
?
BTW I use neither ACR nor Lightroom. I have hitherto used Iridient, but want to move to RT now.
And a new question:
3-
"-I <target XYZ reference values illuminant>, can be specified in the same way as the calibration illuminant (-i). If spectral information is provided in the target the XYZ values will be re-generated according to chosen illuminant (and observer) when possible, and then this parameter is thus ignored. If there is no spectral information it’s however important that the illuminant and observer matches what was used for the target."
I read this as "the illuminant, for which the XYZ reference values for the target were calculated".
Which is that? How can I know? maybe D50, but I want to be sure.
The ImaRes target for the Sony a7r2 was obviously shot with flash. So for the -i flag, I think I have to enter exif tag 4.
I want to make ICC and single-illuminant dcp profiles.
What should I enter for the -I flag?
-
I have hitherto used Iridient
AFAIK iridient can use both "icc/icm" and "dcp" profiles
-
Yes it can. This is not the reason that I want to change.
Good light!
-
1) RT doesn't add anything automatically, but like all raw converters it allows you to add contrast via a curve, but unlike most raw converters you can choose which tone operator to use, "Standard = RGB", "Film-like = ACR", "Perceptual = similar to DCamProf", and a couple of more. With RT you can choose which elements of the DCP to activate, for example you may skip activating the looktable and curve, which for a typical DCamProf profile would leave only the colorimetric aspect. Anyway you don't need to worry, per default RT behaves as you would expect, it's just that you have so much more control, a bit too much control to be that software that everyone wants to use, but it's nice when you're a control freak ;)
2) Uhmm... there's been some changes back and forth on when the NTRO gets activated or not, and I'm really focused on the GUI version so I'm not really sure without testing what happens with a particular parameter combination in the latest version. What should happen though is that as soon as you have a parameter that requires the NTRO to be activated it will. For example if you have linear curve and no gamut compression there will be no NTRO as it would make no difference. If you activate gamut compression, the NTRO will be activate even if you have linear curve as it depends on that, but really the basis of using a tone reproduction operator is that there is a curve other than linear. If you use a linear curve I've assumed that the user is making a profile for reproduction, and then you just want a colorimetric profile.
I'm not too familiar with Iridient so I don't know what it does if the profile lacks an embedded curve, but it would be easy to find out by testing, or just ask Brian.
3) The best is if your target reference file contains spectra, then XYZ values can be recalculated from scratch for any illuminant you choose. If you measure your own targets with a spectrometer you should be able to make one with spectra. If you're using a file provided with DCamProf they all have spectra. If you've got the reference file from the manufacturer it's likely not containing any spectra, and then you need to know what illuminant that was used when the XYZ values in the file was calculated. It's nearly always D50 or D65 with 2 degree 1931 standard observer. Unfortunately there is no standard field to provide information of which illuminant that was actually used, but often it's stated in the description field or some non-standard field, so look at the text file and search for D50 or D65 and see what you find. For example you can find something like this:
[DESCRIPTOR "L* a* b* Batch average data (light D50, viewing angle 2)"] and then you see that D50 with the standard 2 degree 1931 observer has been used.
Exif tag 4? What's that? Flash can generally be approximated to D50. An exact match is not really that important. When the calibration illuminant is said to be different from D50 there will be chromatic adaptation transform applied, the reason to simulate the slight appearance differences of colors under different light, making reds a bit brighter for StdA for example. In theory the color temperature estimation of the DCP becomes better with an exact calibration illuminant, but that's only in theory, and for single illuminant profiles it has no effect, and if you use a raw converter like Lightroom that takes custom wb from the temperature estimate rather than storing the true RGB multipliers you'd probably want to embed the original ACR color matrices anyway. I don't know how Iridient developer does that aspect.
-
Hi Anders
Thank you for your extensive answer. It will take me a while to digest. For now only a reply to your question
> Exif tag 4? What's that?
In the command reference of make-profile, I read:
"-i <calibration illuminant>, this is the illuminant the target was shot under, that is the illuminant the target’s RGB values was generated for. Can be specified as an exif light-source name or number, [...] "
So I entered something like 'exif light source number' into Google and came to this page:
http://www.awaresystems.be/imaging/tiff/tifftags/privateifd/exif/lightsource.html
So how would I enter this with the -i flag? -i 4? or -i D55? or -i 20?
Hoping not to disturb you (much) more in the development of the GUI version ...
-
So how would I enter this with the -i flag? -i 4? or -i D55? or -i 20?
Oh I had forgot about that, you can indeed type "-i 4", or "-i Flash", that's the same thing. However if it complains that it needs spectrum (DCamProf have no spectrum for Flash illuminant in its builtin database, don't think there is a standard for it) you can substitute with -i D55, or use a blackbody spectrum "-i 5500K"
-
Thank you!
-
Now I thought I had digested it all, and composed this command, but I keep getting the manpage (no hints at the top of the Terminal output):
./dcamprof_v104 make-profile -n Sony ILCE-7RM2 y YYZ -5 -i D55 -I D50 -B -S -t linear combo.ti3 -r dump1 HB_a7r2_combo.json
Please help...
-
Now I thought I had digested it all, and composed this command, but I keep getting the manpage (no hints at the top of the Terminal output):
./dcamprof_v104 make-profile -n Sony ILCE-7RM2 y YYZ -5 -i D55 -I D50 -B -S -t linear combo.ti3 -r dump1 HB_a7r2_combo.json
Please help...
A quick and dirty question = why "y" and not "-y" ? and where are the numbers for X, Y, Z ?
-
Ooops! That should have been -y of course. And the numbers - uuh, I thought the -5 would apply to all - X, Y and Z (XYZ, not YYZ, of course). Now I tried with -y -5, which is an exact quote from the documentation, but that results in the manpage, too. The complete command was:
./dcamprof_v104 make-profile -n Sony ILCE-7RM2 -y -5 -i D55 -I D50 -B -S -t linear combo.ti3 -r dump1 HB_a7r2_combo.json
Dropping the -y part alltogether doesn't help either. Nor does quoting the "Sony ILCE-7RM2".
Now I really don't know. Maybe I should wait for the GUI version...
-
Ooops! That should have been -y of course. And the numbers - uuh, I thought the -5 would apply to all - X, Y and Z (XYZ, not YYZ, of course). Now I tried with -y -5, which is an exact quote from the documentation, but that results in the manpage, too. The complete command was:
./dcamprof_v104 make-profile -n Sony ILCE-7RM2 -y -5 -i D55 -I D50 -B -S -t linear combo.ti3 -r dump1 HB_a7r2_combo.json
Dropping the -y part alltogether doesn't help either. Nor does quoting the "Sony ILCE-7RM2".
Now I really don't know. Maybe I should wait for the GUI version...
command line is
dcamprof make-profile [flags] <input-target.ti3> <output-profile.json | .icc | .dcp>
which means all "-" flags shall preceed .ti3 & .json... in your case you put -r flag in the wrong place
and the text for -n shall be quoted if it has spaces
---
dcamprof make-profile -n "Sony ILCE-7RM2" -r dump1 -y -5 -i D55 -I D50 -B -S -t linear combo.ti3 HB_a7r2_combo.json
-
Bingo! Thank you so much, this worked. So the following question is of more theoretical interest:
I should of course have been aware of that the flags must be before any file names. But if I put '-r dump1' as the last flag, but before 'combo.ti3', I get 'Could not open file "-r". No such file or directory.' Why is that?
-
Bingo! Thank you so much, this worked. So the following question is of more theoretical interest:
I should of course have been aware of that the flags must be before any file names. But if I put '-r dump1' as the last flag, but before 'combo.ti3', I get 'Could not open file "-r". No such file or directory.' Why is that?
Each command expects the command line end a file name or two (depending on command), the make-profile command should end with an input file and an output file. The parameter parser is basic and just scans until there's two arguments left on the line and treats them as the two last file names, so the file names must always be last, but they could be named anything, even "-r" if you like :-)
-
Thank you for your reply, Anders. But I don't understand. If I put -r before combo.ti3, then there are 2 file names at the end of the command; and -r looks like a flag. Why is it interpreted as a file?
-
Thank you for your reply, Anders. But I don't understand. If I put -r before combo.ti3, then there are 2 file names at the end of the command; and -r looks like a flag. Why is it interpreted as a file?
Checked the code, it really looks for the first argument which not starting with '-' to detect where the options end, so my above description was not fully correct. Anyway, the standard among command line tools in general is that the command line is ordered "command flags/options files". I'm a bit environmentally damaged :) as work a lot with command line systems in my regular line of work, so I haven't really given it that much thought.
-
Thank you for clarifying! - Good light - Hening.
-
:-) Erik :-)
Checked the code, it really looks for the first argument which not starting with '-' to detect where the options end, so my above description was not fully correct. Anyway, the standard among command line tools in general is that the command line is ordered "command flags/options files". I'm a bit environmentally damaged :) as work a lot with command line systems in my regular line of work, so I haven't really given it that much thought.
-
I'm trying to make new profiles. I have put all the binaries and ref files in one folder, which I then have moved to a new directory. If memory serves me, this has worked before. Now I get "./scanin - no such file or directory". Same with dcamprof (which I ran without parameters just for the test.) Like the screen shot shows, the files are in fact in the directory. What is wrong?? Re-start of the Mac does not help. MacOS 10.10.5. ??
-
It doesn't look like you are in the right directory in the terminal. Try typing "pwd" to show the directory you're in, and "ls" to list which files that are there.
You change directory with "cd", but note that if the directory names contains spaces you need to prepend each space with a "\". Use tab-completion (write the particial directory/filename and press tab to fill in the rest) to make it smoother. (These things are not related to DCamProf but just how the terminal in MacOS works, which is unix-based so it works nearly the same as in Linux.)
-
I'm trying to make new profiles. I have put all the binaries and ref files in one folder, which I then have moved to a new directory. If memory serves me, this has worked before. Now I get "./scanin - no such file or directory". Same with dcamprof (which I ran without parameters just for the test.) Like the screen shot shows, the files are in fact in the directory. What is wrong?? Re-start of the Mac does not help. MacOS 10.10.5. ??
may be you want to download some free visual file manager for OSX like muCommander ( http://www.mucommander.com/index.html#download ) to ease the pain ?
-
Thanks to the both of you for your fast replies.
I am aware of that this has nothing to do with dcamprof, but with Unix. I just don't know where else I should look for help in this case. -
Rather than entering the Path manually, I use ShellHere, which can be seen in the top left of the screen shot I included. You open a folder, then click ShellHere in the folder's menu bar, and a Terminal window is opened "in" that folder.
This has always worked for me. But now I can see that it returns "-bash: !/0-bin: event not found"
(The name of the folder "in" which I am supposed to be is "0-bin+ref dcamprof 1.0.4")
So it looked like ShellHere did not worke this time.
'pwd', entered in the Terminal without anything else, shows me that I am in my Home folder.
MuCommander requires to install an old version of Java which I don't feel safe to do.
Installing a fresh copy of ShellHere, deleting it's plist an restarting the Mac didn't help. Nor did copying the Path that ShellHere displayed and entering it manually in the Terminal. Then I dragged the folder, which is on a disk other than the startup volume, to the Desktop - and that worked!
And finally I figured out what was wrong: The name of an enclosing folder contained a "!" !
Sorry I bothered you with this.
I'll soon get back with a question that HAS to do with dcamprof ;-)
-
MuCommander requires to install an old version of Java which I don't feel safe to do.
you can install the latest then, it is not mandatory to install the old version specifically...
-
Hm - I have Java installed, it seems to be from September 2014, and MuCommander wouldn't launch, required a 'legacy' version. - Anyway, my problem seems to be solved for now.
Thanks for your concern!
-
Oops, now I updated to the latest Java, and now muCommander installs and launches without problems!
Thanks!
-
Oops, now I updated to the latest Java, and now muCommander installs and launches without problems!
Thanks!
There are plenty of non-Java alternatives - http://doublecmd.sourceforge.net/ or http://mac.eltima.com/file-manager.html or http://devstorm-apps.com/dc/
-
Thank you for these tips, Alexej. Anyway, it is a good idea to have the latest Java installed.
-
I'm pausing GUI development for a while to look into color appearance in tungsten light.
95% of the visual color testing and tuning of DCamProf has o far been made in daylight/flash conditions. I myself and most users use the cameras in this condition almost exclusively, so it's been all good. However recently I've got some interest in looking over color appearance in the tungsten light (StdA) range. It seems like it's maybe not as good as it could be there.
I have a bit of research to do, but here's what I know so far. The problem with StdA is that it's way different than the standard illuminant D50 used for screens and printing. A camera profile must convert all colors to D50 as the connection space for the color pipeline uses the D50 whitepoint. So the challenge here is to model the color appearance in StdA and convert each color to a corresponding D50 color. There's already well-established standard models for that, chromatic adaptation transforms (CATs), and the current best is CIECAM02's CAT02 which DCamProf uses today to figure out how colors appear in StdA and convert that to D50. Broadly speaking the largest difference is that reds appear a bit brighter and some colors a bit warmer in tone.
However this is often not enough in terms of warmth, I often find myself setting a creative white balance to a warm tone to better mimic the color appearance. It also seems (I don't know this for sure though) that camera preset white balance for tungsten is often put a bit high in temperature to offer some residual yellow tinting effect. This indicates that there's "partial chromatic adaptation" going on, that is our eyes have only partially adapted to the StdA white and still experience the whites (and all other colors) as tinted slightly yellow.
Something around here it seems to break apart a little, modeling colors after 100% adaptation in the profile, and adjust-to-taste for partial adaptation using white balance maybe doesn't really cut it. Unlike in D50 conditions we got to accept in a larger extent that we can't do anything accurate, as there's too many parameters in human chromatic adaptation, there's short-term effects, long-term effects, and with some mixed lights thrown in there it gets even worse. However, it may be possible to make something better than the current, where it seems to me that to get the warmth I find suitable for some of the saturated colors I need to put so strong off-white white balance that the whites become too tinted. The CAT02 does model partial adaptation which I will experiment with a bit, not sure yet if that will be enough though, or if I'll have to make manual adjustments to it.
-
Nikon seem to have come to similar conclusions to you. The have introduced, in addition to a pure, grey is neutral, white balance additional auto white balance options that retain some warmth that can make portraits look more pleasing and the look closer to what you remember.
Whether it is worth doing anything other than generating neutral white balance and letting the user warm pictures in post according to their own taste, is an open question.
-
I haven't had time to do all testing yet, but it seems like just adjusting the white balance to suitable off-white is pretty close to the result of a CAT partial chromatic adaptation model, for example setting to 80% chromatic adaptation StdA CAT02 is for most color sub 0.5DE from setting white to RGB 1.03,0.98,0.87. This indicates that I won't get much different result from working with the CAT rather than just changing white balance alone.
If so, I think there may be a need to make an own custom CAT just for StdA, because I see some problems in the results, mostly in red-brown colors - simply the hues are clearly different from the real thing, not warm enough and too much green in them. Chromatic adaptation is messy to work experimentally with though so I shouldn't get too fast to conclusions.
I'm a bit surprised though that the CAT from D50 to StdA does not seem to be that accurate, I haven't come across anything in the literature that says that it has these kinds of problems, so it might be some other thing I'm observing. I've looked at each step though, and it does seem like the color error is introduced in the CAT.
The design idea for improvement I have currently is to use a CAT and you specify a partial adaptation you want at profile design time. If you don't specify full adaptation the software calculates an ideal off-white white balance for you that you can use when shooting, and the profile is optimized specifically for that off-white white balance. It's still unclear though if it's necessary to optimize for a specific off-white or if the difference will be insignificant from always designing for 100% adaptation.
-
Findings so far regarding color rendering for tungsten light; cameras are generally more dependent on LUT correction in the StdA range than regular daylight. This means that if you set another white balance than the one the profile was designed for the LUT offset will be wrong and you get the wrong corrections. For a daylight profile it usually doesn't matter much as the matrix is already good and the LUT has very broad corrections. With StdA it matters a bit more.
This is a bit of a problem as in StdA you typically want to have a creative white balance, usually a warm one, as discussion in the previous posts. I'm thus looking into the possibility to offset the LUT so it matches a pre-defined warm white balance setting.
I've also done some matching-color-by-eye tests, and the CAT02 indeed does seem to quite well simulate the color appearance changes when you change from D50 to StdA. However, regardless for how long I sit under StdA, white keeps a warm tone. I reach about 80% adaptation I would say. I've asked others to look and they get the same result. I also have discovered some weakness is the CAT02, which may be a side-effect of this partial adaptation thing, I don't know. To the eyes reds get more saturated and gets a bit orange added to them, and orange-skin-brown gets an overall saturation increase, which CAT02 doesn't model. It's not huge differences, say ~2 DE or so, so my guess is that it's to small to be included in the CAT02 model. However while small, those changes hits a sensitive areas, including skin tones, and it's not just that the CAT doesn't make it as accurate, it makes is subjectively less pleasing too with the cooler under-saturated colors.
I will thus look into making an adjusted CAT to be used when modeling StdA color appearance.
-
One things that I often find is that if I use the Auto WB in Lightroom, I get a less pleasing picture when shooting under tungsten or mixed light settings, than the camera (Leica SL's) chosen balance, even when the latter is not quite right. So I come back to an earlier point: do you want the profiler to create a pleasing picture, or a "correct" one that can be adjusted to taste. Regrettably, I suspect that you will need to implement both options and see what people find, in real life.
-
Here comes some example images.
There's two new things I've implemented demonstrated in these images.
1) A new CAT which makes some adjustments to CAT02 for the tungsten (StdA) color appearance simulation.
2) Possibility to offset the LUT so that it uses a preset creative white balance (warm tone for StdA) instead of white.
These features are useful for tungsten lighting conditions, you don't need them for daylight conditions. The example image shows a CC24 and a cello. It doesn't happen too much to the cello with the CAT change, so you have to look at the CC24 mostly. I actually had a more diverse test scene with more objects but I happened to start put together the various combinations on this image and I don't want to redo it again :-). The cello is however an excellent object to test that the creative white balance is rightly tuned.
The creative white balance was hand-tuned so that the color appearance of the objects would appear as true to the real scene as possible. That white balance ended up being very close to the in-camera tungsten preset, so that is what is used in the end in these images. I think it's often the case that tungsten in-camera WB is set at say 3200K rather than the 2850K of StdA, and this seems suitable to simulate the partial chromatic adaptation that takes place.
So on to commenting the test images. The first row shows with white balance picked from the CC24. While it sort of looks right when briefly looking at the CC24, it becomes rather obvious that an object rich in red like the cello doesn't get the right hue and looks a bit desaturated. This is not a CAT problem, but that we need to use a different white balance as discussed above. I think gray cards / colorcheckers are often mis-used in these conditions, many think that the "correct" way is always to pick the white, which works fine in daylight conditions but makes tungsten scene colors look way too cool.
We can observe the differences between the old (CAT02 original) and new (CAT02 + adjustments) CAT. The new provides slightly increased saturation orange-brown-skin, and in red a further saturation increase plus a slight hue shift towards orange.
The second row shows when we apply the creative white balance. The problem we get here is that the correction LUT assumes that white is white rather than slightly yellow, which means that when we use a creative white balance the LUT gets offset and makes the wrong corrections. This has always been a problem, but for daylight it's minor as temperature changes is often less aggressive, and the LUT corrections are most certainly so. Cameras usually need stronger LUT corrections for tungsten than daylight. The difference between compensated and uncompensated is still not huge though, which is fortunate, as we can only design for one specific white balance in any case. One may need to overlay the images to really see what happens, a slight saturation loss in some colors and a little bit green cast. The exact effect will depend on camera.
The final row shows when both creative white balance is set and the LUT white is adapted to match that, bottom right is thus the new result with both new features, while middle left is the result you get with the currently available DCamProf.
The separate image shows what happens if you use a D50 profile instead of a proper StdA profile. Generally loss of saturation and a slight green cast of warm colors.
While working on this I also noted that if you shoot in light in-between tungsten and daylight, it seems like a dual-illuminant profile is almost necessary to get good results (unless you make a profile for exactly that light of course), but if you only have one profile it's better to use one made for D50. This is not that surprising considering the large differences in matrix/LUT, plus the CAT on top.
-
I don't have a cello to hand, but the first row looks most natural to me; the others are a bit too red (looking through Chrome web browser on a Mac).
-
I don't have a cello to hand, but the first row looks most natural to me; the others are a bit too red (looking through Chrome web browser on a Mac).
String instruments vary quite a lot in color, and I'm told that this cello was the reddest in the orchestra so it's a reason it looks "too red" :-). It's my own photo and I have the objects at hand so I've been able to tune against the real thing. Actually my own hand-tuned white balance made it a tiny bit redder still, but as the fixed in-camera tungsten preset was so close I used that instead. It should also be said that the lamp is at about 2700K, which is in the lower range of what typical indoor light is, eg the light is redder than in many cases.
It is a little bit messy to match tungsten scene with what you have on screen due to the widely different whitepoint, you can't really have them side by side. I had mine set up behind my back so I turned around, let eyes adapt, remember the colors as good as possible and turned back to the screen, let eyes adapt. I used other objects too and my own hand to include some skin, and I'm rather confident that the match is good.
It would have been better to show a skintone sample. The reason I had a cello in there in the first place is that I got reports from a user which had troubles with string instruments in photos of an orchestra not being red or vibrant enough. That particular problem turned out to be mostly related to setting the appropriate white balance (and using an StdA profile rather than D50), but the new modified CAT also helps, especially the less red instruments.
All that said I think there will be quite varying tastes on how off-white you should make the white balance for simulating the color appearance under tungsten. With the appropriate CAT the inter-color relationships are correct regardless of the whitepoint, so it doesn't matter that much, and I expect users come to different conclusions on what's the best approach, and it depends a bit on viewing condition. The white balance was tuned in a dark room condition (the only practical when having the real scene in the same room), and when viewing it in a brighter daylight-lit room like I do now I'd say that indeed the warmness is a little overdone and I would probably back off a little, but certainly not all the way back to a pure white.
There is one thing I've thought about related to this, I don't think it's necessary but I'm not 100% settled, and that is if you may want to make a special CAT that lets white be whiter but keeps redness of the saturated colors. That is you could have the cello as red as in the bottom row, but the whites (almost) as white as in the top row. The reason for that kind of twist would be exactly the stuff discussed above, that is when the photo is viewed in bright conditions you may want to make the whites purer, but still not desature/cool the reds-browns. I wouldn't handle that as part of the CAT though, but sort of a special offset white-balance handling.
-
The second row actually looks the best to me maintaining more color balance relationship. It also has more color gamut where sampling the cyan patch in sRGB shows a low red (17red) and the others show it as more sky blue with red being over 40 in sRGB. If tungsten is suppose to be a yellowish to orangish warm hue cyan shouldn't look sky blue but even more intense cyan which is a bit greenish blue instead of magenta tinted light blue. There are many varieties of tungsten bulbs I'm sure exhibiting different hues so I'm going by regular soft white household bulbs which to me show more yellowish orange than yellowish red where my camera records more of a red bias just with the default ACR profile.
I wonder how this 2700K tungsten profile would render 2700K-ish cloudless sunset landscapes? Thanks for the samples, Anders.
-
Here's an animated GIF switching between the standard CAT02 and my modified CAT02 for 2700K, making the differences more apparent than the side-by-side frames in the previous post. The one with the brighter red is the modified CAT, it's swapped every two seconds. You need to click the image to make it to animate.
This was generated with the dcamprof si-render command from a spectral image using a black-body illuminant. Here white-point is kept at 1,1,1.
For reference I've also attached the plain render under D50 (no CAT).
To recap, what we show here is how colors appear differently under tungsten compared to daylight, even with 100% chromatic adaptation, that is "color inconsistancy" phenomena. To simulate this important if you want colors to look realistic when you make a tungsten camera profile. CAT02, the standardized CAT in the CIECAM02 color appearance model, generally does a good job but I think the brightening/warming effects of the reds and saturation increase of orange-brown-skin is not strong enough, and this is what I've targeted with the modified CAT.
-
Note that this is a work in progress, I'll rest a bit and get back to it, and make a deeper look into skin colors which is the most sensitive to not mess up. The likely result of that is that there will be some scaling back of the effect, ie the final differences will be smaller than it is in the above samples.
-
Note that this is a work in progress
Indeed, had some setbacks. It seems like it won't work out that well for many (possibly most) cameras. The key problem with tungsten lighting is that cameras are made to match colors in daylight/flash conditions, which means that in tungsten you either must have poor color matching or strong LUT correction. There's nothing wrong with a strong LUT correction if it just rotates and increases perceptual distance (DE) between the colors. However if it has to make strong compressions, that is put colors closer together than they are in the matrix, you will likely get tonality issues.
Gradients may still look smooth, but colors in a compressed range may look a bit flatter, and if this happens in the skintone range it can be a serious problem.
Doing manual optimization and tradeoffs these issues can be avoided, but the automatic optimizer still have issues here. I'm probably going to look into a LUT compression limiter parameter and see if that works out. In any case, when the LUT is properly relaxed you may very well end up with a result that is not that similar to what the CAT thinks the colors are, the difference between the old CAT and the new modified discussed above is often ~0 after LUT relaxation.
-
that cameras are made to match colors in daylight/flash conditions
Is that really correct though? Typical sensor CFA are not daylight balanced - closest to daylight balanced were Kodak sensors used in ProBacks and Phase One P20 and P25. The red channel on those is stronger than green and blue is a bit weaker than green which in daylight results in WB closer to unity. In reality though, since daylight is of a wide variety, real unity is rarely achieved.
Most modern sensors SPD are tuned for something else though - not daylight. Typical daylight WB on a lot of modern sensors requires lifting both red and blue (the latter usually a bit more). Cannot say what lighting this is designed for - possibly a compromise between tungsten and daylight?
-
Is that really correct though? Typical sensor CFA are not daylight balanced - closest to daylight balanced were Kodak sensors used in ProBacks and Phase One P20 and P25. The red channel on those is stronger than green and blue is a bit weaker than green which in daylight results in WB closer to unity. In reality though, since daylight is of a wide variety, real unity is rarely achieved.
Most modern sensors SPD are tuned for something else though - not daylight. Typical daylight WB on a lot of modern sensors requires lifting both red and blue (the latter usually a bit more). Cannot say what lighting this is designed for - possibly a compromise between tungsten and daylight?
One would expect 4000K but I recall a claim near 3300K for older Canon sensors.
Met vriendelijke groet, Ernst
http://www.pigment-print.com/spectralplots/spectrumviz_1.htm
February 2017 update, 700+ inkjet media white spectral plots
-
Is that really correct though? Typical sensor CFA are not daylight balanced - closest to daylight balanced were Kodak sensors used in ProBacks and Phase One P20 and P25. The red channel on those is stronger than green and blue is a bit weaker than green which in daylight results in WB closer to unity. In reality though, since daylight is of a wide variety, real unity is rarely achieved.
Most modern sensors SPD are tuned for something else though - not daylight. Typical daylight WB on a lot of modern sensors requires lifting both red and blue (the latter usually a bit more). Cannot say what lighting this is designed for - possibly a compromise between tungsten and daylight?
They all need a matrix of course, but with one a modern camera matches a color checker pretty good, while doing less well in tungsten -- with CAT at least. That's my experience from profiling cameras, I don't have a super-large amount of cameras tested in that range though as most profiles I do are D50-D65 ones.
The thing is that a daylight spectrum is quite even, no exaggeration in any part of the spectrum, while tungsten is very slanted. I think it would be unwise to optimize for tungsten, at least the 2850K ones (StdA). The CAT complicates it too a bit, especially if I go ahead adding non-linearity to it, as the camera then no longer can make a linear match.
So far I haven't profiled a camera that haven't been more difficult to deal with in StdA than daylight, but temperature scale isn't linear and there's a big difference between 2850K and 3300K or 4000K, so it could very well be as you say that many cameras ideally operate in that sort of light.
-
They all need a matrix of course, but with one a modern camera matches a color checker pretty good, while doing less well in tungsten -- with CAT at least. That's my experience from profiling cameras, I don't have a super-large amount of cameras tested in that range though as most profiles I do are D50-D65 ones.
The thing is that a daylight spectrum is quite even, no exaggeration in any part of the spectrum, while tungsten is very slanted. I think it would be unwise to optimize for tungsten, at least the 2850K ones (StdA). The CAT complicates it too a bit, especially if I go ahead adding non-linearity to it, as the camera then no longer can make a linear match.
So far I haven't profiled a camera that haven't been more difficult to deal with in StdA than daylight, but temperature scale isn't linear and there's a big difference between 2850K and 3300K or 4000K, so it could very well be as you say that many cameras ideally operate in that sort of light.
My little experience with cameras says that extra warmth of camera BB for tungsten is caused by trying to achieve accepatble non-magenta skin tones. You may prove wrong, but it seems that in general all hues look OK after demosaicking without an input profile but skin tones are almost always magentish, it is a common trend in DSLRS. There are many points to be discussed on getting accurate\pleasing (as in non-magenta) skin tones and keeping the neutral axis neutral. That is my problem with Argyll - built matrix profiles with good skin tones rendered the gray midtones greenish.
As for the tungsten - CIECAM IMO works good, the most difference between the upper and second row is the gamma curve (lower in the first row, check the neutral axis) hence the saturation loss and "out of balance".
-
If dcamprof manages to solve this problem ... the sky's the limit
-
I've been working on a number of issues. One is a new LUT relax algorithm that is more robust if there are some patches struggling in the opposite directions. I had problems on this with my test camera in tungsten which caused too aggressive compression in the skin-tone range, making tonality suffer. The new relax algorithm will handle those situations better.
I've also looked a bit more on screen-to-reality match, and that's ongoing. I think the dim room conditions I'm having in my home office at night this time of year is disturbing a bit, and the adaptation back and forth to the 6500K whitepoint on screen is a bit wide gap, especially when experimenting with tungsten. I'm getting less certain about the CAT02 lack of red saturation in StdA color appearance, I still think there is some lack of it, but not to the same extent as in the first rounds of experiments. I need to work more on whitepoints and test different screens etc to make sure the tuning is not disturbed by some viewing condition problem.
The off-white LUT anchor feature for StdA is probably not going to be in. It seems to me that in the end a suitable off-white creative balance is about 200K warmer than neutral-white, and that together with the better relax algorithm make the LUT shift so small it's not really worth messing with offsetting it, but it's not final on that either.
Regarding hues in StdA I think that for most cameras the case will be that you will see more of the native properties of the camera's color filters than you do in daylight. Due to the very slanted illuminant spectrum the specific choices of how they've balanced the sensor color filters will show through more. While you in theory can stretch with the LUT and create anything you want, in practice you don't want to do that as it will hurt tonality due to excessive local bending and compression. A good LUT is one that is pretty close to the matrix result.
-
Worked a bit more with adapting whitepoints, even setting the screen extremely low to match my tungsten lamp at hand. The screen does look overly yellow with such a low whitepoint, which is about the mysteries of chromatic adaptation to displays which seem to be partial (that's why many use 6500K to match 5000K prints). Raising a bit I got a better perceptual match.
With (perceptually) matched screen whitepoint the CAT02 seems to be pretty much spot on, no modification required. So it seems I'll be backing out that modification too. I'm not sure yet what I'll end up with, the only modification that seems to stick so far is the improved auto-relax algorithm.
It's not over yet though. I have not yet fully understood the phenomenon that makes you set a warmer white balance to better represent the warmth of a tungsten scene, and that could still have an effect on how we want to deal with the CAT.
-
It is partial adaptation for sure. That is why I do not see any sense to recreate specifically the tungsten "feel" for A conditions. In our memory the only "natural" color we have is basically the skin tone (and foliage to a smaller degree) since all the other significant colors such as sky, sea water, purple flowers, even lemons, peaches, oranges, rainbows - are memorized under all kinds of daylight. The skin tone on the other hand, is just getting WORSE under tungsten as in redder\bluer. That is the main issue. Since I do not believe in LUTs (to many chances to get into weird colors in real scenes) and "accurate" matrices (since any matrix is still a compromise due to non-L-I CFAs to be made between hue, saturation and lightness accuracy), I feel that the ultimate goal is to get a "reliable"and "mallable" matrix for any daylight (and probably an additional for A), which:
1. recreates accurate hues of the NATURAL memory colors (skin tone being of the highest priority for non-heavy postpro PP wedding\fashion guys)
2. provides equal perceptual saturation\lightness for all hues (a precondition for the matrix to give "mallable" material for postpro)
3. is normalized (keeps greys greys)
4. allows some freedom for under\overexposure.
Actually what I have found opening RAWs of different DSLRS is that most natural hues except the skin tones are quite OK without an input profile. The skin tones though are reddish\purplish. The problem with the profiling based on dE verification is that there is no weighting what patches are really of the utmost importance to be accurately reproduced and what deviation from the target is acceptable. For example, a Caucasian skin tone shifted a little bit too green from the (usual rendering) too red gives orange which means too tanned but ok. On the other side, a little too blue from the usual too red gives purple which is a disaster, though people seem to get tolerance and even used to that!
All these thoughts and observations lead me to the conclusion that even matrices should be tweaked by hand. The DNG profile editor comes in very handy, but not without its own limitations like having a base profile and non-linear inbuilt curves which are affetcing the primary tweaking. I would prefer an opportunity to have like an 'identity" matrix and a linear curve to start with, not an ACR profile embedded to the DNG.
-
I don't believe in matrix-only profiles for general-purpose work due to 1) how do you apply a contrast curve to a matrix-only profile? I don't think RGB curve or Adobe hue-stabilized version is the answer, and a linear curve is really only for repro work 2) raw converters expect the profile to deal with gamut compression, and you can't do that in a matrix.
That said I do believe that a LUT working on the colorimetric level should not make large changes to the matrix, and you can see that in commercial profiles to that they have similar approach. The LUT contributions is mostly about tone reproduction, ie the curve, and gamut compression.
The GUI version of DCamProf will allow manual matrix tuning, but not really by entering numbers manually, but through manually-controlled refinements on the automatically derived base matrix, it's basically a graphical version of the make-profile/-v parameter of the command line program. I've thought about including numbers/sliders though for those that would perfer that type of direct tuning, probably not in the first release though.
DCamProf matrix optimizer is currently limited to make white-point preserving matrices, that is you cannot make a matrix that tints the neutral axis (you can make an ICC profile that does that, but that's about white balance compensation which is another thing). I have so far not seen any reason to change that. DNG profiles actually requires the ForwardMatrix to be white-point-preserving or else Adobe Camera Raw won't accept the profile.
I often do tweak the matrix by hand, "because I can". My favorite modifications is to match skin better and let others color suffer a bit, make sure deep blue is not too dark and is rather turning towards cyan than magenta, and also looking at the reds which I rather have warm than cool. Then the LUT make minor adjustments on top of that. Greens are generally stable as they have strong contribution from all three channels. This is for daylight. For StdA I still need to gain some more experience.
Lightness control is an issue for matrices. Matrices have pretty large lightness errors in general, but I think it's not a big issue as while we can detect lightness errors very easily in an A/B swap, we don't really remember lightness well, or maybe more accurately put, a color slightly light or dark is not seen as unnatural while a hue that is off or under/over-saturation can be quite conspicuous. I think this is because in an image lightness could be the result of the light in the scene, so the eye doesn't really know what's the right level, while (normal) light doesn't cause hues or saturation to change so we react to that even when we have no reference to compare to. So I think lightness should be the least thing to focus at.
To make the matrix "mallable" it should not contain any strong negative factors, which can lead to clipping. One thing I've worked with quite much in DcamProf project is the overly sensitive blue channel (probably for better ISO and tungsten performance) which cause strong blue subtraction in daylight which can cause clip to black for saturated colors. The solution is to sacrifice lightness accuracy and lighten deep blues significantly. Likewise early results suggest that reds must be lighter than natural for an StdA matrix in order to provide stability.
-
I don't believe in matrix-only profiles for general-purpose work
Would not matrix profiles give you something LUT generally could not - smoothness?
Technically speaking a combined approach is possible - apply raw corrections prior to demosaicing, demosaic, apply matrix profile, apply corrections needed at this stage and save (say in UpLAB). Then assign profile with correcting LUTs to work on the image and prepare final output image.
This approach will work with correcting LUTs and look LUTs - Kodak used something of the kind in their camera SDK (if in-camera matrices were used) though with different working space.
-
apply raw corrections prior to demosaicing
which ones - some black point offset and such, or any color corrections as well?
-
which ones - some black point offset and such, or any color corrections as well?
In the context of profiles discussion - it does not matter.
But if you take RPP for example quite a few: WB, exposure correction, contrast, black levels and raw channels sharpening.
-
A LUT that makes colorimetric corrections can indeed cause issues with smoothness, or tonality. That's why I think a general-purpose LUT should be close to the matrix result. Scaling lightness is extra risky.
Then you can have a second stage in the LUT that makes gamut compression, and a stage that makes tone operator adjustments, and possible perceptual look adjustments (cooling shadows, warming highlights and such). It's relatively easy to maintain smoothness in such stages, but you can mess up there too of course.
Matrix-only does have some strong merits, especially in technical photography, such as merging HDR and and other applications when linearity is important. Still I don't see any camera manufacturer or big name raw converter maker do matrix-only profiles, and the reason is as said that they want to control the tone operator, gamut compression and have the possibility to add non-linear looks. Sure one can happen to like and prefer matrix profiles, but to the overall photographic community it's a small curiosity.
-
This is also a good read
https://photographylife.com/how-to-color-calibrate-cameras/
-
The answer to the question "why are colors so wrong?" is because they're designed to be "wrong". No of the big name raw converters try to make accurate/realistic color, they make pleasing color in the same way Fuji Velvia film or similar, twist colors to make more flattering result. Some have a signature "brand look", and the warm rendering of Capture One I'd say is such a thing. The make colors warm, because their users like it.
-
The answer to the question "why are colors so wrong?" is because they're designed to be "wrong". No of the big name raw converters try to make accurate/realistic color, they make pleasing color in the same way Fuji Velvia film or similar, twist colors to make more flattering result. Some have a signature "brand look", and the warm rendering of Capture One I'd say is such a thing. The make colors warm, because their users like it.
I dare to insist that their warm matrices are dictated by need to get the correct hue of the skin tones under daylight of any temperature as well as tungsten. Basically the RAWs wihout input profile look all red-cyan (with skin being red and all kinds of blue gravitating to cyan). The matrix twists that red-cyan colors to have also a yellow-blue axis. The balance of the red-cyan and yellow-blue in terms of correct hue, lightness and saturation makes a good matrix. Strong blue and purple is great but then we have purple pimples and lips, which overweights having strong blue. Also, having both cyan and blue is usually requires that both colors are not saturated and skin tones get pinky-red.
-
I dare to insist that their warm matrices are dictated by need to get the correct hue of the skin tones under daylight of any temperature as well as tungsten. Basically the RAWs wihout input profile look all red-cyan (with skin being red and all kinds of blue gravitating to cyan). The matrix twists that red-cyan colors to have also a yellow-blue axis. The balance of the red-cyan and yellow-blue in terms of correct hue, lightness and saturation makes a good matrix. Strong blue and purple is great but then we have purple pimples and lips, which overweights having strong blue. Also, having both cyan and blue is usually requires that both colors are not saturated and skin tones get pinky-red.
How is raw without input file defined in this case? Linear white-balanced raw data direct-mapped to sRGB color space? Raw data can look quite different depending on how you pre-process it.
If this would be the only way to make skin tones correct it's sort of strange that not everyone is making this color warmup, but rather I think C1 is one of the few. Look at the blues and look at the reds.
I have contact with some DCamProf users via email now and then. Most users use my software as they need something that is more accurate/realistic but still not reproduction (ie they need a contrast curve) than bundled profiles can offer. However there's also the users that don't even compare to the real thing, but just compares with other renderings from other software/profiles, and decide by taste what "looks best" -- this is the normal mode of the typical photographer. To a software developer it can be quite frustrating as there's a bunch of different tastes, and you need to figure out what elements that they do like (they usually can't say, only tell if one image looks nicer than the other) and which they don't, and then in the end be the final decider of what gets in there and and what does not.
In the medium format segment I've looked a bit at Leaf, Phase One and Hasselblad all very respected color renditions, all very different even when the hardware is practically the same. None is very accurate. All have gone through the process of making a subjective profile to suit their customers, and probably having some in-house person being the "master decider" on how the final look should be.
-
raw in this case mean RT's Neutral profile, demosaiced manually whitebalanced, prophoto as working profile (clipped to monitor space) with some tonal curve (I do not know what curve this is).
-
The answer to the question "why are colors so wrong?" is because they're designed to be "wrong". No of the big name raw converters try to make accurate/realistic color, they make pleasing color in the same way Fuji Velvia film or similar, twist colors to make more flattering result.
I seriously doubt they (raw converters) really have that specific intention in their profiles. Firstly films were not just designed by engineers - but profiles and cameras nowadays unfortunately mostly are. And then looking at Adobe profiles for quite a few cameras - the results they provide out of the box are very far from pleasing. For me that was precisely the reason I stuck with older cameras and learning how to get camera SSF and getting into profiling myself.
-
raw in this case mean RT's Neutral profile, demosaiced manually whitebalanced, prophoto as working profile (clipped to monitor space) with some tonal curve (I do not know what curve this is).
Why ProPhoto? It may not even be close to get any representative colour. Why not XYZ directly then in this case (it is after all also sort of RGB space).
-
I seriously doubt they (raw converters) really have that specific intention in their profiles. Firstly films were not just designed by engineers - but profiles and cameras nowadays unfortunately mostly are. And then looking at Adobe profiles for quite a few cameras - the results they provide out of the box are very far from pleasing. For me that was precisely the reason I stuck with older cameras and learning how to get camera SSF and getting into profiling myself.
I think companies like Phase One has had much more thought put into profile making than Adobe. Phase One makes cameras too. And I'm quite sure they have been involving their photographers when designing profiles. At least that is what they say, also when it comes to their cameras.
One can see when studying old Adobe profiles that profile design has changed over the years quite much, they have become better with time. They too have learnt stuff. Maybe they even have some good profiles today, I don't know as I'm not the kind of guy that buys and tests the latest all the time :-)
-
No of the big name raw converters try to make accurate/realistic color,
P1 at least does supply repro profiles (and fully linear .fcrv curves) for C1 for their backs ...
-
camera SSF
how is your monochromator/integrating sphere device doing, you did not post anything about it for a long time...
-
P1 at least does supply repro profiles (and fully linear .fcrv curves) for C1 for their backs ...
Is it bundled in the main version nowadays, or do you need to purchase a "cultural heritage" edition? Anyway the point was that the reason colors are very inaccurate in the general-purpose profiles is that they are designed that way. The linked article above mentions a few technical reasons too, which is probably just right, but the overall reason is -- designed color. Or perhaps lack of design if you look at some early Adobe profiles, if you just slap some random contrast curve on a matrix-calculated-from-a-target you get quite large effects on the colors too.
Here's a "market speak" description of how Hasseblad developed their look, which they've even given a name "Hasselblad Natural Color Solution" http://static.hasselblad.com/2015/02/hncs.pdf doesn't really say much, except that it's a combined effort of both color science and involving photographers and their color perception.
-
Is it bundled in the main version nowadays
they (profiles & curves) were always supplied with a regular C1
-
they (profiles & curves) were always supplied with a regular C1
Ahh now I see them, I was playing with P45+ pictures, and the repro profiles didn't start until the IQ series, and not for all backs, at least not in my old version 8 on this computer.
-
Ahh now I see them, I was playing with P45+ pictures, and the repro profiles didn't start until the IQ series, and not for all backs, at least not in my old version 8 on this computer.
well, at least they do for some... and in any case it was nice for P1 to include them with regular distribution
-
Why ProPhoto? It may not even be close to get any representative colour. Why not XYZ directly then in this case (it is after all also sort of RGB space).
Downloaded an XYZ D50 adopted profile from color.org, will report back. Actually we might have a chat somewhere in Russian, your monochromator setup keeps me wondering if it is worth trying and how are resulting profles in terms of color constancy over the tungsten-D65 or so range of the black body SPD light (OK, I know the D is not ABB, still the locuses are close enough).
Edit: Just have found out that RT does not allow to choose an arbitrary color space described in a profile. The ProPhoto seems to be the largest of the available WORKING color spaces in RT and it has worked great so far for visual verifying of target-derived or base-matrix-tweaked profiles.
-
Actually we might have a chat somewhere in Russian
wow camera forum has the original topic
-
wow camera forum has the original topic
Yes, I`ve read that one too.
But talking Anders into making a GUI-based tool for matrix creating-tweaking with some freedom regarding exposure and preview tone curve seems sweet.
:D
-
I think companies like Phase One has had much more thought put into profile making than Adobe. Phase One makes cameras too. And I'm quite sure they have been involving their photographers when designing profiles. At least that is what they say, also when it comes to their cameras.
One can see when studying old Adobe profiles that profile design has changed over the years quite much, they have become better with time. They too have learnt stuff. Maybe they even have some good profiles today, I don't know as I'm not the kind of guy that buys and tests the latest all the time :-)
I cannot say for modern cameras but their latest incarnation of profiles rebuilt for older ones have actually been quite inconsistent and worsened in quite a few cases (older Nikons and Kodaks from my experience). Better results could be obtained by reverting to older ACR versions of those (in ACR calibration tab - thankfully it is still possible).
-
how is your monochromator/integrating sphere device doing, you did not post anything about it for a long time...
It is at the finishing stages of v2 of that. The manual version that I posted here was v0 ;)
I spent last year and a half (with lots of help and guidance from Iliah) assembling controller boards for fully automated setup with real time light specta measurement (concurrent with the shots taken). The setup is now assembled and partially debugged (firmwares are written for several Hamamatsu micro spectrometers) and the overall controlling software (home grown QT UI) is slowly progressing.
It will hopefully be ready and opensourced before summer.
-
Here's a "market speak" description of how Hasseblad developed their look, which they've even given a name "Hasselblad Natural Color Solution" http://static.hasselblad.com/2015/02/hncs.pdf doesn't really say much, except that it's a combined effort of both color science and involving photographers and their color perception.
In the days of film they involved people with art background rather than photographers from what I understand
-
Hi,
My feeling is that vendors want to surround colour handling with mystery. In the real world, I would guess that there is a bit more science and a bit less of mystery…
Best regards
Erik
In the days of film they involved people with art background rather than photographers from what I understand
-
I've been working lately with alternative tone operators. The "neutral" one is the DCamProf signature tone operator and will continue to be so, but I cannot deny that there are users that like the look an RGB curve (S-curve applied on R, G and B channels separately) provides. In addition to pure RGB and Adobes ACR (which works the same as an RGB curve, but with HSV-Hue kept constant) which by raising them to status of tone operator can be combined with gamut compression and look operators and be used in both ICC and DCP, I'm adding a variant called "RGB-Luminance".
It works according to a technique often used in RawTherapee where you have the possibility to manipulate two tonecurves. You then split your contrast into two curves, and apply one tone operator on one of the curves and another on the other. This way you can add saturation and hue control to an RGB curve by combining it with a Luminance curve for he shadow dip. Interestingly it seems this is the same type of technique C1 has in their profiles. By separating out the shadow dip you get a smoother flatter curve used by RGB which reduces its hue shift issues, and you also reduce the oversaturation issues, while keeping some of the subjectively attractive features of and RGB curve. The hue shift is good in some cases as the "hue spreading" means that you make colors more separated than they otherwise would be. At some point I may look into this a bit more also for the neutral tone operator, as I've seen in rendering of woods that its hue accuracy may compress the tones a bit too much, and one may need to make a tradeoff between perceptual hue accuracy and perceptual contrast/separation accuracy, eg spread the hues a bit to gain more separation.
DCamProf uses ProPhoto primaries for the curves, same as DCP/Adobe. This is an oversized colorspace but works well when the curve doesn't cause hue shifts, like Adobes curve or DCamProf's own neutral operator. However with an RGB curve if you apply that with Prophoto primaries you get issues especially that blues have a tendency to turn purple. I've scratched my head a good while over that, and I still can't provide an exact intuitive explanation what properties in the Prophoto that causes this, but on a high level I guess you can say that the extreme "virtual blue" primary (the coordinate is outside human locus) makes the balance between reds and blue a bit unstable, so the curves get an exaggerated effect and thus the hue shift is enlarged.
In short, for an RGB curve, Prophoto is not a good color space to apply it in. What I've done is to add a special "curve color space", which has the red and green primary at its original xy coordinates but I've moved in the blue to the locus. I need an as large color space as possible otherwise the curves won't render high saturation colors properly (as it won't reach them). With the rebalanced modified prophoto the red/blue balance is alright again and the hue-shift problem is gone, or rather, is back to what's normal for an RGB curve. I get a new problem to handle though, that colors may be outside the gamut of this slightly smaller color space. I could just clip them, but that would kill some tonality. Instead I make an auto-extension of the curves below 0.0 and above 1.0. The effect of this is a slight gamut compressing effect of (extreme saturation) reds. I was puzzled at first that when moving the blue primary it was the red channel that was affected, but that is because although the red primary is at the same xy coordinate it's brightness is reduced to balance out with the new blue primary to keep the whitepoint where it should be (D50).
About the curve color space, it's also something that has obviously been thought about in C1 which use a plain RGB curve for the user-selectable curve and it's important for them to minimize hue shifts. I'm not sure if they use the same color space for all cameras, I would rather guess that it varies a little between model to model. In any case it's a large color space, but smaller than ProPhoto, as it seems you can't make a space of that size and keep sane balance between the primaries. Other techniques they use to minimize hue shift is to use the split curve approach (the main shadow dip is in the ICC LUT as a luminance-like curve), gamut-compression to minimize/avoid unstable high saturation colors, and their general warmup of colors (adding in green) does stabilize things a bit. DCamProf profiles are not fully as stable as I've chosen other tradeoffs, preferring more accurate hues and have a bit less aggressive gamut compression.
Adobe on the other hand specifies in the DNG specification that the profile-embedded tone-curve must be applied with Prophoto primaries, which would be really bad if the curve was an RGB curve. The DNG spec doesn't actually specify curve type, but the defacto standard via their own software and their published SDK source code is their own type of Hue-corrected RGB curve and you then don't have the hue shift problem anyway.
-
And here's an example crop, using Bart's image of flowers.
The "original" is actually not original but using the hue-preserving ACR tone curve, to have the similar brightness/saturation to make it easier to compare.
As you can see in the prophoto color space the RGB curve makes a clear hue shift to purple, even if we in this case have removed the shadow dip of the curve (reducing hue shift issues). The modified color space has much less hue shift. It's not zero though, and it shouldn't be as slighter hue shift contributes to the look of this curve type that some like. Its strong aspects are artificial hue separation, that some mistake for being more "accurate", and extremely robust and smooth transition into clipping (great for sunsets, high key portraits etc). For the record the neutral tone reproduction operator has borrowed some clipping behavior from the RGB curve, but not the hue spreading part (so far at least)...
The question "what is the optimal color space to apply a camera profile RGB tone curve?" probably doesn't have a straight answer. If someone think it has, I'd love to hear the discussion. Something "big and balanced", but the exact placement of primaries is not too important is my conclusion so far.
-
The question "what is the optimal color space to apply a camera profile RGB tone curve?" probably doesn't have a straight answer.
Just a thought; When I've been capturing and profiling large color gamut subject matter I've often exported into Joseph Holmes's Dcam 5 or 4 profile for a workspace profile as their chroma variants allow for +/- perceptual colour scaling before final output profiling without messing with tone curve.
Alex
-
@torger I'm intrigued as to how you are going to design a GUI interface to wrangle all this tractably. ;D
-
The question "what is the optimal color space to apply a camera profile RGB tone curve?" probably doesn't have a straight answer. If someone think it has, I'd love to hear the discussion. Something "big and balanced", but the exact placement of primaries is not too important is my conclusion so far.
Does it have to be RGB? Could you not use Bruce Lindbloom Up Lab (http://www.brucelindbloom.com/index.html?UPLab.html)?
-
Does it have to be RGB? Could you not use Bruce Lindbloom Up Lab (http://www.brucelindbloom.com/index.html?UPLab.html)?
It has to be RGB when you apply an "RGB curve", that is apply the same contrast curve separately on the R, G and B channels.
That "tone operator" has special look properties. It shifts saturated hues as it won't scale R G and B with the same amount. For example if you have R at 60%, green at 40% and blue at 80% before the S-curve, you will get a lower G value, a higher R value and a somewhat higher B value, a new channel mix which results in a more purple hue than the original. This effect can be exaggerated depending on how the colorspace is configured, as demonstrated in the crops above.
Hue shifts is not the big thing though, what it also does is that it separates the channels more, ie introduces a saturation increase. As we know in human visual perception contrast and saturation is connected, so if you increase contrast also saturation must be increased otherwise the result looks desaturated. That is the super-simple RGB curve does the basics of what any tone operator should do, and in the early days of digital photo editing RGB curve was the thing so it's a look we've got used to. It also have the nice property of super-robust clipping as it rolls off each channel separately. It does mean that bright reds become orange before clipping etc, but it becomes a really smooth transition.
An RGB curve does over-saturate things though, but as many prefer an over-saturated look it's not always seen as a problem. However by combining it with a luminance curve for the shadow dip you can soften the hue shift and over-saturation effects and get a more neutral curve.
When Adobe made the DCP they came up with their own RGB curve variant. The basically do the same thing regarding saturation increase, but correct the HSV-Hue so it stays the same, ie no hue shift (within the limits of the perceptual accuracy of HSV-Hue which is pretty good but not 100% perfect). I'm guessing a bit here but it seems their early attempts of profiles is simply to make a colorimetric base profile shooting some target and slapping on this curve. It makes an okay profile with no/little hue shift problems, but it's certainly not a full take on how to render color. That ACR curve results in grayish blue skies for example, as mathematically keeping the HSV-Hue correct only allows for strong desaturation to clipping. A pure RGB curve will instead twist the hue towards cyan of those skies and keep more saturation.
UP Lab, similar to Lab2000HL, is a good color space for gamut compression as you can scale chroma (saturation) without shifting hues or changing lightness. CIECAM02 JCh is quite good at that too. However applying a curve in that space it would mean to apply it to the L channel, the lightness channel. This works with the same principle as a "luminance curve" that is the end result is that you scale R G and B with an equal amount, which means no hue shift and no saturation change. This is a different tone operator than an RGB curve, with different properties. If you don't want hue shift, you get that. It has a problem though, and that is that it doesn't modify saturation. Intuitively that might seem as a good thing, but as human color perception has contrast and saturation connected increased contrast with kept saturation with perceptually be perceived as increased contrast with reduced saturation.
Unfortunately it's not as easy to just add some saturation to the luminance curve and you're fine, as the perception is not really linear. This is what all the work put into DCamProf's neutral tone reproduction operator is about.
If you ask me one of the few simple tone operators that has decent performance and look is the RGB-Luminance combination variant. I'm sure there are fans of the RGB and ACR curves too though.
-
Just a thought; When I've been capturing and profiling large color gamut subject matter I've often exported into Joseph Holmes's Dcam 5 or 4 profile for a workspace profile as their chroma variants allow for +/- perceptual colour scaling before final output profiling without messing with tone curve.
Alex
In theory you could have a linear output from camera, reduce to your desired output space (gamut compression etc), and then apply the tone curve in the final output space. It's probably a good approach, but not for "regular Joe User". A general-purpose profile must be designed to work as raw converters expect them to work, and they expect that the curve is applied directly on camera output, that is before we know what the user's output space will be. It's the same with gamut compression, we have to apply that before we know the output space. It's surely non-optimal, but the way raw converters are designed today it's the task of the camera profile. If you're an advanced user and have the tools you can change the workflow though.
It would be problematic to apply it on the output space too though, as it would slightly change the look depending on output space, while if you apply it on a fixed space and then use ordinary gamut mapping/clipping to the output space the look is more uniform with different output spaces.
Why not apply it directly on the camera's "raw" color space? I haven't actually tested that yet, but I think it's a bad idea as depending on how the camera raw to rgb matrix becomes (decided by the target matching) I guess we can get the same "un-balanced" issue as with the ProPhoto, and some cameras would have issues with hue shift and others not etc, so it's probably better to have a fixed space, or at least some space with certain fixed properties. I haven't fully figured out what primary positions that work or not regarding hue shifts, more than I have spaces that work and those that don't.
-
Here's a few examples of how much (or little) difference tone operators do.
I've used the Hasselblad X1D Raw Sample http://www.hasselblad.com/inspiration/gallery/sample-image-downloads by Chris Crooze as a demo. I don't really have a colorchecker shot from an X1D so I had to use some less-than-optimal-lit H5D-50c CC24 to make a profile. So I can't really guarantee the accuracy of the profile, but that's anyhow not the point: we can pretend that the middle shot represents the colorimetric truth. It's linear curve -- that is no tone operator active, and pushed one stop to be easier to compare to the others.
The others all have the exact same contrast curve. From left to right:
ACR - Adobe Camera Raw's HSV-Hue-corrected RGB curve
NTRO - DCamProf's neutral tone reproduction operator
linear - the linear reference
RGB-Lum - RGB curve with luminance for the shadow dip
RGB - pure RGB curve.
Actually side-by-side comparison is not that good, when evaluating for real I never do that. A/B swapping with a few seconds adaptation is the the way to do it. Here side by side the eye don't really get a chance to adapt properly so it sees "the mean" of the all, so the NTRO and RGB-Lum may look a bit under-saturated, while more on target with A/B swap. The difference are also much easier seen with A/B swaps.
The intention of the NTRO is to show perceptually the same image as linear, contrast being the only difference. I'm pleased with the performance. Should one point out some error I think the lipstick is probably a bit too dark.
RGB/RGB-Lum shifts skintone hue towards yellow, and clearly desaturates highlights. The cheeks on the portrait may be seen as more "lively" due to the variations in color-shift (more neutral in darker areas, and more shifted and desaturated in lighter) which is lacking in ACR and NTRO. I was a bit surprised of the quite large color shift, and actually checked if C1 has the same color shift in portraits -- and it has. But most likely they have pre-modified the hue of the linear profile so it ends up where they want it when the RGB curve is applied (C1 shows yellow skin anyway, but I think it's a design choice).
While color-shift introduces more "color separation" in some aspects of the skin, it instead reduces color separation in the yellow rose, as the weakly orange tones in the center of the petals are shifted towards yellow, the same tone as the sides of the petals, while this hue is kept in ACR and NTRO.
-
For my use the NTRO curve would be a great help in speeding things up in that as I'm often having to start from a more linear base upon which I am adding a luminance curve and then finally chroma. Is the
NTRO curve tweakable?
Will this be available in the GUI version of the program?
Alex
Sent from my iPhone using Tapatalk
-
Don't know why all your versions are much lighter than what I got in CS5 with embedded profile. I downloaded the file and opened in ACR 6.7 and it didn't let me choose different profiles like other Raw files I've downloaded from other camera manufacturer sites that are compatible with CS5.
I got the model's forehead to read (L*68 a 20, b 20) switching to Linear curve, +1.0 Exposure, +10 Fill, +3 Black, +10 Brightness, +40 Contrast, +10 Clarity, -5 Saturation. Decent results.
Maybe there's a display calibration discrepancy where everyone's gamma curve is slightly more or less contrasty. Not sure. This is why I've attached a screengrab of my ACR edits from Photoshop preview.
-
For my use the NTRO curve would be a great help in speeding things up in that as I'm often having to start from a more linear base upon which I am adding a luminance curve and then finally chroma. Is the
NTRO curve tweakable?
Will this be available in the GUI version of the program?
I was using the GUI to make the examples above. Not sure how I'm going to do with the RGB-Lum yet. Designing a profile in DCamProf can be seen as a multi-stage serial process, 1) make a colorimetric base profile, with possible "subjective" adjustments regarding the unavoidable errors, eg prefer having skintones a little bit too warm rather than too cool etc, 2) choose gamut compression (possibly with tuned parameters), 3) apply a curve via a tone operator, normally the NTRO, possible with tuned parameters 4) apply subjective color adjustments.
That is the major subjective part is left until the last stage, and to make this logical it requires that the tone operator doesn't distort too much. If you however use something like the RGB-Lum it's probably wiser to make subjective adjustments *before* the curve is applied. The reason being that it's difficult to cancel out the color shifts cause by RGB-Lum afterwards, so it's better "pre-distort" colors so they end up where you want them. While you can do this with DCamProf and the GUI it's not really designed for that order of workflow so it's a bit awkward.
So I'm thinking of making it possible to blend in some RGB-Lum elements in the NTRO as a part of tuning that or some other way. I'll probably keep in the RGB/ACR/RGB-Lum/RGB operators as plain choices just for reference anyway (good to compare to, to get a feeling of the effect of tone operators), but currently the only way to make a "high end" profile in a smooth and easy way is to use the NTRO.
I'll think some more about it and make some more experiments and see where it ends up.
The tone curve is tunable, I've put quite some effort into the curve editor so it should be effective in fine-tuning a tone-curve. The NTRO has a bunch of parameters in addition to the tone curve, all are tunable via JSON configuration like in command line but in the GUI the only tunable factor I've exposed is the overall saturation factor, which normally is left unchanged at its automatic value. I think this will be enough for the first version.
-
Don't know why all your versions are much lighter than what I got in CS5 with embedded profile. I downloaded the file and opened in ACR 6.7 and it didn't let me choose different profiles like other Raw files I've downloaded from other camera manufacturer sites that are compatible with CS5.
I got the model's forehead to read (L*68 a 20, b 20) switching to Linear curve, +1.0 Exposure, +10 Fill, +3 Black, +10 Brightness, +40 Contrast, +10 Clarity, -5 Saturation. Decent results.
Maybe there's a display calibration discrepancy where everyone's gamma curve is slightly more or less contrasty. Not sure. This is why I've attached a screengrab of my ACR edits from Photoshop preview.
When it comes to Hasselblad cameras and ACR there is no profiles provided, that's why you can't choose different profiles. I don't know if this is because they have integrated Hassy's native color model, or if they only do the basic embedded color matrix. It's a mystery on its own as Hasselblad has claimed excellent Lightroom compatibility etc, but if that doesn't come with high end color profiles it wouldn't be true.
The difference in brightness is because I've pushed the exposure, forgot to mention that, and the reason for that is because I think the original image is a bit under-exposed. With portraits and skintones profiles are generally (indirectly) optimized to provide the best results at a certain exposure. Much of the tunings are non-linear and as such won't work if the face isn't put at the expected range of the tone curve. The differences between different profiles also turn out more if the portrait is somewhat high key, as the rendering of skintone highlights is very telling.
More "accurate" profiles often show problems with skintone highlights, they simply keep too much of the original hue so the result looks a bit flat and muddy, while classic RGB-style operators will shift and desaturate a bit making the skin looking more fresh. Some of this has already been taken into account with the NTRO, which in its first incarnations had some of those issues. I may even integrate more of this effect, it's a work in progress.
After a contrast curve has been applied color value readings doesn't mean much. Basically the only thing you can trust is hue (as our hue perception is approximately constant with contrast), but then you need to take hue readings in a color space which has straight hue lines, UP-Lab, Lab2000HL, CIECAM02 JCh, and actually RGB-HSV is not too bad except perhaps in the blue range. Our perception of saturation changes with contrast so saturation readings become meaningless, and lightness is part of the contrast of course. The rolloff range into the white-point is also longer than one may think, the desaturation process related to that starts quite early. So evaluating result in the end is about using our eyes.
-
Here's an image to more clearly point out what I mean with what one could see as "livelier" skin-tones and "desirable hue shifts". I'm not saying that it's necessarily better, I personally prefer the NTRO rendering, but there may/might be elements to borrow from RGB-Lum.
I've marked the left cheek with a circle and split it in two. It looks like a "forbidden sign" I know, but it's entirely unintentional :-). The lower-left half the RGB-Lum is quite hue accurate, while in the brighter upper-right half there's some hue shift and stronger desaturation effect, while NTRO maintains correct hue better. In other words the RGB-Lum introduces some "artificial color separation" and gives a slightly more contrasty look despite having the same tone curve. (They're not ideally comparable as the global saturation of the RGB-Lum is a little bit lower than NTRO).
-
So evaluating result in the end is about using our eyes.
I added one full stop exposure in ACR on that sample as well but I could've gone farther but stopped to avoid blowing out the spectral highlights in the buttons on the dress which you've cropped out of your sample.
All I'm concerned about by going through the trouble of making a custom profile is maintaining consistency when increasing exposure or brightening an image but at the same time preserve modeling detail in such surfaces as fabric and non-spectral skin highlights in the area of 200RGB. Of course the only way I've been able to achieve this in ACR is by using PV2010's linear styled function engineered in its Exposure slider after zeroing out all other sliders in the Basic Panel and adjusting for a neutral or slightly off white white balance.
Your attempt at building DcamProf profiles to control skin highlight hues nearing max brightness seems to leave out or avoid mitigating against the hue shifts from spectral fluorescing of foundation makeup which this sample image has at near pancake levels making this sample image not ideal IMO for testing the effects of various contrast curves on hue/saturation shifts.
-
Here's an image to more clearly point out what I mean with what one could see as "livelier" skin-tones and "desirable hue shifts". I'm not saying that it's necessarily better, I personally prefer the NTRO rendering, but there may/might be elements to borrow from RGB-Lum.
I've marked the left cheek with a circle and split it in two. It looks like a "forbidden sign" I know, but it's entirely unintentional :-). The lower-left half the RGB-Lum is quite hue accurate, while in the brighter upper-right half there's some hue shift and stronger desaturation effect, while NTRO maintains correct hue better. In other words the RGB-Lum introduces some "artificial color separation" and gives a slightly more contrasty look despite having the same tone curve. (They're not ideally comparable as the global saturation of the RGB-Lum is a little bit lower than NTRO).
I don't really see any issues with hue/saturation shifting between all three. I'm seeing a problem with NTRO and RGB-Lum curve shape that abruptly renders tone coming out of the eyelash and iris area of the models face that doesn't look right.
It's a look I've come to recognize just using ACR's default profile (which is not the cause) combined with Medium Tone Curve where it makes dark small detail in areas of eyes and tree foliage near black seem to appear like holes of pitch black due to there not being a more perceptually smooth and even grade out of these areas.
The linear version in the middle appears to need only a small increase of local contrast provided by a broad and gentle S-curve combined with a bit of Clarity.
-
I don't really see any issues with hue/saturation shifting between all three. I'm seeing a problem with NTRO and RGB-Lum curve shape that abruptly renders tone coming out of the eyelash and iris area of the models face that doesn't look right.
I should probably have used a softer tone curve, and indeed it's common to do that with portraits. This one has a quite strong shadow dip, that's what's making eyes and the lips a little bit too dark in the NTRO too. It's not a default Adobe curve, it's one of my own, not adapted for portraits though.
In most cases with most profiles ACR applies automatic black subtraction, which increases contrast and makes dark stuff go close to black or even clip to black. C1 also seems to apply automatic black subtraction, and in-camera JPEGs generally also do, so it's typical. The strength of it varies between converters though. In these example images no black subtraction has been done, but instead a curve with a stronger shadow dip has been used that emulates the effect of a typical black subtraction -- which as you point out is actually too strong for this portrait.
-
Even though the curve is a bit too strong for portraits, I quite like the NTRO and RGB-Lum results.
On a totally different note, I'd be interested in the ability to apply a 3D Lut "look" to a colorimetric base profile, and save the resulting profile so it could be used in C1 for instance.
Designing/editing LUTs with 3DLut Creator or similar programs is fairly easy these days. Being able to incorporate those directly into a profile would be really great AFAIC.
ps : I *do* know you're not into "look", Anders, but NTRO is already one, sort of ;)
-
In most cases with most profiles ACR applies automatic black subtraction, which increases contrast and makes dark stuff go close to black or even clip to black. C1 also seems to apply automatic black subtraction, and in-camera JPEGs generally also do, so it's typical. The strength of it varies between converters though. In these example images no black subtraction has been done, but instead a curve with a stronger shadow dip has been used that emulates the effect of a typical black subtraction -- which as you point out is actually too strong for this portrait.
I totally agree with you on the black subtraction of Raw converters in general. But from editing over 1000 Raws and sensing a pattern of tonal distribution characteristics in the shadow portion of the base tone curve to normalize a dark linear rendering after demosaicing that suggests the wonky shaped portion of this base tone curve is the real culprit in this. It's compounded by under exposures for preserving highlights in high contrast scenes like sunset lit clouds while attempting to lift shadows in foreground objects like tree foliage or rocks.
The blonde model sample image had her eyeliner reading 000RGB black before I noticed ACR applied +5 on the Black slider. What's particularly interesting about this issue is how much this affects overall contrast and thus this perceptually smooth gradation out of black.
That's why I set my ACR defaults zeroing out the Black slider and use the Parametric curve settings shown in the screengrab below which points to this nonlinear gradation behavior of tone coming out of black (that the base tone curve should fix but doesn't) that prevents or fights my applying a perceptually smooth gradation and leaving me with what appears as black dropout holes in the image even when they read above 555RGB black.
-
Well this is hard. I don't much like any of the portrait shots, at least as rendered through a web browser. They look more they seem too contrasty and saturated (other than the linear one, which looks slightly corpse-like, perhaps because the colour balance is off).
For landscape / cityscape shots, a bit of excess colour saturation and lots of colour separation are what is needed.
This is interesting research, Anders. Makes you wonder why the likes of dxo choose to use the Adobe RGB space internally.
Makes you wonder whether it would not be easier to make the aim to be to produce an image that does not aim to be particularly pleasing but one that maximises the opportunities for customised enhancement. This is probably what your neutral rendition aims to do, or you could argue that the linear version does this.
As things stand, trying to tune the rendition by producing a bespoke camera profile may just be too difficult for users without your understanding of colour theory.
-
Even though the curve is a bit too strong for portraits, I quite like the NTRO and RGB-Lum results.
On a totally different note, I'd be interested in the ability to apply a 3D Lut "look" to a colorimetric base profile, and save the resulting profile so it could be used in C1 for instance.
Designing/editing LUTs with 3DLut Creator or similar programs is fairly easy these days. Being able to incorporate those directly into a profile would be really great AFAIC.
ps : I *do* know you're not into "look", Anders, but NTRO is already one, sort of ;)
You will be able to design looks and apply to a colorimetric base profile in the GUI, I do like the concept of subjective adjustments, although I don't aim for very strong looks though so the GUI is not well suited to design any sort of look, more like fine-tuning the base result out of the tone operator. I don't have any integration by third-party LUT formats yet though like from 3D Lut creator, and it probably won't be in the first version, but it sounds like an interesting idea. It would probably be quite easy to implement. I think I'll have a look. The more stuff I put in there, the more I delay a release though... :-)
-
I came up with an answer to "what's the perfect RGB color space for the RGB curve?". Why not take the purest red possible with the purest blue and green, and move it all the way to the Locus and make a color space out of that? The idea is that the color space should be as balanced as possible, so when you for example apply an adjustment to the green channel it doesn't skew the red/blue balance.
The problem is how do we define the purest red/green/blue? Well, we need to look into some color appearance model, and CIECAM02 is pretty good. Just look at which angles you can get the highest saturation out of red/green/blue. In CIECAM02 you get that red at ~27.0 degrees, green at ~147.2 degrees and blue at ~248.1. Interestingly enough this color space ends up pretty close to the manually adjusted prophoto space I used in previous experiments. Reds get rendered a bit darker though, so you get a bit more contrast in the portrait example.
-
Well this is hard. I don't much like any of the portrait shots, at least as rendered through a web browser. They look more they seem too contrasty and saturated (other than the linear one, which looks slightly corpse-like, perhaps because the colour balance is off).
For landscape / cityscape shots, a bit of excess colour saturation and lots of colour separation are what is needed.
This is interesting research, Anders. Makes you wonder why the likes of dxo choose to use the Adobe RGB space internally.
Makes you wonder whether it would not be easier to make the aim to be to produce an image that does not aim to be particularly pleasing but one that maximises the opportunities for customised enhancement. This is probably what your neutral rendition aims to do, or you could argue that the linear version does this.
As things stand, trying to tune the rendition by producing a bespoke camera profile may just be too difficult for users without your understanding of colour theory.
Skintones is much more than just the profile. It's the makeup, the lighting, the model, the white balance, and the colorimetric precision of the profile which as said is not a part of that series of experiments as I didn't really have a quality X1D target shot, just a less-than-optimal H5D-50c shot (but usually it doesn't make that big a difference, it's quite robust). I compared to Hassy's JPEG to check that it was not too off.
What I look at in this experiment is the effect of the tone operators, how they change the look in the linear reference shot. I put little value into if the reference shot looks good or not. I don't it's a task of the profile to change a real scene you don't like into a picture you like, and even quite heavily subjective profiles doesn't do that either, they mostly just enhance what's there.
I've noted that Adobe does what you suggest for some camera profiles, that is make something that's not really much of a finished look, but more like a starting-point for creative post-processing. If I remember correctly the A7r-II and Pentax 645z stock profiles are like that, the stock profiles produce an undersaturated look, not particularly pleasing nor realistic, but low saturation profiles are more stable and can as such be better if you make heavy post-processing anyway.
Indeed the NTRO can be used for this too. It's agnostic about the tone curve, and a post-process-friendly profile would have quite low contrast, little shadow dip, and if DCP you'd disable black subtraction which you can do with a profile flag. It's also easy to control the overall saturation with the chroma scaling parameter which I've exposed in the GUI. With weaker contrast the NTRO result will be increasingly close to RGB-Lum to finally end up the same as linear.
I'm not too worried that it would be too difficult to make a bespoke profile. Or rather, I don't think the majority will want to do that. The result using defaults is intended to be really good, and I think it is. The typical purpose of using DCamProf is if you just want a profile that produces a neutral realistic but still attractive result, and this is what the default will provide.
But then there are those special users that want to tune stuff. You still don't need to know particulary much color theory to do so, it won't help you much. What will help you is a good monitor in good viewing conditions and a good eye for color. That last one is the most difficult. Most can say which one of A and B they prefer, but few can say why they do it, and even fewer how to modify the color to get it there. It's difficult to me too, so it's generally a bit of trial-and-error involved. The problem won't be knowledge of color science, but knowing what one wants, and having the eyes to tune it.
The discussions here become quite technical, as it's about the implementation. In the end I just put in the curve color space into the software and you don't get to choose it, as for the moment I don't see any reason to make it configurable. It may be that in the command line software though, which will allow for more experimentation than the GUI. In the GUI I try shave off things that is more a curiousity than important to configure for bespoke profiles.
That DxO is using AdobeRGB as working space I guess is just a design choice. If you indeed know that no user will want to have more saturated colors than AdobeRGB provides, then setting that as a fixed working space and compressing colors to that will simplify things. I'd think it's a bit small though, and with video standards becoming increasingly popular AdobeRGB may fall out of fashion in favour of colorspaces like Rec2020. So for the moment I prefer to provide an oversized colorspace with various gamut compression options.
-
Hi,
That lady portrait is quite horrible, IMHO.
Tim Ashely shared some raw images which may or may not be usable, he even included a ColorChecker shot.
http://www.getdpi.com/forum/medium-format-systems-and-digital-backs/61114-someone-having-decent-raw-x1d-share.html#post722738
Keep on mind that Tim just shot some raw files to share, most kind of him. Just some honest snapshots, I would guess.
Best regards
Erik
-
Raw files of professionally made portraits with models with makeup are pretty hard to come by, especially if you want a colorchecker shot with it. I'm not a professional portrait photographer myself so I don't have any of my own. It's a popular reference subject for "skintones", although everyday people without makeup is probably just as important (I have more of those images...).
And somehow people always complain about the quality of the portraits :-), I guess most have a very narrow taste when it comes to how a portrait may look. I think one of the reasons is that "great portraits" we are used to see really are quite heavily post-processed.
To me it's just a standard lit 1A portrait, and as such is useful to demonstrate several aspects of skintones. The hue of the skin is not that important for evaluation, luckily as humans comes in all sorts of colors, however the brightness is. So it would be great with some examples of dark skin people too, which is even harder to come by. I've managed to find some IQ280 shots though, but I don't have an IQ180-IQ380 colorchecker shot, and the CFA of the 80MP CCD is not the same as the 60MP.
Back in the days films were quite "racist", was modeled after fair caucasian skin and rendered dark skin almost pitch black. This has fortunately changed since then. I can see in some camera models that the dark skintone patch is rendered lighter than it is with a linear matrix match, so the color filters in the camera can have compensation too. The contrast curves of today's digital are also less extreme than yesterday's color photography so it's less of a problem.
-
You will be able to design looks and apply to a colorimetric base profile in the GUI, I do like the concept of subjective adjustments, although I don't aim for very strong looks though so the GUI is not well suited to design any sort of look, more like fine-tuning the base result out of the tone operator. I don't have any integration by third-party LUT formats yet though like from 3D Lut creator, and it probably won't be in the first version, but it sounds like an interesting idea. It would probably be quite easy to implement. I think I'll have a look. The more stuff I put in there, the more I delay a release though... :-)
Thanks for considering it :)
I'm not using LUTs for strong effects either, more like adjustments making my post-processing easier and faster. Desaturating close to neutrals except in the highlights, warming up shadows or desaturating dark blues, or fixing a specific color issue on a set of interior pictures are really easy to perform with LUTs.
I know the same can be achieved with dcamprof, but for now the instant visual feedback in 3D Lut Creator lets me get to the desired adjustments much quicker.
-
Thanks for considering it :)
I'm not using LUTs for strong effects either, more like adjustments making my post-processing easier and faster. Desaturating close to neutrals except in the highlights, warming up shadows or desaturating dark blues, or fixing a specific color issue on a set of interior pictures are really easy to perform with LUTs.
I know the same can be achieved with dcamprof, but for now the instant visual feedback in 3D Lut Creator lets me get to the desired adjustments much quicker.
In the GUI version there's a tool that is similar to the color editor in Capture One, with realtime(ish) update. Those changes you mention above I think will be quite effective to do with that. At least that's the intention :)
Regarding development status I'm now in a phase where I'm not implementing new features but stabilizing and testing. I would think it's ~2 months until a release, but it can change depending on how busy I get with other projects and if I can't avoid adding some additional "must have" feature... At some point it's better to get something out than trying to have every aspect perfect though, but it needs to be more worked through than a first release of an open-source project as there will be paying customers.
-
By the way anyone who know of public data for
1) X-Rite Coler-checker passport, preferably including spectral data
2) Datacolor SpyderCheckr, preferably including spectral data
...I'd like to include it in future revisions of DCamProf and the GUI.
-
By the way anyone who know of public data for
1) X-Rite Coler-checker passport, preferably including spectral data
2) Datacolor SpyderCheckr, preferably including spectral data
...I'd like to include it in future revisions of DCamProf and the GUI.
Hi Anders,
You probably already know of Babelcolor's ColorChecker pages (http://babelcolor.com/colorchecker.htm) with on page 2 links to (spectral) data from the various ColorChecker cards.
Cheers,
Bart
-
Hi Anders,
You probably already know of Babelcolor's ColorChecker pages (http://babelcolor.com/colorchecker.htm) with on page 2 links to (spectral) data from the various ColorChecker cards.
Cheers,
Bart
I already have good data with spectrum for before and after 2014 X-Rite Colorchecker. I don't have spectrum for all patches on a colorchecker passport though, and not for SpyderCheckr. I'd also need a source where it's okay to redistribute with both free (DCamProf) and commercial (upcoming Lumariver Profile Designer, ie DCamProf GUI).
SpydrCheckr without spectra is here http://spyder.datacolor.com/wp-content/uploads/datacolorproductliterature/SpyderCheckr%20Color%20Data.pdf seems to be Lab values with D65 whitepoint.
-
2) Datacolor SpyderCheckr, preferably including spectral data
I am not sure why 'd u bother with DataColor target - the quality of patches (how they are painted) are subpar vs X-Rite products... however, feel free to use w/o any strings attached whatsoever the following, this is from a sample DataColor SpyderChecKR 24.... spectrolino was re-calibrated 3 times before each row (6 patches) reading started, instrument was covered by black cloth for each calibration / patch reading to prevent any foregin light leaks :
CGATS.17
DESCRIPTOR "DataColor SpyderCheckr 24 (3810-0337, 22964416-C19D2A-BFC48C-4B48) Spectral Measurement, 1 pass, 380nm : 10.0nm : 730nm, with Spectronio, S/N 17984, Part No: 36.55.52, P/D 9/8/2004, firmware 1.68"
ORIGINATOR "BabelColor PatchTool, version 5.1.0 b409 with the data from: spotread.exe -v -A N -s"
LGOROWLENGTH 4
CREATED "2017-03-10" # Time: 17:47:30
INSTRUMENTATION "Spectrolino"
MEASUREMENT_SOURCE "Illumination=D50 ObserverAngle=2 WhiteBase=Abs Filter=No"
ILLUMINATION_NAME "D50"
OBSERVER_ANGLE "2"
FILTER "Unknown"
MEASUREMENT_CONDITION "M0"
WEIGHTING_FUNCTION "ILLUMINANT, D50"
WEIGHTING_FUNCTION "OBSERVER, 2 degree"
KEYWORD "DEVCALSTD"
DEVCALSTD "GMDI"
#
KEYWORD "SAMPLE_ID"
KEYWORD "SAMPLE_NAME"
NUMBER_OF_FIELDS 46
BEGIN_DATA_FORMAT
SAMPLE_ID SAMPLE_NAME XYZ_X XYZ_Y XYZ_Z LAB_L LAB_A LAB_B LAB_C LAB_H nm380 nm390 nm400 nm410 nm420 nm430 nm440 nm450 nm460 nm470 nm480 nm490 nm500 nm510 nm520 nm530 nm540 nm550 nm560 nm570 nm580 nm590 nm600 nm610 nm620 nm630 nm640 nm650 nm660 nm670 nm680 nm690 nm700 nm710 nm720 nm730
END_DATA_FORMAT
NUMBER_OF_SETS 24
BEGIN_DATA
1 1 29.132137 40.170447 31.747831 69.5908 -33.4205 2.1125 33.49 176.4 0.108360 0.162790 0.237760 0.284860 0.300000 0.312580 0.326830 0.347430 0.378260 0.419010 0.463690 0.512960 0.553390 0.571310 0.565800 0.542300 0.507790 0.464690 0.415240 0.369960 0.327610 0.286320 0.246290 0.217540 0.202940 0.197140 0.194730 0.193830 0.196620 0.204270 0.215710 0.228070 0.239280 0.244830 0.243300 0.244680
7 7 37.718350 28.096402 3.949050 59.9757 38.1912 58.3826 69.76 56.8 0.040170 0.041180 0.042230 0.042960 0.043520 0.044300 0.044490 0.046320 0.048060 0.047940 0.047130 0.049340 0.053340 0.055460 0.061350 0.084860 0.140950 0.205470 0.269030 0.331770 0.389850 0.451240 0.503980 0.540800 0.563910 0.581190 0.600080 0.620480 0.640120 0.656840 0.672630 0.693050 0.716680 0.740300 0.759030 0.774690
13 13 12.148603 17.726475 27.377056 49.1627 -30.2142 -26.1011 39.93 220.8 0.089310 0.126010 0.177300 0.214490 0.231750 0.251570 0.279120 0.312010 0.349790 0.392610 0.419400 0.425060 0.407890 0.372370 0.324030 0.269580 0.217450 0.170040 0.130150 0.103910 0.088650 0.079930 0.072870 0.068260 0.066400 0.065660 0.065520 0.066650 0.068340 0.069530 0.069290 0.068370 0.066800 0.066420 0.068600 0.073300
19 19 85.828391 88.920236 71.671770 95.5471 0.1645 1.5065 1.52 83.8 0.136030 0.215920 0.404750 0.667010 0.828640 0.862320 0.868720 0.873250 0.876950 0.880570 0.882780 0.885720 0.886180 0.887270 0.887860 0.886040 0.885830 0.886800 0.883990 0.885620 0.888550 0.893660 0.896350 0.898350 0.897420 0.897500 0.902330 0.909110 0.913790 0.915430 0.918470 0.920440 0.922690 0.926740 0.927970 0.927830
2 2 23.240182 22.223977 32.295751 54.2638 8.3018 -25.1452 26.48 288.3 0.116670 0.182730 0.293110 0.371720 0.392150 0.400020 0.406050 0.410260 0.409270 0.402040 0.384940 0.352020 0.309720 0.272910 0.237650 0.210970 0.201210 0.197280 0.189100 0.183670 0.189190 0.206530 0.224220 0.226320 0.215250 0.211900 0.231510 0.270630 0.317580 0.352450 0.364740 0.365110 0.369490 0.382460 0.398780 0.420200
8 8 11.340654 10.633763 26.367087 38.9567 8.0925 -41.9737 42.75 280.9 0.096860 0.143610 0.218210 0.278440 0.305550 0.326620 0.347960 0.362110 0.352450 0.325800 0.287600 0.236930 0.190870 0.157910 0.130400 0.110780 0.101280 0.093560 0.084210 0.077950 0.076450 0.077970 0.078850 0.077590 0.076230 0.078060 0.084800 0.095110 0.105100 0.108830 0.104620 0.099350 0.098320 0.102050 0.108380 0.116400
14 14 28.374129 18.216185 21.302905 49.7573 49.1331 -13.9694 51.08 344.1 0.113400 0.175100 0.273150 0.334140 0.338840 0.327160 0.304950 0.278700 0.252920 0.228390 0.201940 0.175970 0.157110 0.141130 0.119980 0.100710 0.094560 0.097090 0.095730 0.093720 0.108690 0.154110 0.228270 0.347720 0.492870 0.603010 0.674780 0.722630 0.754390 0.773000 0.782900 0.790310 0.801740 0.815110 0.826920 0.839460
20 20 54.379152 56.361742 45.730464 79.8193 0.0835 0.9321 0.94 84.9 0.133160 0.210030 0.361290 0.501980 0.543230 0.549160 0.554210 0.558090 0.558570 0.557960 0.556470 0.556910 0.557570 0.558860 0.561130 0.562040 0.562890 0.563040 0.562530 0.565330 0.566550 0.568020 0.567700 0.567890 0.566700 0.565140 0.564540 0.564170 0.563030 0.561170 0.559690 0.557750 0.556620 0.555970 0.554550 0.552640
3 3 9.943611 12.193913 5.016540 41.5227 -13.4712 20.5388 24.56 123.3 0.045320 0.051010 0.057820 0.060410 0.057700 0.055540 0.054070 0.054170 0.055650 0.058840 0.065040 0.074700 0.086710 0.101900 0.126310 0.151510 0.161400 0.154460 0.140140 0.127010 0.116730 0.108820 0.101390 0.094500 0.090220 0.088710 0.088790 0.089400 0.091430 0.096350 0.103570 0.112770 0.123210 0.130690 0.132700 0.133960
9 9 28.921080 18.838315 10.135851 50.4975 48.0658 15.2358 50.42 17.6 0.090530 0.117320 0.140950 0.144740 0.139880 0.135080 0.131270 0.126390 0.121650 0.117620 0.112490 0.106560 0.102250 0.098460 0.093290 0.088900 0.088600 0.092770 0.097930 0.104210 0.131800 0.220460 0.354280 0.471970 0.547690 0.586510 0.604860 0.614250 0.618650 0.620200 0.621230 0.621910 0.623370 0.625840 0.625690 0.626770
15 15 60.634117 60.858251 6.709872 82.3025 4.6476 82.8433 82.97 86.8 0.047960 0.047330 0.047520 0.047420 0.047830 0.049180 0.050790 0.053390 0.059980 0.074040 0.099850 0.141440 0.205030 0.289320 0.386320 0.498430 0.613350 0.685990 0.716870 0.737430 0.750430 0.754260 0.743780 0.728260 0.717980 0.715480 0.717400 0.721460 0.730400 0.742110 0.755260 0.770050 0.789280 0.802270 0.804360 0.811750
21 21 33.055217 34.255219 27.747783 65.1646 0.0882 0.8664 0.87 84.2 0.117460 0.176620 0.261850 0.314020 0.324730 0.329780 0.335660 0.339700 0.340050 0.338770 0.337280 0.337120 0.337490 0.339000 0.340580 0.341440 0.341990 0.342400 0.341720 0.343490 0.344750 0.346710 0.346970 0.346170 0.344100 0.342070 0.341290 0.341310 0.340290 0.338250 0.336000 0.333660 0.331930 0.331250 0.330530 0.329840
4 4 16.235624 17.788109 25.465207 49.2381 -5.0997 -22.6701 23.24 257.3 0.109560 0.166520 0.255210 0.319490 0.335510 0.335240 0.330100 0.322690 0.311930 0.298930 0.285030 0.271070 0.256130 0.240630 0.224710 0.207960 0.192020 0.177790 0.165330 0.156130 0.148420 0.142050 0.136510 0.131910 0.128110 0.124850 0.123850 0.124810 0.126070 0.127040 0.127980 0.130530 0.135750 0.143030 0.150390 0.156960
10 10 8.845869 6.743107 11.792594 31.2148 21.9925 -23.1559 31.94 313.5 0.081940 0.110730 0.151250 0.174930 0.185970 0.187880 0.178390 0.161080 0.140410 0.121050 0.102100 0.083530 0.071080 0.063310 0.056540 0.051660 0.050520 0.051350 0.051360 0.052610 0.057860 0.068580 0.082350 0.088910 0.083620 0.081330 0.093860 0.128030 0.196860 0.301450 0.425910 0.549140 0.652630 0.728580 0.778200 0.810430
16 16 21.462040 11.987628 3.645325 41.1964 56.4795 27.9159 63.00 26.3 0.056870 0.054180 0.052120 0.050710 0.048970 0.047870 0.046060 0.044540 0.043240 0.042140 0.041230 0.040780 0.040520 0.040770 0.041250 0.040890 0.040980 0.042630 0.045680 0.049880 0.059060 0.097350 0.178490 0.291860 0.427760 0.563260 0.665630 0.729330 0.765460 0.781360 0.783820 0.787950 0.803080 0.821480 0.835520 0.848580
22 22 17.771757 18.445763 14.983140 50.0324 -0.0767 0.6002 0.61 97.3 0.091670 0.124170 0.158050 0.173440 0.176160 0.178410 0.181780 0.183990 0.183330 0.182040 0.181010 0.180790 0.181110 0.182280 0.183540 0.184350 0.184680 0.184720 0.184350 0.185520 0.186260 0.186830 0.186440 0.185640 0.184510 0.182990 0.181770 0.180890 0.179460 0.178070 0.176700 0.175390 0.174050 0.172950 0.171720 0.170400
5 5 37.374825 33.341760 17.947940 64.4367 17.8494 18.4103 25.64 45.9 0.099590 0.137230 0.178470 0.195930 0.196660 0.196580 0.198150 0.203000 0.212190 0.225140 0.242900 0.263840 0.279550 0.284550 0.282280 0.276880 0.269730 0.265030 0.269900 0.287380 0.324480 0.412150 0.471950 0.494650 0.504600 0.512710 0.525050 0.541930 0.563140 0.589840 0.622940 0.661360 0.704690 0.749020 0.787510 0.819790
11 11 33.943447 42.279697 8.136953 71.0633 -22.2304 57.7133 61.85 111.1 0.058570 0.065560 0.067290 0.067010 0.066970 0.068520 0.069840 0.072750 0.078590 0.089560 0.110750 0.151310 0.224840 0.329830 0.437420 0.509480 0.532570 0.526350 0.508870 0.489770 0.460640 0.418530 0.368300 0.329170 0.307270 0.296900 0.291380 0.288020 0.291070 0.302330 0.319650 0.339650 0.358540 0.367700 0.363880 0.365220
17 17 13.521294 21.292530 6.748263 53.2682 -38.8044 32.6193 50.69 139.9 0.048470 0.051830 0.054490 0.055690 0.056700 0.058090 0.060480 0.064130 0.069630 0.078900 0.096370 0.126540 0.173900 0.229650 0.276480 0.308210 0.323160 0.310470 0.271560 0.226990 0.186150 0.148980 0.113710 0.088290 0.074820 0.068540 0.065020 0.062120 0.061160 0.062240 0.064960 0.069380 0.074070 0.076710 0.075520 0.073910
23 23 7.994945 8.293339 6.702184 34.5865 -0.0165 0.6075 0.61 91.6 0.059900 0.070660 0.079540 0.081330 0.081500 0.081590 0.082260 0.082160 0.081430 0.080500 0.079660 0.079440 0.079360 0.079610 0.080550 0.082460 0.084500 0.084970 0.084040 0.083670 0.083660 0.084170 0.083900 0.083370 0.082540 0.081780 0.081130 0.080610 0.079990 0.079380 0.078590 0.078020 0.077490 0.077090 0.076460 0.076020
6 6 10.613477 9.165177 4.620120 36.3004 14.1898 13.6634 19.70 43.9 0.048200 0.050180 0.050960 0.052390 0.053350 0.054690 0.055710 0.055950 0.056180 0.055820 0.055670 0.055860 0.057230 0.060170 0.066070 0.071680 0.073610 0.074630 0.078090 0.087550 0.101820 0.116760 0.127730 0.134910 0.139890 0.144780 0.151290 0.160270 0.172340 0.188060 0.208480 0.233520 0.263360 0.296390 0.328810 0.358710
12 12 48.668431 41.721694 5.755508 70.6786 24.4841 67.1218 71.45 70.0 0.048580 0.050180 0.052400 0.052760 0.053550 0.055200 0.056950 0.059440 0.062800 0.068610 0.077800 0.091890 0.111450 0.138150 0.179270 0.241840 0.320310 0.392160 0.440820 0.477570 0.528220 0.621370 0.656220 0.660400 0.660750 0.660270 0.663530 0.669180 0.676610 0.684980 0.694280 0.707590 0.725420 0.745380 0.762690 0.778440
18 18 6.221247 5.116482 19.017924 27.0641 14.9189 -48.3693 50.62 287.1 0.056280 0.066370 0.086190 0.110220 0.127910 0.159590 0.222270 0.286380 0.296720 0.277450 0.228350 0.166810 0.112490 0.074270 0.052860 0.042920 0.038890 0.036750 0.035760 0.035340 0.035190 0.035310 0.035390 0.035860 0.036500 0.036840 0.037300 0.037840 0.037990 0.037920 0.037750 0.037950 0.038410 0.040200 0.042460 0.045650
24 24 2.455784 2.551778 2.163518 18.1512 -0.0946 -0.5323 0.54 259.9 0.025560 0.026300 0.026790 0.026620 0.026480 0.026580 0.026340 0.026180 0.026340 0.026060 0.025920 0.025980 0.025870 0.025810 0.025770 0.025630 0.025590 0.025570 0.025460 0.025500 0.025360 0.025340 0.025310 0.025310 0.025220 0.025110 0.025130 0.025250 0.025160 0.025350 0.025070 0.025150 0.025300 0.025270 0.025270 0.025520
END_DATA
this was one pass (not averaged), however here is the consistency vs a 2nd measurement with the same instrument and same protocol
(https://s26.postimg.org/7js8mlill/consistency.jpg)
so shall be OK
-
Thanks! I guess you don't have for the 48 patch version? The 24 will do though. The reason I'm including it is that it's the second most common target after X-Rite's (I think) so people may having one laying around so I thought it would be nice to provide some basic support, despite that it's not the best product.
-
So, spectral files for X-Rite ColorChecker Classic, pre 2014 and new, ColorChecker SG pre 2014 and new, QP-Card 202/203, SpyderCheckr 24, and an IT8.7 Reflective target, plus custom targets (rectangular grids), and free-form grids (for a collection individual patches of whatever in for multi-target setups).
What I'm still sort of missing is ColorChecker Passport. You can use the ColorChecker Classic data of course which should be close (not sure if it's exactly the same though), but it would be nice to support both pages.
Is there a spectral measurement of that available somewhere?
-
So, spectral files for X-Rite ColorChecker Classic, pre 2014 and new, ColorChecker SG pre 2014 and new, QP-Card 202/203, SpyderCheckr 24, and an IT8.7 Reflective target, plus custom targets (rectangular grids), and free-form grids (for a collection individual patches of whatever in for multi-target setups).
What I'm still sort of missing is ColorChecker Passport. You can use the ColorChecker Classic data of course which should be close (not sure if it's exactly the same though), but it would be nice to support both pages.
Is there a spectral measurement of that available somewhere?
I can measure, but my sample of X-Rite Passport is quite old (not in terms of exposure to any light - but when it was made) and even w/ X-Rite you don't know if there were any changes or not ... I mean it is not like X-Rite suddenly volunteered about pre 2014/post 2014 changes for CC24/CCSG a year+ later after the fact, but was rather bugged by customers...
But it is better to ask Iliah Borg if he can contribute his data specifically for DCamProf & whatever (non free) software you develop - as he owns much-much better equipment for reflective spectral measurements (albeit the point of getting so "precise" data he will of course question, I bet)
-
feel free to use w/o any strings attached whatsoever the following, this is from a sample of X-Rite Passport's ColorChecker Page.... i1Pro2 & tile/target were covered by black cloth for each calibration / patch reading to prevent any foregn light leaks, averaged from 3 passess :
CGATS.17
DESCRIPTOR "X-Rite Passport S/N 008784, ColorChecker Page, 3 passes/averaged, 380nm : 10.0nm : 730nm, with i1Pro2"
ORIGINATOR "BabelColor PatchTool, version 5.1.0 b409"
LGOROWLENGTH 4
CREATED "2017-03-13" # Time: 17:43:34
INSTRUMENTATION "i1Pro2, S/N 1044184, with 'Lamp Restore' before each pass"
MEASUREMENT_SOURCE "Illumination=D50 ObserverAngle=2 WhiteBase=Abs Filter=No"
ILLUMINATION_NAME "D50"
OBSERVER_ANGLE "2"
FILTER "Unknown"
MEASUREMENT_CONDITION "M0"
WEIGHTING_FUNCTION "ILLUMINANT, D50"
WEIGHTING_FUNCTION "OBSERVER, 2 degree"
KEYWORD "DEVCALSTD"
DEVCALSTD "XRGA"
#
KEYWORD "SAMPLE_ID"
NUMBER_OF_FIELDS 42
BEGIN_DATA_FORMAT
SAMPLE_ID LAB_L LAB_A LAB_B LAB_C LAB_H nm380 nm390 nm400 nm410 nm420 nm430 nm440 nm450 nm460 nm470 nm480 nm490 nm500 nm510 nm520 nm530 nm540 nm550 nm560 nm570 nm580 nm590 nm600 nm610 nm620 nm630 nm640 nm650 nm660 nm670 nm680 nm690 nm700 nm710 nm720 nm730
END_DATA_FORMAT
NUMBER_OF_SETS 24
BEGIN_DATA
1 39.4235 13.6420 14.7292 20.08 47.2 0.054323 0.061430 0.064497 0.065727 0.065567 0.065013 0.065197 0.065177 0.065393 0.065393 0.065537 0.066423 0.068947 0.074923 0.082027 0.085487 0.086950 0.089727 0.095770 0.107790 0.124053 0.139443 0.147693 0.151227 0.157223 0.168960 0.183893 0.197623 0.206273 0.205387 0.196127 0.187867 0.189767 0.199357 0.212033 0.228843
2 63.0527 34.7398 57.6569 67.31 58.9 0.049753 0.052137 0.054180 0.055090 0.055723 0.056037 0.056460 0.057080 0.057780 0.058587 0.060260 0.063297 0.070773 0.093640 0.134200 0.165327 0.182713 0.205203 0.251853 0.335850 0.443080 0.539497 0.588617 0.598283 0.596017 0.593650 0.592187 0.592827 0.600860 0.617577 0.638103 0.656363 0.671253 0.676397 0.672853 0.676937
3 29.9762 14.9465 -48.9216 51.15 287.0 0.069193 0.107640 0.157050 0.200683 0.231177 0.259480 0.289403 0.309053 0.303377 0.275497 0.231880 0.178920 0.133440 0.099007 0.072517 0.057817 0.051533 0.047143 0.043313 0.041670 0.041753 0.042257 0.042133 0.041780 0.042213 0.044447 0.049563 0.057587 0.065767 0.068577 0.064487 0.060243 0.060690 0.065153 0.073650 0.088133
4 96.9587 -0.8060 4.5231 4.59 100.1 0.140963 0.223867 0.396870 0.625843 0.779280 0.827933 0.846710 0.861647 0.875410 0.884560 0.892443 0.900353 0.906603 0.910987 0.916167 0.920120 0.923780 0.926103 0.925940 0.929890 0.929393 0.930057 0.930150 0.930943 0.931287 0.931010 0.932063 0.933447 0.934860 0.934000 0.933227 0.932100 0.934203 0.935260 0.933933 0.935943
5 65.7260 19.7235 17.1096 26.11 40.9 0.095833 0.131417 0.166043 0.182107 0.187177 0.190870 0.197613 0.209367 0.229883 0.262420 0.300080 0.330810 0.347697 0.344077 0.308813 0.267597 0.255670 0.264887 0.266180 0.279643 0.335680 0.419090 0.488413 0.531463 0.557607 0.577400 0.597673 0.621027 0.648453 0.679750 0.715003 0.752107 0.789967 0.821847 0.845423 0.865250
6 39.9853 11.0436 -45.7504 47.06 283.6 0.096913 0.157520 0.234380 0.293713 0.329570 0.360207 0.389173 0.405880 0.401030 0.375067 0.330897 0.273640 0.219320 0.173837 0.136000 0.111690 0.099373 0.091297 0.083723 0.079493 0.079127 0.080820 0.082017 0.082243 0.082717 0.086690 0.096383 0.111977 0.132720 0.157877 0.186690 0.221323 0.264147 0.313137 0.363160 0.413220
7 55.4273 -38.0011 31.3003 49.23 140.5 0.048910 0.052213 0.054403 0.055773 0.057177 0.059277 0.062553 0.068710 0.080217 0.100960 0.132330 0.175637 0.230340 0.288673 0.327600 0.336603 0.326173 0.304517 0.276440 0.244990 0.205850 0.163597 0.128453 0.108077 0.098607 0.094463 0.092190 0.091297 0.093413 0.099557 0.108523 0.118207 0.126183 0.128337 0.125773 0.127717
8 81.3253 -0.6203 0.8342 1.04 126.6 0.134157 0.206230 0.344040 0.491420 0.559777 0.575283 0.582817 0.588390 0.589317 0.586723 0.584690 0.584717 0.585987 0.587227 0.589113 0.590897 0.592350 0.593100 0.592747 0.595463 0.595030 0.594067 0.591547 0.588960 0.585583 0.581937 0.579027 0.576770 0.574500 0.570610 0.566893 0.562930 0.561450 0.559293 0.555833 0.553993
9 49.8322 -5.2307 -22.0571 22.67 256.7 0.106927 0.168450 0.252323 0.310703 0.327350 0.330337 0.331137 0.328313 0.317943 0.301377 0.287690 0.278183 0.267270 0.252240 0.234203 0.212830 0.196017 0.185733 0.173860 0.157903 0.142777 0.135290 0.134337 0.136460 0.138710 0.140007 0.141467 0.145520 0.151563 0.154083 0.150580 0.143973 0.136170 0.130617 0.133453 0.148037
10 51.6703 49.3776 17.9426 52.54 20.0 0.087873 0.111653 0.131267 0.135180 0.132190 0.129237 0.126890 0.124847 0.121340 0.116183 0.110813 0.105843 0.101017 0.095807 0.091110 0.089123 0.089100 0.090443 0.093807 0.105703 0.155180 0.266640 0.403603 0.511080 0.571587 0.598180 0.608977 0.613383 0.615570 0.615133 0.614510 0.613620 0.615237 0.615847 0.614730 0.615853
11 43.6642 50.8524 28.6515 58.37 29.4 0.048817 0.050373 0.050670 0.051303 0.051123 0.051130 0.051037 0.051477 0.051110 0.050290 0.049530 0.049600 0.051123 0.054767 0.058440 0.059557 0.059093 0.058373 0.059783 0.066563 0.081800 0.124110 0.215983 0.347073 0.474523 0.557940 0.599090 0.618203 0.627877 0.632410 0.635620 0.638077 0.642433 0.645417 0.646507 0.649663
12 66.7963 -0.7783 0.3283 0.84 157.1 0.118367 0.181763 0.269553 0.329993 0.348920 0.355190 0.360980 0.365533 0.365967 0.363447 0.361370 0.361040 0.361840 0.362443 0.363700 0.364927 0.365747 0.365983 0.365403 0.366777 0.366407 0.365580 0.363380 0.360807 0.357657 0.354380 0.351853 0.349650 0.347253 0.343993 0.341083 0.337993 0.336370 0.334110 0.331013 0.329230
13 43.6602 -12.0357 21.5544 24.69 119.2 0.050817 0.055413 0.058593 0.059713 0.060080 0.060710 0.061783 0.063380 0.064983 0.066517 0.068373 0.071223 0.078927 0.104327 0.149523 0.179370 0.181413 0.168417 0.150000 0.135057 0.125940 0.120627 0.115087 0.110603 0.109683 0.112447 0.116567 0.119690 0.121207 0.120543 0.117883 0.115690 0.116877 0.120120 0.123347 0.127450
14 30.8835 22.2694 -19.1960 29.40 319.2 0.085530 0.122440 0.161450 0.179600 0.178753 0.168287 0.153047 0.137183 0.119663 0.102800 0.089117 0.077273 0.067720 0.061837 0.057747 0.053543 0.051273 0.051507 0.052397 0.051853 0.050220 0.051227 0.058453 0.075237 0.100337 0.127737 0.154950 0.184577 0.220123 0.265920 0.323447 0.393627 0.471770 0.548180 0.615437 0.671707
15 82.8863 3.2394 80.8508 80.92 87.7 0.058013 0.057563 0.057840 0.058363 0.058503 0.058897 0.060137 0.063060 0.068797 0.079180 0.099297 0.138950 0.210843 0.325417 0.467833 0.580847 0.642110 0.673813 0.692600 0.710153 0.721650 0.731420 0.738597 0.744937 0.749750 0.753527 0.757800 0.762253 0.766280 0.768240 0.769713 0.770970 0.775250 0.778173 0.779070 0.782380
16 50.8671 -0.1432 0.2247 0.27 122.5 0.093663 0.127157 0.159997 0.177737 0.182533 0.186177 0.190347 0.193360 0.193427 0.191657 0.190140 0.189543 0.189783 0.190017 0.190217 0.190447 0.190863 0.191267 0.191350 0.193053 0.194800 0.196080 0.195133 0.192720 0.189400 0.186590 0.184783 0.183503 0.182140 0.180560 0.179213 0.177757 0.177153 0.175823 0.173900 0.172450
17 55.3222 8.5829 -23.9750 25.46 289.7 0.117973 0.195927 0.314020 0.405847 0.434410 0.435940 0.430130 0.421573 0.405467 0.381897 0.358620 0.333890 0.307827 0.282767 0.257293 0.233167 0.218770 0.212133 0.203467 0.194067 0.189697 0.194540 0.207660 0.224180 0.240873 0.263077 0.294363 0.328430 0.353750 0.357837 0.342757 0.328567 0.331753 0.347387 0.367893 0.395830
18 72.6258 -23.4345 56.4427 61.11 112.5 0.059633 0.061477 0.063667 0.064663 0.065460 0.066780 0.069513 0.075233 0.086960 0.110220 0.148273 0.203270 0.277480 0.375157 0.474773 0.535580 0.555533 0.550843 0.531333 0.505190 0.468790 0.425557 0.381543 0.349807 0.331887 0.322387 0.316323 0.312633 0.314730 0.323927 0.337540 0.350997 0.361670 0.363970 0.359293 0.360257
19 52.8978 50.3977 -12.6869 51.97 345.9 0.109733 0.176390 0.274863 0.342673 0.359917 0.352473 0.333813 0.310857 0.283197 0.253653 0.223543 0.194533 0.172563 0.154227 0.130373 0.109127 0.103440 0.108463 0.110633 0.118227 0.149000 0.206133 0.288440 0.403850 0.539670 0.657307 0.737223 0.784967 0.812487 0.827130 0.835377 0.840670 0.847373 0.851980 0.853513 0.857297
20 35.9982 -0.7639 -0.8198 1.12 227.0 0.069873 0.079067 0.087703 0.090900 0.091413 0.092060 0.093537 0.094260 0.093483 0.092027 0.090870 0.090503 0.090533 0.090723 0.090970 0.091260 0.091267 0.091100 0.090700 0.090830 0.090470 0.089883 0.088950 0.088007 0.086907 0.085710 0.084670 0.083853 0.083110 0.082120 0.081150 0.080367 0.079820 0.079210 0.078373 0.077980
21 70.5440 -33.7237 -0.3952 33.73 180.7 0.109977 0.168633 0.242387 0.293417 0.317233 0.333863 0.353037 0.379777 0.420133 0.470357 0.515297 0.549767 0.568607 0.572230 0.564923 0.547947 0.522427 0.487977 0.446173 0.401323 0.349653 0.295023 0.245403 0.212743 0.196247 0.188550 0.184083 0.181960 0.185473 0.195690 0.209233 0.221930 0.231063 0.232423 0.227380 0.228340
22 72.1878 19.2226 68.6255 71.27 74.4 0.058570 0.061300 0.063413 0.064297 0.064307 0.064727 0.065237 0.066320 0.067257 0.068187 0.069853 0.073333 0.082997 0.118853 0.207383 0.317450 0.397817 0.443900 0.476257 0.515523 0.558520 0.598010 0.624987 0.640790 0.649373 0.654420 0.659510 0.664650 0.669807 0.673400 0.677297 0.681020 0.687170 0.691827 0.694083 0.697963
23 50.4139 -27.8057 -28.3447 39.71 225.5 0.085787 0.131720 0.191280 0.237860 0.262627 0.286277 0.318640 0.353297 0.389400 0.424160 0.440383 0.437983 0.418310 0.382230 0.332400 0.278347 0.228157 0.182147 0.143310 0.116233 0.098960 0.088173 0.080057 0.074707 0.072243 0.071343 0.071223 0.072297 0.074327 0.075587 0.075227 0.073487 0.071517 0.070480 0.072523 0.078830
24 20.5523 -0.0334 -0.6690 0.67 267.1 0.033263 0.033553 0.033197 0.032800 0.032877 0.032780 0.032483 0.032367 0.032230 0.032200 0.031947 0.031820 0.031713 0.031653 0.031443 0.031423 0.031377 0.031357 0.031220 0.031260 0.031113 0.031033 0.030990 0.030990 0.030957 0.030953 0.030967 0.031067 0.031050 0.031057 0.031083 0.030993 0.031067 0.031227 0.031170 0.031177
END_DATA
consistency (worst off measurement vs averaged one) :
(https://s26.postimg.org/vg9m2zb6x/consistency.jpg)
-
and the 2nd page from Passport, done the same way
CGATS.17
DESCRIPTOR "X-Rite Passport S/N 008784, Creative Page, 3 passes/averaged, 380nm : 10.0nm : 730nm, with i1Pro2"
ORIGINATOR "BabelColor PatchTool, version 5.1.0 b409"
LGOROWLENGTH 1
CREATED "2017-03-13" # Time: 19:40:26
INSTRUMENTATION "i1Pro2, S/N 1044184, with 'Lamp Restore' before each pass"
MEASUREMENT_SOURCE "Illumination=D50 ObserverAngle=2 WhiteBase=Abs Filter=No"
ILLUMINATION_NAME "D50"
OBSERVER_ANGLE "2"
FILTER "Unknown"
MEASUREMENT_CONDITION "M0"
WEIGHTING_FUNCTION "ILLUMINANT, D50"
WEIGHTING_FUNCTION "OBSERVER, 2 degree"
KEYWORD "DEVCALSTD"
DEVCALSTD "XRGA"
#
KEYWORD "SAMPLE_ID"
NUMBER_OF_FIELDS 42
BEGIN_DATA_FORMAT
SAMPLE_ID LAB_L LAB_A LAB_B LAB_C LAB_H nm380 nm390 nm400 nm410 nm420 nm430 nm440 nm450 nm460 nm470 nm480 nm490 nm500 nm510 nm520 nm530 nm540 nm550 nm560 nm570 nm580 nm590 nm600 nm610 nm620 nm630 nm640 nm650 nm660 nm670 nm680 nm690 nm700 nm710 nm720 nm730
END_DATA_FORMAT
NUMBER_OF_SETS 26
BEGIN_DATA
1 96.5548 -0.6254 4.1612 4.21 98.5 0.140893 0.225553 0.400790 0.629710 0.780420 0.827160 0.843943 0.857770 0.870197 0.878327 0.885030 0.892020 0.897290 0.901187 0.905963 0.909493 0.912897 0.915233 0.915487 0.919813 0.919753 0.920713 0.920937 0.921637 0.922083 0.921763 0.922767 0.924107 0.925520 0.924523 0.923483 0.922283 0.923963 0.924817 0.923673 0.925643
2 88.7632 -0.6293 0.9117 1.11 124.6 0.138153 0.221877 0.387210 0.583320 0.688640 0.713667 0.722277 0.729750 0.734867 0.736927 0.738467 0.740817 0.742053 0.741683 0.740830 0.739413 0.738887 0.737473 0.733510 0.733700 0.733463 0.734903 0.734360 0.733423 0.733473 0.735103 0.739147 0.743440 0.745827 0.743723 0.741073 0.739050 0.741080 0.743920 0.745370 0.749690
3 81.0773 -0.5659 0.8842 1.05 122.6 0.133160 0.212320 0.354053 0.495790 0.557083 0.570573 0.577293 0.582387 0.583610 0.581687 0.579873 0.580333 0.581670 0.582653 0.584593 0.586517 0.588217 0.588847 0.588153 0.590280 0.589333 0.588330 0.586213 0.584343 0.582133 0.579523 0.577593 0.576377 0.575013 0.572120 0.569307 0.566713 0.566063 0.564730 0.561900 0.561367
4 73.7462 -0.5054 0.7985 0.95 122.3 0.129003 0.199843 0.312950 0.407713 0.441440 0.450047 0.456157 0.460960 0.461527 0.458937 0.456710 0.456827 0.457887 0.459050 0.461120 0.463167 0.464643 0.465407 0.465287 0.467337 0.466950 0.466203 0.464130 0.462050 0.459477 0.456467 0.454120 0.452260 0.450360 0.447200 0.444087 0.441030 0.439897 0.438040 0.435013 0.433627
5 27.6753 -0.8456 -0.1391 0.86 189.3 0.043153 0.048493 0.050943 0.051857 0.052487 0.053103 0.053987 0.054217 0.054147 0.053513 0.053217 0.053240 0.053537 0.053690 0.054100 0.054333 0.054440 0.054283 0.053910 0.053677 0.053183 0.052760 0.052317 0.051920 0.051570 0.051183 0.050873 0.050693 0.050350 0.050063 0.049707 0.049447 0.049433 0.049293 0.049097 0.048990
6 25.5330 -0.7520 -0.5673 0.94 217.0 0.040580 0.043570 0.045467 0.045857 0.046300 0.046877 0.047297 0.047533 0.047323 0.046807 0.046510 0.046423 0.046523 0.046563 0.046683 0.046783 0.046813 0.046557 0.046183 0.046040 0.045570 0.045123 0.044807 0.044543 0.044257 0.043977 0.043787 0.043720 0.043543 0.043330 0.043067 0.042900 0.043013 0.042883 0.042543 0.042583
7 22.2771 -0.3065 -0.6362 0.71 244.3 0.035103 0.036960 0.036780 0.036957 0.036640 0.036987 0.037147 0.037297 0.037197 0.036873 0.036630 0.036607 0.036473 0.036363 0.036313 0.036277 0.036270 0.036163 0.035953 0.035950 0.035733 0.035507 0.035353 0.035253 0.035227 0.035103 0.034970 0.035103 0.035077 0.035020 0.035003 0.034950 0.034993 0.034947 0.035003 0.035030
8 20.2239 0.0599 -0.4889 0.49 277.0 0.028710 0.030307 0.031663 0.031533 0.031547 0.031460 0.031423 0.031330 0.031147 0.030963 0.030863 0.030743 0.030817 0.030667 0.030530 0.030450 0.030480 0.030443 0.030403 0.030460 0.030307 0.030270 0.030283 0.030317 0.030310 0.030387 0.030380 0.030470 0.030600 0.030597 0.030673 0.030573 0.030707 0.030793 0.030783 0.030913
9 80.8678 -5.4030 -6.2711 8.28 229.3 0.125737 0.212553 0.373297 0.546197 0.628037 0.645490 0.651177 0.656050 0.659020 0.658587 0.657203 0.656237 0.653950 0.648917 0.641793 0.631237 0.617293 0.597647 0.572253 0.551583 0.537667 0.530687 0.523400 0.519233 0.523327 0.535300 0.549910 0.561430 0.566267 0.560540 0.549517 0.540280 0.540987 0.547300 0.554517 0.565230
10 80.9682 -2.9907 -2.8847 4.16 224.0 0.128373 0.212130 0.368620 0.531633 0.603577 0.615483 0.617833 0.619973 0.620633 0.619030 0.617050 0.616070 0.614403 0.611140 0.607447 0.602950 0.598113 0.592200 0.584743 0.580480 0.574447 0.568463 0.558913 0.549947 0.545380 0.543273 0.544277 0.549830 0.557403 0.558687 0.553730 0.545010 0.534563 0.526840 0.530740 0.551910
11 80.6218 -0.1689 1.2925 1.30 97.4 0.118163 0.201253 0.348007 0.487813 0.545233 0.557353 0.562510 0.567110 0.570237 0.571017 0.571493 0.572820 0.573993 0.574127 0.574850 0.576023 0.577430 0.578273 0.578030 0.580567 0.580413 0.580700 0.580337 0.580230 0.579737 0.578877 0.578857 0.579383 0.579767 0.578470 0.577323 0.576047 0.577000 0.577503 0.576320 0.577000
12 81.2019 4.8042 0.8064 4.87 9.5 0.125867 0.207260 0.351230 0.488433 0.547203 0.565000 0.578197 0.588250 0.591710 0.589627 0.586403 0.584523 0.582863 0.578857 0.572470 0.567437 0.566213 0.563120 0.554403 0.558877 0.591153 0.625873 0.640160 0.643973 0.643943 0.642840 0.642967 0.643850 0.644067 0.642253 0.640803 0.639340 0.640043 0.640040 0.638387 0.638900
13 80.7173 6.5494 0.6135 6.58 5.4 0.127173 0.207947 0.349687 0.483917 0.540930 0.558237 0.572083 0.582710 0.586010 0.582897 0.578583 0.575737 0.572753 0.566893 0.559743 0.555757 0.553707 0.545927 0.532197 0.535637 0.578737 0.628100 0.649237 0.654977 0.655563 0.654327 0.653600 0.653360 0.653013 0.650897 0.648930 0.647060 0.647447 0.647327 0.645220 0.645590
14 81.2981 -10.5471 -4.5884 11.50 203.5 0.124530 0.209933 0.366283 0.525637 0.596030 0.614210 0.625327 0.636833 0.647897 0.658067 0.666973 0.674473 0.678147 0.676597 0.670213 0.658327 0.642257 0.620413 0.592480 0.567587 0.545037 0.527690 0.509217 0.494310 0.487327 0.485087 0.487057 0.496143 0.508840 0.513320 0.507320 0.495187 0.479303 0.467073 0.473140 0.505050
15 80.7653 -7.4942 -2.7829 7.99 200.4 0.123130 0.206823 0.357753 0.509577 0.577527 0.593943 0.602287 0.610800 0.618520 0.623520 0.626127 0.628187 0.628557 0.626650 0.624080 0.619983 0.614483 0.605733 0.592600 0.579683 0.563243 0.546023 0.526170 0.509730 0.499980 0.494953 0.492777 0.493933 0.498833 0.504060 0.508190 0.509790 0.508950 0.505850 0.505413 0.514730
16 81.1494 -5.4864 -1.4474 5.67 194.8 0.120073 0.203480 0.354580 0.505583 0.572237 0.588510 0.596273 0.603750 0.610570 0.614913 0.617163 0.619310 0.620063 0.618823 0.617547 0.615227 0.611813 0.606267 0.597050 0.588657 0.576640 0.564033 0.548850 0.536123 0.528543 0.524550 0.523080 0.524407 0.528527 0.532457 0.535673 0.537017 0.537013 0.535163 0.534617 0.541990
17 82.0943 -2.7244 -0.0614 2.73 181.3 0.121517 0.204463 0.357043 0.511110 0.579413 0.594537 0.600833 0.607150 0.612300 0.615047 0.616737 0.618923 0.619987 0.619523 0.619010 0.617740 0.616400 0.613863 0.608850 0.605740 0.599673 0.594020 0.586930 0.580867 0.577263 0.575207 0.574807 0.576307 0.578857 0.579453 0.579473 0.578277 0.577530 0.575987 0.575700 0.581363
18 81.2087 -0.1052 1.5835 1.59 93.8 0.117147 0.198437 0.345563 0.488353 0.548757 0.562637 0.568443 0.574167 0.578530 0.579827 0.580763 0.582697 0.583963 0.584163 0.585027 0.585990 0.587157 0.588147 0.588180 0.591060 0.591343 0.592000 0.591860 0.592130 0.592007 0.591490 0.591773 0.592673 0.593410 0.592447 0.591590 0.590737 0.592043 0.592627 0.591817 0.592913
19 51.5034 54.5887 -2.4594 54.64 357.4 0.108013 0.158267 0.220920 0.255583 0.260867 0.254707 0.242003 0.226783 0.209163 0.190540 0.170360 0.150557 0.135547 0.122053 0.104257 0.089443 0.085727 0.089313 0.091203 0.097897 0.127927 0.198047 0.305997 0.435530 0.570467 0.687390 0.771393 0.822300 0.849733 0.862910 0.870203 0.874870 0.881623 0.887117 0.889517 0.894133
20 50.6334 28.7382 -29.0639 40.87 314.7 0.109620 0.185790 0.306527 0.404990 0.442903 0.446720 0.433840 0.410857 0.379873 0.344223 0.303430 0.260653 0.225987 0.195877 0.162223 0.136717 0.129777 0.132890 0.131807 0.137367 0.165387 0.213693 0.257573 0.273440 0.266527 0.270040 0.304143 0.366463 0.442797 0.513293 0.564877 0.604030 0.640677 0.673493 0.700223 0.726263
21 50.6650 -3.7311 -50.6936 50.83 265.8 0.097310 0.166700 0.284490 0.401947 0.468600 0.515773 0.569447 0.619413 0.636747 0.620427 0.577037 0.510810 0.438137 0.365617 0.293263 0.236050 0.195940 0.161290 0.130487 0.111327 0.103667 0.101463 0.098623 0.096377 0.098847 0.108580 0.124727 0.142693 0.155053 0.153797 0.140803 0.129527 0.129707 0.138733 0.152733 0.173857
22 60.9219 -29.3765 -27.6237 40.32 223.2 0.100430 0.163507 0.252987 0.327893 0.368377 0.399527 0.437150 0.486507 0.546597 0.605157 0.637683 0.641770 0.618013 0.568740 0.499220 0.421757 0.347423 0.279397 0.222913 0.186077 0.166210 0.153860 0.141773 0.133243 0.132477 0.138590 0.147347 0.155340 0.161937 0.164907 0.162423 0.159083 0.161860 0.168967 0.176777 0.189640
23 61.0045 -61.3012 28.8176 67.74 154.8 0.052780 0.061547 0.066247 0.069703 0.072353 0.076060 0.081903 0.092090 0.110827 0.145473 0.200163 0.280353 0.383980 0.489203 0.542017 0.523047 0.461950 0.381563 0.301170 0.233427 0.177403 0.133077 0.101747 0.084890 0.077660 0.074713 0.073370 0.073120 0.075430 0.080757 0.088027 0.095370 0.100873 0.102103 0.100827 0.104087
24 86.4316 3.0124 86.1147 86.17 88.0 0.051887 0.053957 0.056090 0.057220 0.057930 0.059423 0.062730 0.068073 0.076963 0.091503 0.110143 0.131623 0.163533 0.246240 0.428117 0.629850 0.750763 0.803923 0.823930 0.834123 0.832913 0.825750 0.812447 0.800790 0.794800 0.792840 0.793887 0.797030 0.804267 0.814207 0.825233 0.834313 0.842753 0.846837 0.846020 0.849807
25 73.6552 35.9368 68.7293 77.56 62.4 0.059320 0.065073 0.069820 0.071130 0.071143 0.071290 0.072207 0.073917 0.075410 0.076567 0.077763 0.080587 0.090060 0.123823 0.189577 0.248023 0.284807 0.326137 0.403140 0.528787 0.661220 0.754607 0.802203 0.823770 0.833443 0.837840 0.841633 0.844973 0.847730 0.847823 0.848173 0.848247 0.851023 0.852930 0.852340 0.854683
26 52.0205 56.8288 29.8147 64.18 27.7 0.069217 0.079083 0.085970 0.087753 0.087043 0.086137 0.085653 0.085167 0.084113 0.082347 0.080493 0.078767 0.077173 0.075023 0.073073 0.073013 0.074583 0.077890 0.083693 0.098093 0.147960 0.260060 0.411763 0.555000 0.656257 0.712493 0.742663 0.761870 0.777110 0.789480 0.802077 0.814607 0.828890 0.840980 0.848977 0.857423
END_DATA
consistency (worst off measurement vs averaged one) :
(https://s26.postimg.org/us0rk1ch5/consistency.jpg)
-
Thank you very much. Actually I'm not that worried about having the "perfect measurement" for these, because as can be seen in BabelColor's multi-measurements there are quite significant variations between colorcheckers even if made in the same time period. If one wants perfect there's no way around measuring one's own copy, and then it's probably better to use a larger target like the colorchecker classic as the larger patches are easier to measure.
-
I've just noticed that DXOmark publishes colour response data "Sensitivity metamerism index", "White balance scales" and colour matrix (ISO 17321), for daylight and tungsten.
https://www.dxomark.com/About/In-depth-measurements/Measurements/Color-sensitivity
Example: https://www.dxomark.com/Cameras/Leica/SL-Typ-601---Measurements
Are these measurements of any use with DCamProf or do they do nothing much more than replicate the embedded profiles in DNG raws?
-
I've just noticed that DXOmark publishes colour response data "Sensitivity metamerism index", "White balance scales" and colour matrix (ISO 17321), for daylight and tungsten.
Are these measurements of any use with DCamProf or do they do nothing much more than replicate the embedded profiles in DNG raws?
Not particularly useful imho, that's why they do not promote the Color Response tab. Deriving the white balanced raw to sRGB matrices may yield some insights but I do not believe it does anything for DCamProf other than provide a reference. More on how those matrices are obtained here (http://www.strollswithmydog.com/determining-forward-color-matrix/).
Jack
-
Shame. I was hoping to avoid some of the tedious bits of camera calibration.
-
I haven't personally thought about those numbers, if I remember correctly someone in the RawTherapee project derived color matrices from them, but I'm not sure.
I don't think they have any relation to the DNG raw color matrices, but they are the result of actual measurements they do according to some standard protocol. DCamProf can also derive matrices from measurements so it wouldn't be a help to DCamProf but rather a replacement :).
-
I was not expecting a replacement but some data that would obviate the tedious measurement task.
-
They do provide some sort of matrix, if one has access to the ISO document one could find out exactly what type of matrix it is and how it's derived. In any case it can almost certainly be transformed into a matrix used for a matrix-only DCP or DNG profile, and then you can use the json2dcp or json2icc commands in DCamProf to make a profile.
-
EDIT: I received help below. Removed files.
I hoping you guys can help me out. First off thank you for building this awesome tool. I use Capture One Pro 10 and I'm not happy with the default color profiles. Between my cameras I get different color output and on my Sony RX100 V the default color profile is just terrible.
I followed the directions in the "Easy" guide for C1 to create the ICC profiles for my cameras. The color output is better than the default profiles in C1. However it seems for all of the profiles that I have created there is a brightness added compared to the default profiles. At first I thought it was a tone curve issue, but now I'm not sure. You can see the histogram shifts to the right a little bit.
I'm using the color targets from the imaging resource. This one here (http://www.imaging-resource.com/PRODS/sony-rx100-v/RXC5hVFAI00125.ARW.HTM) for example.
These are the commands that I have run (using latest version of dcamprof 1.0.5).
scanin.exe -v -p -dipn cc24.tif ColorChecker.cht cc24_ref.cie
dcamprof make-target -X -f cc24.tif -p cc24.ti3 linear-cc24.ti3
dcamprof make-profile -i D50 -C linear-cc24.ti3 my-profile.json
dcamprof tiff-tf -f cc24.tif cc24_film.tif tone-curve.json
dcamprof make-icc -f cc24_film.tif -t tone-curve.json my-profile.json prelim-profile.icc
dcamprof make-icc -n "Sony DSC-RX100M5" -f cc24_film.tif -t tone-curve.json -t lut-curve.json -g adobergb-strong my-profile.json SonyRX100M5-Standard-D.icc
I appreciate any help you can provide.
-
What I have discovered is the Film Standard curve that is applied to the image from the RX100V doesn't achieve the same brightness level in C1 as the images for my Sony A7Rii and A6300. The darker the image prior to export the brighter the color profile will be it seems.
What I mean by that is you have X brightness with the Linear curve and Y brightness level with the Film Standard curve.
For A7Rii and A6300 you get the same relative boost in image brightness (Y) with Film Standard curve applied.
When I apply the Film Standard curve to the RX100V test image for export the brightness level Y is only 60-75% what it is compared to the other cameras. I don't know why. Seems C1 applies a different Film Standard curve to the RX100V test image compared to the other cameras and that messes up the color profile in the end. As when you load a regular image it seems C1 applies a comparable Film Standard curve that is applied to my A7Rii and A6300 test images.
In order to match the brightness level of the "Generic" profiles in C1 it seems you want your test image to show a color readout of ~230 for the D1 white color patch before exporting the files for dcamprof.
This is my first go at this so I could be doing it all wrong. However I get pleasing "true to life" color for my A7Rii and A6500. While the color profiles are different between those two cameras I'm finding the color profile I generated for the A6300 works well with my A6500 (same sensor - no surprise) and also works well with my RX100V (where as the A7Rii profile doesn't work well with my A6500 or RX100 V).
I think the best thing to do would be to get a CC24 color target and take my own images outside under sunlight.
Hope this helps someone else. And if anyone can make sense of what I've experienced and has more knowledge to share to help me I'd appreciate it.
-
Hi Dan
As you already found out the darker the shot of the target is, the brighter the profile will be. Actually it's the best to create a shot with the exact brightness you need for the target. For D1 this will be around 243 (with sRGB). It's better to use a brighter shot (without clipping) if you don't hit the exact value.
Then there is also the lens in front of your camera that has an influence. Some lenses adds contrast and get clippings faster than others. If I'm shooting the same target with the same settings with different lenses I'll get different values. Here are three D1 values from a few shots I made recently of the same setting with three different lenses on my Sony a7RII: Sony 90 G Macro: 244, Sony 85 GM: 230, Sony 70-300 G: 255 (clipped). If the colors of these lenses are not completely different, you can still use one profile for all lenses. But expect different steps in processing (especially for contrast, clipping, brightness, exposure, ...). As you can identify out of my answer a different lens will end up in different behaviour with the same curve (no matter of film standard or linear curve).
I hope this will answer most of your questions along with the PM I'll sent to you.
Cheers, seb
-
Late to the party, I know, but it was interesting reading about RGB-Lum and CAT tweaks a few pages back. I'm sure I'll enjoy trying the new features out once they're released.
-
I haven't written here for a while, but I've been busy working with the GUI version. It's in feature freeze (only bug fixing left), and I'm working on the manual. It's going a bit slow and I'm starting a new project (in my main business, telecom) plus some stuff happening in my personal life which will make it harder to spend a lot of time on this. I still hope I can release late May or early June.
The CAT tweaks were eventually canned, didn't work out in the end. RGB-Lum is in there though, but will be in the GUI version only first, until l have the time to update DCamProf with the new features. There's quite much new stuff, not just RGB-Lum. The big thing is a full 3D LUT intended for reproduction profiles, plus a new smarter LUT relax algorithm.
Thanks for providing answers above by the way, I don't have as much time to provide support for the moment while trying to finalize this GUI version.
-
I hope this will answer most of your questions along with the PM I'll sent to you.
Cheers, seb
Yes it does. Thank you very much for the reply. That was very helpful.
I will test a bit more and see what results I get.
EDIT: Removed files as the tone curves are all wrong. I have updated ones I created with Lumariver which are much better.
Dan
-
Hi Jack,
torger has correctly diagnosed the problem - the ColorChecker recognition layout is not identical to that of the Passport. I have hacked together a half Passport .cht file that may work better if only the ColorChecker 24 side of the chart has been shot, and it is here (http://www.argyllcms.com/ColorCheckerHalfPassport.cht).
When using the HalfPassport.cht how should the image be oriented? Black patch lower right OR Black patch upper left?
Thanks for your help.
-
When using the HalfPassport.cht how should the image be oriented? Black patch lower right OR Black patch upper left?
It shouldn't matter, but CC convention is black square lower right.
-
It shouldn't matter, but CC convention is black square lower right.
Many thanks for the swift reply.
-
First, many thanks for your software. I admit I am still struggling to comprehend all the subtleties.
One question, and I hope this is not a repetition of something appearing earlier in this thread:
I quote from your instructions:
"As I also contribute to the RawTherapee project, there’s now actually a “perceptual” curve there which is based on DCamProf’s neutral tone reproduction operator (a bit simplified), so for RawTherapee you can provide a profile without a curve and instead apply it using the built-in curve adjustment,"
You make use of this curve in the command for merging into a dual illuminant profile:
dcamprof make-dcp -n "Canon EOS 5D Mark II" -d "My Profile" -i StdA -I D50 \ -b 0.3 -t curve.rtc -g adobergb-strong \ final-2.json final-1.json final-dual.dcp
Here it is referred to as curve.rtc. Nowhere can I find the instructions for creating this file.
Can you help? Again many thanks.l
-
Clearly I am a rank beginner just exploring the possibilities of using DCamProf to create better camera profiles (DCP).
After spending a good bit of time reading the documentation I decided to put a toe into the water. After just about knee depth I ran into a problem. Not really a problem but an unexpected result. Just as a sanity check I attempted to create a (Argyll type)-.val file:
I made a raw image of the Color Checker Passport, opened it in RawTherapee; Neutral profile. Cropped the image fairly tightly to just the color patch page of the Color Checker. Ensure the black patch is positioned at lower right. In RT Color Management tool: click "Save reference image for profiling"; apply white balance NOT checked. Resulting image file name: CCPP.tif.
I then proceeded to create a .val file. This file should contain the RGB values for each of the 24 color patches. First opened a command promp in the folder containing the tif image.
scanin -o -dipn CCPP.tif %ARGLREF%\ColorCheckerHalfPassport.cht diag.tif
The process ran smoothly with no errors reported. the resulting output files: diag.tif when opened showed perfect registration and no alignment errors.
The file CCPP.val gave R, G & B values for each of the color patches as expected.
The PROBLEM: The RGB values shown by the file did not come even close to approximating the reference values for the ColorCheckerPassport derived from several online sources. Is this to be expected? If not, where have I gone wrong?
I have searched the 77+ pages of this thread for appropriate clues but, no luck. I hope I am not re-covering subjects which have long been answered.
Many thanks for any help offered.
P.S. Since this is primarily an Argyll exercise, I should probably address the question to Mr. Gill..
-
If it is not close two things come to my mind: your shot of the target is not good or you have bad/wrong reference values.
May you share your target shot, so we can have a look at?
-
I made a raw image of the Color Checker Passport, opened it in RawTherapee; Neutral profile. Cropped the image fairly tightly to just the color patch page of the Color Checker. Ensure the black patch is positioned at lower right. In RT Color Management tool: click "Save reference image for profiling"; apply white balance NOT checked. Resulting image file name: CCPP.tif.
I then proceeded to create a .val file. This file should contain the RGB values for each of the 24 color patches. First opened a command promp in the folder containing the tif image.
The PROBLEM: The RGB values shown by the file did not come even close to approximating the reference values for the ColorCheckerPassport derived from several online sources. Is this to be expected? If not, where have I gone wrong?
Think about what you are doing. You are taking a picture with the device you are hoping to profile. By definition, the RGB values you get back are in the cameras native colorspace, not some sort of "reference" values. (Any such "reference" RGB values nonsense anyway, the CCP's actual reference values are the spectral reflectances or CIE XYZ values, a CCP target is not an RGB device!).
By being able to correspond the actual colors of the CCP target with the cameras RGB values you can construct a camera profile, which is capable of converting between the camera RGB values and the corresponding CIE XYZ colors. You can then use that profile to convert camera RGB values into other colorspaces (such as sRGB colorspace, or your displays colorspace), by linking it with the desired destination profile, i.e. Camera RGB -> CIE XYZ -> sRGB. This is the basics of color profiles and color management.
-
By definition, the RGB values you get back are in the cameras native colorspace, not some sort of "reference" values. (Any such "reference" RGB values nonsense anyway, the CCP's actual reference values are the spectral reflectances or CIE XYZ values, a CCP target is not an RGB device!).
GWGill:
Many thanks for your reply. I do understand what you have explained. I will now carry on and create the camera profile, and see how it comes out. Thus far I am pleased that I experienced no difficulty using the scanin command and capturing the image with no registration problems.
-
Getting closer to release of Lumariver Profile Designer aka DCamProf GUI, here's a quick peek:
https://www.youtube.com/watch?v=UUt3jWs5vTk
-
Getting closer to release of Lumariver Profile Designer aka DCamProf GUI, here's a quick peek:
https://www.youtube.com/watch?v=UUt3jWs5vTk
Any chances to support different targets such as SpyderChekr, Colorchecker 24 etc. and simple matrix profiles tweaked from a zero state by means of Hue, Sat and perhaps a Lightness fader and maintaining a neutral axis? For a LUT profile, I believe the Passport has too few patches and I prefer matrix profiles in general. What I can see now - there is too many controls and too little data to judge based upon. Anyway, a really good job done.
-
Any chances to support different targets such as SpyderChekr, Colorchecker 24 etc. and simple matrix profiles tweaked from a zero state by means of Hue, Sat and perhaps a Lightness fader and maintaining a neutral axis? For a LUT profile, I believe the Passport has too few patches and I prefer matrix profiles in general. What I can see now - there is too many controls and too little data to judge based upon. Anyway, a really good job done.
There's support for several targets built-in (SpyderChekr 24 is among the supported, not the 48 patch version though, have no data for that), and for the more expensive Repro edition also support for custom targets (then you can scan your 48 patch SpyderChekr), combining multiple targets and even loose patches not in a grid (for spot colors).
I can still guarantee that not all wishes will be fulfilled :-), one thing I've learnt about profiling is that many users have strong opinions on how things should be and they're often contradicting, so one have to choose which users to satisfy. Matrix-only profiles can be made, but it's not the top of the list concerning use case support. Central to the software is still the LUT-based functionality. In fact I don't really think it's that necessary to have a profile maker at all when making a matrix only profile, it's just nine values. It seems like most matrix-only users just want a matrix of those nine values which they can enter by hand. DCamProf command line is more than enough for those hardcore users, so spending lots of time optimizing for that use case when noone will buy it for that anyway seems a bit of a waste.
Concerning patch count I think that relatively few patches like in a CC24 is an advantage for matrix optimization. You just need a few good handles as reference to steer the optimizer. As a matrix is linear it can't do much with lots of patches anyway. There can be an advantage to have many patches to choose from I guess so you can pick exactly the color of the handles you want, but when tuning optimization you only need a few. If you have lots of patches when optimizing the matrix will just lock down to a good average fit and you can't affect it much, which is fine for automatic optimization but not so easy to work with when tuning manually to taste.
For a reproduction I indeed recommend a larger target like CCSG, for which there is built-in support. A large glossy target requires very careful made shooting setup to minimize glare issues though. For a general-purpose LUT profile I think the CC24 is still adequate. The thing is that the LUT should only do very minor adjustments, otherwise smoothness is at risk. The main use of the LUT in a general-purpose profile is to provide the tone reproduction operator, gamut compression and look adjustments, the colorimetric aspect is quite minor. It's even quite common to disable the colorimetric LUT and only use it for the other parts, that is use a matrix-only profile as the base. One can use larger targets though, and multi-target just as with the DCamProf advanced tutorial show, but my experience concerning general-purpose profiles is that it's more a curiosity than a real benefit. Taste may differ there though, and in this case the software does allow for both approaches. Just like DCamProf it's so feature rich that I as the developer can't really predict all use cases users will try. It shall be interesting to see.
Concerning the neutral axis I don't understand what the problem is. Neutral is simply set by the camera's white balance, right? The matrix doesn't affect that, at least not the forward matrix and that's the only matrix that is (optionally) tunable with Lumariver Profile Designer as that is the only thing that matters in 98% of the use cases. Should be said that both DCamProf and Lumariver Profile Designer is "limited" to make only "whitepoint preserving" matrices, that is matrices that don't shift the whitepoint. DNG profiles require such a matrix, but I think it's a general good design property so it cannot be turned off. If you use some other software or just manually enter matrix values you can indeed make matrices that does shift the neutrals away from the white balance setting.
(I still need to do some mundane testing, but it seems quite feasible to make a public release of version 1.0 in 2-3 weeks.)
-
The video looks very promising. I'm looking forward to the release.
-
"DCamProf GUI" the commercial Lumariver Profile Designer 1.0 is now released, see http://www.lumariver.com/#LumariverPD
It's available for MacOS and Windows.
-
Matrix-only profiles can be made, but it's not the top of the list concerning use case support. Central to the software is still the LUT-based functionality. In fact I don't really think it's that necessary to have a profile maker at all when making a matrix only profile, it's just nine values. It seems like most matrix-only users just want a matrix of those nine values which they can enter by hand. DCamProf command line is more than enough for those hardcore users, so spending lots of time optimizing for that use case when noone will buy it for that anyway seems a bit of a waste.
I have a suggestion which I think was suggested before (may be even by myself) - how about an option to make a LUT profile where LUTs are simply imitating the pure matrix profile (doing nothing really in addition to the matrix transform) but still the tables are in the profile, but then the user can either take it to Luma* Profile Designer (or to C1 ColorEditor which requires LUT tag A2B0 to be present to work) and do minor tweaks ...
-
I have a suggestion which I think was suggested before (may be even by myself) - how about an option to make a LUT profile where LUTs are simply imitating the pure matrix profile (doing nothing really in addition to the matrix transform) but still the tables are in the profile, but then the user can either take it to Luma* Profile Designer (or to C1 ColorEditor which requires LUT tag A2B0 to be present to work) and do minor tweaks ...
You can sort of do that already, by not making any LUT corrections, but still forcing a LUT to be included by using gamut compression or a tone reproduction operator that requires it, or including a look operator (look adjustments editor in LRPD) that does nothing. Using the latter thing (dummy look operator) I think you can make a LUT that does precisely nothing.
LRPD doesn't have an import function though, that is it cannot import finished DNG or ICC profiles and modify them (except in the inspect/edit tool). Importing a matrix would be a fairly easy feature to add though (importing LUT is harder as the internal LUT format is entirely different from a DCP or ICC LUT)
-
LRPD doesn't have an import function though, that is it cannot import finished DNG or ICC profiles
how about dcamprof own native ".json" intermediate profiles from "dcamprof make-profile" output ?
-
how about dcamprof own native ".json" intermediate profiles from "dcamprof make-profile" output ?
Yes that could be done, I don't think it would be a much used function though, using both DCamProf and LRPD simultaneously, but maybe I'm wrong?
-
Yes that could be done, I don't think it would be a much used function though, using both DCamProf and LRPD simultaneously, but maybe I'm wrong?
for example if I use SSF then it is faster to create something baseline using dcamprof and then may be I want to tune couple of things visually in GUI ... I think it is logical to provide such a path where GUI product can take over from dcamprof at this point - more so you can write to your own .json profile format whatever support information needed for GUI product ...
-
Well done Anders - it looks great!! I will give it a go alongside the dcamprof
"DCamProf GUI" the commercial Lumariver Profile Designer 1.0 is now released, see http://www.lumariver.com/#LumariverPD
It's available for MacOS and Windows.
-
Any tips on how I can use a IT8.7 target for a quick-dirty camera profile? I have a xenon flash and shoot through umbrella at my disposal.
-
Any tips on how I can use a IT8.7 target for a quick-dirty camera profile? I have a xenon flash and shoot through umbrella at my disposal.
if this is from usual source (like Faust) then it is gloss/semi-gloss and extra care shall be taken to avoid reflections ... you also want to avoid spilling the light all over place so why 'd you use umbrella ? covering everything (sides, top, bottom) in matte black cloth (make some kind of a box inside a room) is a good idea and directing light at angle and lens though holes in that cloth
-
IT8.7 or other (semi-)glossy targets aren't that good for quick-and-dirty profiles as they're so sensitive to glare. You need to really work hard to reduce glare to a minimum to make good use of such a target, otherwise any matte target will be better.
There's some shooting tips here https://www.ludd.ltu.se/~torger/dcamprof.html#shoot_target and here http://www.ludd.ltu.se/~torger/photography/camera-profiling.html#step3 and you can find more if searching the web for reproduction photography tips.
-
BTW, for people who can't DIY from LED strips -- here is finally an affordable decent LED light to illuminate targets with ~"5600K" spectrum assembled to accept 110-220V AC
https://store.yujiintl.com/products/bc-series-high-cri-led-3030-led-floodlight-pack-1pcs