Pages: 1 ... 51 52 [53] 54 55 ... 78   Go Down

Author Topic: DCamProf - a new camera profiling tool  (Read 911769 times)

mi-fu

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 50
Re: DCamProf - a new camera profiling tool
« Reply #1040 on: October 28, 2015, 12:41:49 pm »

yes! permission was the problem. it works now!! thanks!!  ;D

I doubt you did something wrong.  Have you downloaded other versions that I've created?

What appears to have happened, and this is not overly unusual when doing file transfers is that the Unix 'execute' bit
gets turned off (for security purposes).

So, you often have to turn it back on.

If you do a   'ls  -lt  ./dcamprof-v0.10.2'  command, does the left side show something like rw-r--r--   ???

If so, then do a  'chmod  755  ./dcamprof-v0.10.2'  and then you should be able to run it.

I will say that the various versions have been downloaded a total of 34 times and this is the first 'problem' report so
I dont think it's an actual problem, just a user education/knowledge issue.

I suppose if I 'zip'd' the file before I uploaded then the bit would remain set but that would force people to unzip.  Potato-Potatoe
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #1041 on: October 28, 2015, 01:53:24 pm »

AlterEgo,

what is the latest/best ColorChecker SG spectral measurement data you have? Can I include and distribute with DCamProf as a ccsg_ref.cie file?
Logged

AlterEgo

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1995
Re: DCamProf - a new camera profiling tool
« Reply #1042 on: October 28, 2015, 02:02:05 pm »

AlterEgo,

what is the latest/best ColorChecker SG spectral measurement data you have? Can I include and distribute with DCamProf as a ccsg_ref.cie file?

didn't I post the measurements here ? I need to check, but I think I did and did also after X-Rite replaced my original "defective" chart following some protracted dispute between me and their CSRs about the measurements (numbers) of the black patches ... in any case all the files I ever attached here in LuLa forum with whatever measurements _I did_ are absolutely free to use by anybody for whatever purposes...
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #1043 on: October 28, 2015, 02:08:56 pm »

didn't I post the measurements here ? I need to check, but I think I did and did also after X-Rite replaced my original "defective" chart following some protracted dispute between me and their CSRs about the measurements (numbers) of the black patches ... in any case all the files I ever attached here in LuLa forum with whatever measurements _I did_ are absolutely free to use by anybody for whatever purposes...

The thread is quite long hard to find. I think this is the latest but I'm not sure:
http://forum.luminous-landscape.com/index.php?topic=100015.msg848342#msg848342
Logged

AlterEgo

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1995
Re: DCamProf - a new camera profiling tool
« Reply #1044 on: October 28, 2015, 02:16:15 pm »

...
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #1045 on: October 28, 2015, 04:41:10 pm »

...

Thanks. Which one of the two do you think is more accurate? The Spectrolino or the Ipro? If I remember correctly there are quite some difficulties to measure the black patches dark enough.
Logged

Alexey.Danilchenko

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 257
    • Spectron
Re: DCamProf - a new camera profiling tool
« Reply #1046 on: October 29, 2015, 05:18:35 am »

Thanks. Which one of the two do you think is more accurate? The Spectrolino or the Ipro? If I remember correctly there are quite some difficulties to measure the black patches dark enough.

I used the ones from Iliah MakeInpitICC (I believe they were measured with Spectrolino) to make profiles from SSF - they worked great.
Logged

AlterEgo

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1995
Re: DCamProf - a new camera profiling tool
« Reply #1047 on: October 29, 2015, 03:27:49 pm »

Thanks. Which one of the two do you think is more accurate? The Spectrolino or the Ipro? If I remember correctly there are quite some difficulties to measure the black patches dark enough.

I really do not know (I resorted back to using x-rite passport target shots for CMF/SSF approximation)... I, for example, do not even know whether XRGA or non XRGA mode is more valid ;D ... that XRGA was brought by X-Rite to make the results from instruments from 2 different companies (GmB and X-Rite, after GmB was purchased by X-Rite) closer... but that brings a question - who was more precise in the first place  :o ( https://www.xrite.com/documents/literature/en/L7-462_XRGA_WhitePaper_en.pdf ) ?
Logged

AlterEgo

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1995
Re: DCamProf - a new camera profiling tool
« Reply #1048 on: October 29, 2015, 03:45:48 pm »

I used the ones from Iliah MakeInpitICC (I believe they were measured with Spectrolino) to make profiles from SSF - they worked great.
the questions is - can you trust your target... the one I got first turned out to be defective... now looking @ manufacturers of the targets, X-Rite, with all fine print, seems to be the best one, but can you trust w/o measuring and when measuring - can you trust the instrument ?

Iliah's data close to GmB's, but in one patch quite different from GmB's data for SG

my Spectrolino (the really old one device) measurements of my (X-Rite made in 2014) target is closer to Iliah's data than my new i1Pro2 (in XRGA mode, mind you) to the same

but both of my measurements are as close (or may be even closer) between themselves as Iliah's vs GmBs... so may be, may be... the culprit is really the target(s) that we are measuring ?!

PS: I also want to confess that I did not gather a strength to do multiple measurements of SG and average those...
« Last Edit: October 29, 2015, 06:41:05 pm by AlterEgo »
Logged

AlterEgo

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1995
Re: DCamProf - a new camera profiling tool
« Reply #1049 on: October 29, 2015, 03:46:53 pm »

here is comparison between one of my Spectrolino's in non XRGA mode and i1Pro2 in XRGA mode with my SG target :

===

PatchTool COMPARE TOOL - COMPARE STATS REPORT

This file combines and compares the data of two files which have the same number of samples.
This report presents statistical data derived from the differences between the color values of these two data sets.

Date:    "2015-10-29"
Time:    "03:45:08 PM"
Version:    "4.7.0 b362"

REFERENCE
- Name:    "CCSG(v2014.11)-10.0nm-i1Pro2-XRGA (SN#1044184,2015-07-09).cie (M0)"
# The SOURCE data type is < spectrum >.

SAMPLE
- Name:    "CCSG(v2014.11)-10.0nm-Spectrolino (SN#17984,2015-07-09).cie (M0)"
# The SAMPLE data type is < spectrum >.

STATS-SETTINGS
Delta parameter:    "E*"
Illuminant:    "D50"
Observer:    "2 deg."
Delta-E* formula:    "CIEDE2000"
Absolute values:    "NO"
Separate Neg./Pos. stats:    "NO"
Negative samples:    0
Positive samples:    140
Number of samples:    140

AVERAGE
All samples:    0.64
Best 90%:    0.60
Worst 10%:    0.95

STANDARD-DEVIATION
All samples:    0.19
Best 90%:    0.17
Worst 10%:    0.05

MAXIMUM-ERROR
10th percentile:    ±    0.38
Median (50th perc.):    ±    0.64
90th percentile:    ±    0.89
95th percentile:    ±    0.93
Of all samples:    ±    1.06

HISTOGRAM-DATA
Bin-size:    0.2000
Bin center    0.1000   0.3000   0.5000   0.7000   0.9000   1.1000
No patches    2   13   43   53   27   2

CUMULATIVE RELATIVE FREQUENCY (CRF) DATA
CRF of all-samples:
Max-error   0.1393   0.3129   0.3830   0.4242   0.4484   0.5067   0.5377   0.5636   0.5943   0.6248   0.6438   0.6581   0.6804   0.7226   0.7456   0.7722   0.8126   0.8463   0.8856   0.9325   1.0585
CRF   0%   5%   10%   15%   20%   25%   30%   35%   40%   45%   50%   55%   60%   65%   70%   75%   80%   85%   90%   95%   100%
Logged

AlterEgo

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1995
Re: DCamProf - a new camera profiling tool
« Reply #1050 on: October 29, 2015, 03:48:18 pm »

and here is Iliah's vs GmB's own :

===

PatchTool COMPARE TOOL - COMPARE STATS REPORT

This file combines and compares the data of two files which have the same number of samples.
This report presents statistical data derived from the differences between the color values of these two data sets.

Date:    "2015-10-29"
Time:    "03:47:45 PM"
Version:    "4.7.0 b362"

REFERENCE
- Name:    "ColorCheckerSG_spectral.cie (M0)"
# The SOURCE data type is < spectrum >.

SAMPLE
- Name:    "Iliag Borg = DigitalColorCheckerSG.cie (M0)"
# The SAMPLE data type is < spectrum >.

STATS-SETTINGS
Delta parameter:    "E*"
Illuminant:    "D50"
Observer:    "2 deg."
Delta-E* formula:    "CIEDE2000"
Absolute values:    "NO"
Separate Neg./Pos. stats:    "NO"
Negative samples:    0
Positive samples:    140
Number of samples:    140

AVERAGE
All samples:    0.67
Best 90%:    0.60
Worst 10%:    1.32

STANDARD-DEVIATION
All samples:    0.29
Best 90%:    0.19
Worst 10%:    0.30

MAXIMUM-ERROR
10th percentile:    ±    0.39
Median (50th perc.):    ±    0.59
90th percentile:    ±    1.08
95th percentile:    ±    1.22
Of all samples:    ±    2.32

HISTOGRAM-DATA
Bin-size:    0.5000
Bin center    0.2500   0.7500   1.2500   1.7500   2.2500
No patches    40   84   15   0   1

CUMULATIVE RELATIVE FREQUENCY (CRF) DATA
CRF of all-samples:
Max-error   0.0932   0.3225   0.3893   0.4479   0.4766   0.4903   0.5066   0.5382   0.5627   0.5827   0.5917   0.6119   0.6391   0.6698   0.7114   0.7769   0.8541   0.9236   1.0752   1.2188   2.3185
CRF   0%   5%   10%   15%   20%   25%   30%   35%   40%   45%   50%   55%   60%   65%   70%   75%   80%   85%   90%   95%   100%
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #1051 on: October 29, 2015, 06:34:39 pm »

I'm working on a new glare modeling algorithm and had some promising results. The old linearization (available in the testchart-ff command with the -L flag) does not work that well, but with the new one it seems like some sort of results can be had even with 1+ stop glare in the CCSG shot, like there is in for example Imaging Resource's raw files.

I'll also make it a bit easier than before to make use of the white-gray-black calibration patches along the CCSG border.

In my new glare approach I don't linearize the camera raw data, but instead add glare to the reference values (or reference spectra when available) to match the camera's glare. It's not as elegant and not how I'd want to do it, but it seems to be more robust. The key advantage of doing this way is that you know the CMF of the observer, you don't know the SSF of the camera (well if you did you wouldn't need glare modeling), and by that you can make demarcations in the model like keeping hue constant.

Glare modeling will still be no replacement for shooting the target properly though. I don't think I have seen a single CCSG shot with less than at least 1 stop of glare. Shooting glossy targets are tricky indeed. A CC24 shot with low amounts of glare will most likely make a more accurate profile than a CCSG shot with high amounts of glare, even if compensated with a glare reduction model, as there's too many unknowns when it comes to glare to make an accurate model.
« Last Edit: October 29, 2015, 06:45:54 pm by torger »
Logged

Alexey.Danilchenko

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 257
    • Spectron
Re: DCamProf - a new camera profiling tool
« Reply #1052 on: October 29, 2015, 07:08:46 pm »

the questions is - can you trust your target... the one I got first turned out to be defective... now looking @ manufacturers of the targets, X-Rite, with all fine print, seems to be the best one, but can you trust w/o measuring and when measuring - can you trust the instrument ?
If you read what I said then the answer is right there ;)

I can because my target is synthetic - generated from CCSG data, illuminant and SSF by dcamprof. I used that synthetic target to build and optimise matrix profile and results came out very good and close to the one Iliah built and tuned by hand (for Kodak SLR/n)
Logged

AlterEgo

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1995
Re: DCamProf - a new camera profiling tool
« Reply #1053 on: October 29, 2015, 07:48:36 pm »

my target is synthetic
right, then it does not matter much which data - you can as well use GmB's own spectral data too (from profilemaker distribution).
Logged

sTi

  • Newbie
  • *
  • Offline Offline
  • Posts: 6
Re: DCamProf - a new camera profiling tool
« Reply #1054 on: October 30, 2015, 05:56:39 am »

Thanks for making available this tool!
I am just starting out and followed the "easy way" tutorial to build DCP profiles.
After reading through the documentation I am still unsure about two issues:

1. I use a Colorchecker Passport as a target. In your documentation you write that the cc24_ref.cie from DCamProf should be used in combination with ColorChecker.cht from Argyll. However, Argyll comes also with a ColorcheckerPassport.cht file, wouldn't this be a better choice for the passport target, or would this not be compatible with the cc24_ref.cie you provide?

2. Should the -C parameter also be used when creating a Tungsten profile, or is this only meant for the D50 one?
Quote
dcamprof make-profile -i StdA -C cc24.ti3 my-profile.json

Would this then be the correct command for merging the profiles?

Quote
dcamprof make-dcp -n "Camera Name" -d "Displayed Name of Profile" -i StdA -I D50 -t acr -g adobergb my-profile2.json my-profile1.json my-profile-dual.dcp
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #1055 on: October 30, 2015, 06:51:31 am »

I don't have a spectral reference file for the color checker passport, so you have to use only the CC24 part of it, and then you use ColorChecker.cht.

If someone has a high quality spectral measurement file for the passport, it would be nice if I could get it for inclusion. But anyway, there are a few more patches on the passport but I don't think they add much to the CC24 as they are of similar color.

-C is used if you want the colors to look as if shot under D50. I only use it for light sources close to D50 or if I have some special need to match D50 despite different light. So the default choice is not to use it.

Say if you do reproduction work and use tungsten lights instead of flash, it's likely that you want it to look the same as if you where using a D50-ish flash, and then -C would be appropriate. For general purpose profiles I think it makes more sense to simulate the colors we actually see under that light. Tungsten is a bit complicated though as it's often so low in temperature that we don't do a full chromatic adaptation (ie we see that the light is yellow) and raw converters are not really designed to simulate that.

The merge command look correct just make sure that the profiles are listed in the right order, stda first then d50.

Thanks for making available this tool!
I am just starting out and followed the "easy way" tutorial to build DCP profiles.
After reading through the documentation I am still unsure about two issues:

1. I use a Colorchecker Passport as a target. In your documentation you write that the cc24_ref.cie from DCamProf should be used in combination with ColorChecker.cht from Argyll. However, Argyll comes also with a ColorcheckerPassport.cht file, wouldn't this be a better choice for the passport target, or would this not be compatible with the cc24_ref.cie you provide?

2. Should the -C parameter also be used when creating a Tungsten profile, or is this only meant for the D50 one?
Would this then be the correct command for merging the profiles?
Logged

Alexey.Danilchenko

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 257
    • Spectron
Re: DCamProf - a new camera profiling tool
« Reply #1056 on: October 30, 2015, 11:07:09 am »

right, then it does not matter much which data - you can as well use GmB's own spectral data too (from profilemaker distribution).

I prefer to use a real measured one from which good profiles were made. And Xrite has not published spectral data for them - only LAB that I could find
Logged

AlterEgo

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1995
Re: DCamProf - a new camera profiling tool
« Reply #1057 on: October 30, 2015, 11:39:13 am »

And Xrite has not published spectral data for them - only LAB that I could find

spectral data was supplied for example with ProfileMaker distro (the same data comes with BabelColor Patchtool)

also, for those who do not use SSF/CMF (and hence any reasonable variations not important - I mean for SSF/CMF case they are not important) and actually use the real SG target for profile making = http://xritephoto.com/ph_product_overview.aspx?ID=938&Action=Support&SupportID=5884&catid=28

X-Rite finally bothered to post this info officially this week

===

difference between Iliah's spectral data and and X-Rite LAB data for new SG targets (I have /and measured/ a new one, batch from Nov 2014) :

Quote
PatchTool COMPARE TOOL - COMPARE STATS REPORT

This file combines and compares the data of two files which have the same number of samples.
This report presents statistical data derived from the differences between the color values of these two data sets.

Date:    "2015-10-30"
Time:    "11:34:00 AM"
Version:    "4.7.0 b362"

REFERENCE
- Name:    "Iliag Borg = DigitalColorCheckerSG.cie (M0)"
# The SOURCE data type is < spectrum >.

SAMPLE
- Name:    "ColorCheckerSG_After_Nov2014.txt (M0)"
# The SAMPLE data type is < XYZ >.
- Observer:    "2 degree (fixed)"

STATS-SETTINGS
Delta parameter:    "E*"
Illuminant:    "D50"
Observer:    "2 deg."
Delta-E* formula:    "CIEDE2000"
Absolute values:    "NO"
Separate Neg./Pos. stats:    "NO"
Negative samples:    0
Positive samples:    140
Number of samples:    140

AVERAGE
All samples:    0.87
Best 90%:    0.78
Worst 10%:    1.68

STANDARD-DEVIATION
All samples:    0.44
Best 90%:    0.35
Worst 10%:    0.32

MAXIMUM-ERROR
10th percentile:    ±    0.37
Median (50th perc.):    ±    0.75
90th percentile:    ±    1.39
95th percentile:    ±    1.52
Of all samples:    ±    2.39

HISTOGRAM-DATA
Bin-size:    0.5000
Bin center    0.2500   0.7500   1.2500   1.7500   2.2500
No patches    28   60   44   6   2

CUMULATIVE RELATIVE FREQUENCY (CRF) DATA
CRF of all-samples:
Max-error   0.0837   0.2188   0.3742   0.4567   0.4963   0.5528   0.5869   0.6166   0.6403   0.7193   0.7514   0.8216   0.9676   1.0391   1.1621   1.2296   1.3036   1.3256   1.3940   1.5171   2.3893
CRF   0%   5%   10%   15%   20%   25%   30%   35%   40%   45%   50%   55%   60%   65%   70%   75%   80%   85%   90%   95%   100%




===

difference between my (one time, not averaged) i1Pro2 XRGA measured spectral data and X-Rite LAB data for new SG targets (I have /and measured/ a new one, batch from Nov 2014) :


Quote
PatchTool COMPARE TOOL - COMPARE STATS REPORT

This file combines and compares the data of two files which have the same number of samples.
This report presents statistical data derived from the differences between the color values of these two data sets.

Date:    "2015-10-30"
Time:    "11:36:29 AM"
Version:    "4.7.0 b362"

REFERENCE
- Name:    "CCSG(v2014.11)-10.0nm-i1Pro2-XRGA (SN#1044184,2015-07-09).cie (M0)"
# The SOURCE data type is < spectrum >.

SAMPLE
- Name:    "ColorCheckerSG_After_Nov2014.txt (M0)"
# The SAMPLE data type is < XYZ >.
- Observer:    "2 degree (fixed)"

STATS-SETTINGS
Delta parameter:    "E*"
Illuminant:    "D50"
Observer:    "2 deg."
Delta-E* formula:    "CIEDE2000"
Absolute values:    "NO"
Separate Neg./Pos. stats:    "NO"
Negative samples:    0
Positive samples:    140
Number of samples:    140

AVERAGE
All samples:    0.63
Best 90%:    0.59
Worst 10%:    1.02

STANDARD-DEVIATION
All samples:    0.23
Best 90%:    0.19
Worst 10%:    0.18

MAXIMUM-ERROR
10th percentile:    ±    0.30
Median (50th perc.):    ±    0.65
90th percentile:    ±    0.89
95th percentile:    ±    0.93
Of all samples:    ±    1.60

HISTOGRAM-DATA
Bin-size:    0.5000
Bin center    0.2500   0.7500   1.2500   1.7500
No patches    37   99   3   1

CUMULATIVE RELATIVE FREQUENCY (CRF) DATA
CRF of all-samples:
Max-error   0.0792   0.1841   0.2959   0.4175   0.4697   0.4942   0.5256   0.5392   0.5822   0.6039   0.6505   0.6760   0.6970   0.7164   0.7524   0.7763   0.7959   0.8382   0.8854   0.9291   1.6020
CRF   0%   5%   10%   15%   20%   25%   30%   35%   40%   45%   50%   55%   60%   65%   70%   75%   80%   85%   90%   95%   100%
« Last Edit: October 31, 2015, 10:33:16 am by AlterEgo »
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #1058 on: October 30, 2015, 04:55:21 pm »

Just released v0.10.3

Included CCSG data. The main new feature is "glare matching", a (for me) new approach to deal with glare. Instead of trying to linearize the raw samples glare is simulated and added to the reference values/spectra to match the camera.

It's now also possible to do flatfield and glare matching directly in the make-profile command by providing a target layout JSON pointing out white, black and middle gray patches. I've added example for the CCSG (ccsg-layout.json) and CC24 (cc24-layout.json). The SG with its border of white/gray/black patches supports full flatfield correction and locally varying glare matching. The CC24 with only one white patch doesn't support flatfield correction, but a global glare matching can be made. CC24 shots generally have low amounts of glare as it's a matte target but still I've added the use of glare matching in the basic use case examples. It makes a little difference, making dark colors a tiny bit lighter and a tiny tiny bit lower saturation.

To see stronger effect of glare matching you need to use a glossy target like the CCSG. I've attached an animated GIF that makes an A/B swap between the results of two profiles, one with glare matching one without (click on the image to start the animation). The shot comes from Imaging Resource and suffers quite some glare on the right side which you can see by that despite even light (all white patches equally bright) the black patches considerably brighter on the right side.

If you don't compensate glare make-profile will think that the camera renders dark colors very light and overall a bit low contrast and saturation and will thus make a profile that compensates, and thus dark colors get very dark and the profile becomes a little over-saturated (hard to see in the animated gif which is limited to 256 colors). This is most easily seen in the top right corner.

The other is the profile made on this target shot but with glare matching. I don't own a CCSG so I can't compare with my eyes but I think it makes a quite good job. I still strongly recommend against relying on glare matching rather than shooting your targets properly, but sometimes you get shots from third parties and you just need to make the best out of it.

The patch that really gives it away that one of the profiles have incorrect color is the pink patch in the CC24 section of the target (located in upper center in the target). In the profile without glare matching the pink is rendered much darker pretty close to the neighboring red patch, not at all how the CC24 looks.

Example command for CCSG:
scanin -v -dipn ccsg.tif ColorCheckerSG.cht ccsg_ref.cie
dcamprof make-profile -g ccsg-layout.json -y -100 -l 0 -t acr ccsg.ti3 glare-demo-full-correct.dcp

I included AlterEgo's spectrolino data as ccsg_ref-new.cie, the ccsg_ref.cie is for the old charts before November 2014. For the CC24 I only have for the old charts so far.
« Last Edit: October 30, 2015, 05:14:14 pm by torger »
Logged

AlterEgo

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1995
Re: DCamProf - a new camera profiling tool
« Reply #1059 on: October 30, 2015, 05:44:06 pm »

0.10.3 build for Windows ( mingw = dcamprof.exe + libgomp_64-1.dll + both manual & tutorial / = copies of Torger's web pages / in 3 formats : IE archive .mht, Mozilla archive .maff and regular .pdf ) : https://app.box.com/DCamProf
Logged
Pages: 1 ... 51 52 [53] 54 55 ... 78   Go Up