DCamProf for dummies?

[howardm]

OK so emphasis on the dummy over here...I'm making progress here but when I try to run step 4 (as far as I've gotten) in the ICC profile instructions, i get this error:

dyld: Library not loaded: /opt/local/lib/liblcms2.2.dylib
  Referenced from: /Applications/Argyll_V1.8.0/bin/dcamprof
  Reason: image not found
Trace/BPT trap: 5

I exported from Capture one (following your directions) so I assume this step is necessary...I'm so close...I tried downloading little cms and executing the configure file but i guess that doesn't work? so close.....

Regarding all that, here is what is happening.  Whoever built dcamprof had littlecms installed in /opt/local/....   That tells me that he used something like the Mac 'port' system (that lets you build all sorts of open source software for the odd OSX environment.  LittleCMS naturally
installs itself in the more 'common' place of '/usr/local/....'  

Simply running the 'configure' script isn't going to do much for you.  It just runs a bunch of tests and configures the build/compile process.  You still ahve to run the compiler which you CANT unless you have the free Apple Developer membership (which gives you access to the 'command line developer toolset' (ie. compiler & linker).

For *maximum* portability, the person who compiled dcamprof should have compiled it *statically*  (ie. no dependencies on external libraries)

Be that as it may, here is a Dropbox link to the liblcms library that is missing...........

https://www.dropbox.com/s/hl9zdnqpoeq3uvf/liblcms2.2.dylib?dl=0

download it and then create a folder (if it doesn't already exist on your machine which it probably doesn't)  /opt/local/lib
then copy the liblcms.2.2.dylib to that new folder.  You will probably be asked to authorize (ie. your password) to do it.

After that, your copy of dcamprof should run.

I just read the beginning part of the thread so HERE IS WHAT YOU'RE LOOKING FOR

because I have no life, here is a just freshly compiled version (0.9.5) of dcamprof that is fully static (no external library dependencies) and compiled for Mac OSX (which doesn't support openmp so it will be a bit slower) (my machine runs 10.10 so don't ask me for others (maybe 10.9 is possible if there is beer involved )

https://www.dropbox.com/s/t86pvyrfl78512r/dcamprof-static?dl=0

[professorbalrog]

because I have no life, here is a just freshly compiled version (0.9.5) of dcamprof that is fully static (no external library dependencies) and compiled for Mac OSX (which doesn't support openmp so it will be a bit slower) (my machine runs 10.10 so don't ask me for others (maybe 10.9 is possible if there is beer involved )

https://www.dropbox.com/s/t86pvyrfl78512r/dcamprof-static?dl=0

ha what do you think I was doing on a Friday night? Yeah...trying to make profiles. I actually was able to figure it out by using homebrew to install all the dependencies that were missing, I wish someone had mentioned that as an option because it's SUPER easy to get argyll and lcms up and running with homebrew. On the plus side, next time something like this comes around I'll have a half-idea what the hell I'm doing.

I'll be taking your compiled version to my poor work computer next week that has libraries from all this stuff in about 30 different folders. whoops. Thanks dude!!!

[professorbalrog]

Added "the easy way" for Capture One users. The same workflow as in the reference doc, but with finished relax recipe and described with some more words:

http://www.ludd.ltu.se/~torger/photography/camera-profiling.html#the_easy_way_c1

finally got everything running properly, thanks to this...

you're my hero. this just made my whole weekend!!!

[hk1020]

Thanks a lot for your reply. So I wasn't completely off base with what I did. Good to know.

I introduced some important fixes of the ICC LUT generation in 0.9.4 so if you didn't use the latest version, please re-run with the new.

I did and the result is almost perfect. The hue shifting is gone, colors look perfect. I did both methods, from ssf and the target you suggested in one of the later replies. The results are almost identical.  This what I did with the ssf:

../dcamprof make-target -c nex5.json -p cc24 target.ti3
../dcamprof make-profile target.ti3 profile.json
../dcamprof tiff-tf -f linear_DSC04159_C11.tif auto_DSC04159_C11.tif tone-curve.json
../dcamprof make-icc -n 'nex5' -f auto_DSC04159_C11.tif -t tone-curve.json profile.json prelim-profile.icm


But here's the catch: When I use the prelim-profile there is still a problem at the low end in the shadows. The shadows are too bright and I can't fix them with the curve.  If I try I get artifacts with colored seams in the dark areas and other strange behavior. I suppose there is still a linearity problem similar to what we had at the bright end. Do you have any idea?  I've put the profile here: https://www.dropbox.com/s/06q08jya3yyk0o1/prelim-0.9.4a-profile.icm?dl=0

This is really promising and I hope this last problem can be ironed out. Thank you very much.

[professorbalrog]

But here's the catch: When I use the prelim-profile there is still a problem at the low end in the shadows. The shadows are too bright and I can't fix them with the curve.  If I try I get artifacts with colored seams in the dark areas and other strange behavior. I suppose there is still a linearity problem similar to what we had at the bright end. Do you have any idea?

Same here with the brightness in the shadows. The improvement in color is SO significant that this is trivial in comparison, but that's the only flaw I can see so far. Also it seems like C1's profiles allow you to make some pretty extreme adjustments to the levels without any serious stepping/posterization but with the custom profile it doesn't seem to tolerate huge adjustments. Again though, I am literally bouncing off the walls at how much better the color is with this method.

[torger]

I did both methods, from ssf and the target you suggested in one of the later replies. The results are almost identical.

This indicates that there's indeed no pre-matrixing done for that camera model, so SSF is working. In my testing I've worked with P45+ files, and those are pre-matrixed, but perhaps it's a thing they only did with a few older models. Let's hope that.

[torger]

I've put the profile here

I've looked at the profile and I can confirm that there is issues with low end shadows. I don't know what the reason is though. I need to replicate the workflow and debug step by step.

EDIT: just managed to replicate shadow issues with a very basic ICC workflow, so consider the too light shadows and other strange effects in that range a bug. You have to live with it for now. I'll start working on a fix, but it may be complicated so it may take a while. We'll see.

[torger]

Found and fixed the bug, v0.9.6 is out now.

The shadow problem should be gone in that version.

The profile will still be less "stable" concerning extreme adjustments than the typical native C1 profile. I think this is because the C1 profiles make some gamut rolloff just to make the profile more robust, but does as as side effect reduce color accuracy for high saturation colors. DCamProf will try to reproduce colors as accurately as possible to the end of the gamut and then just hard-clip, and I think that is what may make it less robust.

Rolling off the gamut in the ICC LUT has been proven to be quite difficult, and I'm not sure yet if it's valuable. Please test and report back. If the C1 profiles is only behaving nicer on super-crazy settings like a white balance that make the image turn all blue and things like that, then I think DCamProf's LUTs can stay the way they are. But if posterization or other strange effects arise within the normal range of adjustments, then it should be fixed of course.

In my own photography I use DNG profiles and RawTherapee on Linux, so the ICC+C1 use case doesn't see much testing from myself, so I appreciate bug reporting.

[hk1020]

Found and fixed the bug, v0.9.6 is out now.

The shadow problem should be gone in that version.

Unfortunately, it isn't. There is almost no difference. I tried it with the ssf in case that matters. Here is the profile again:
https://www.dropbox.com/s/s7sfujjobcbvp71/prelim-0.9.6-profile.icm?dl=0
As it is the profile is unfortunately not usable yet. If you want I could upload pictures showing the effect.

[torger]

Unfortunately, it isn't. There is almost no difference. I tried it with the ssf in case that matters. Here is the profile again:
https://www.dropbox.com/s/s7sfujjobcbvp71/prelim-0.9.6-profile.icm?dl=0
As it is the profile is unfortunately not usable yet. If you want I could upload pictures showing the effect.

If you can upload your target.ti3 you run make-profile on it would be nice.

Can you try add -o standard to make-icc command line? It will disable the neutral tone reproduction operator (and affect color appearance negatively), but if the strange shadow stuff disappears I at least know it's related to the tone reproduction:

../dcamprof make-icc -n 'nex5' -f auto_DSC04159_C11.tif -t tone-curve.json -o standard profile.json prelim-profile.icm

[torger]

The profiles 0.9.4a and 0.9.6 are exactly the same, except the timestamp. Could you double-check that you're running dcamprof 0.9.6 and not the old version?

[hk1020]

If you can upload your target.ti3 you run make-profile on it would be nice.

Here it is: https://www.dropbox.com/s/11bhqdqyqtp6152/target.ti3?dl=0

Can you try add -o standard to make-icc command line? It will disable the neutral tone reproduction operator (and affect color appearance negatively), but if the strange shadow stuff disappears I at least know it's related to the tone reproduction:

../dcamprof make-icc -n 'nex5' -f auto_DSC04159_C11.tif -t tone-curve.json -o standard profile.json prelim-profile.icm

No that has almost no influence. FWIW, 0.9.3 did not have this problem.

[torger]

Here it is: https://www.dropbox.com/s/11bhqdqyqtp6152/target.ti3?dl=0
No that has almost no influence. FWIW, 0.9.3 did not have this problem.

I tested to run you commands (although with tiffs coming from a P45+) with 0.9.5 - got shadow issues, and then 0.9.6, got no shadow issues. That is I get no problem. If I can't reproduce the problem, I can't debug it. I guess I need the other files too then so I can run the exact same commands with the exact same data. When it comes to fixing bugs it's more common than not that one ends up needing all data to reproduce the bug, so as a rule of thumb providing all input files is good.

0.9.3 had a completely different LUT generator, but it had some other issues.

I'll go to bed now, but will look in the morning, but I need the rest of the files to be able to debug further.

[hk1020]

I tested to run you commands (although with tiffs coming from a P45+) with 0.9.5 - got shadow issues, and then 0.9.6, got no shadow issues. That is I get no problem. If I can't reproduce the problem, I can't debug it. I guess I need the other files too then so I can run the exact same commands with the exact same data. When it comes to fixing bugs it's more common than not that one ends up needing all data to reproduce the bug, so as a rule of thumb providing all input files is good.

Thanks a lot.  Here's the data: https://www.dropbox.com/sh/pwx90uiyu40mbox/AACm9z2-_S0iRfvpPZhHuS9Ga?dl=0 The doit.new is the script I used.
The DSC0963* files are just to illustrate what you end up with. The _C1 is stock Capture One export.

I'll go to bed now, but will look in the morning, but I need the rest of the files to be able to debug further.

So do I. see you tomorrow.

[torger]

The DSC0963* files are just to illustrate what you end up with. The _C1 is stock Capture One export.

It would be great if you could lend me that test file with all those black shadows, I don't have many of those kind laying around.

[torger]

Tested the workflow, I don't get the same result. I suspect that your 0.9.6 build actually is an older. Can you run dcamprof without parameters and see what the first line it outputs is? It should say "DCamProf v0.9.6 - a digital camera profiling tool."

I've attached the profile I get when running your script. Let us know if it works better.

[howardm]

my Dropbox link to a Mac OSX compiled version of dcamprof-static has been updated to v0.9.6

[Frederic_H]

The guide seems to recommend filtered halogen lamps for the D50 illuminant and don't say much about studio strobes. Any reason not to use them for creating ICC profiles ?

[torger]

The guide seems to recommend filtered halogen lamps for the D50 illuminant and don't say much about studio strobes. Any reason not to use them for creating ICC profiles ?

Studio strobes are good, very good spectrum. The light temperature can vary a bit shot to shot but I can't really figure out any case where that would be a real problem in practice.

Studio strobes often light up a lot around the target so you may possibly get more issues with flare. With flags I guess you could minimize the stray light though.

Had I been a studio pro I'd probably would have shown a flash example. I like continuous light as it's user friendly to set up, and you can then also have the scene available for viewing "by eye" to make comparisons between screen and real objects. It's also easy to measure the light spectrum as it's continuous light. I don't think I can measure a flash with my Colormunki.

In terms of profile quality though studio strobes should be just as good as halogen. The halogen I suggest for D50, the Solux on overdrive is also a bit controversial, some are suspicious about it's quality and have had bad results, so I know some people prefer flash over halogen. My results with the Solux have been excellent though so that's why I'm recommending it.

[Frederic_H]

If a precise light temperature reading is important I could still make a spotread with our spectro when we shoot the chart, unless it's overkill ?

As for flare, we'll probably manage to deal with it using the right modifiers and checking the raw histograms.
Regarding those white and black patches I read an article suggesting synthetic ones be added to ti3 files. It features some other advices too, and made me wonder if they'd apply to Dcamprof as well ?

http://ninedegreesbelow.com/photography/well-behaved-camera-profile.html