Pages: [1] 2 3 ... 5   Go Down

Author Topic: DCamProf for dummies?  (Read 56371 times)

professorbalrog

  • Newbie
  • *
  • Offline Offline
  • Posts: 11
DCamProf for dummies?
« on: September 03, 2015, 12:45:20 pm »

Would any of you who are smarter than me using Argyll and DCamProf on a Mac be willing to help me out with it, or point me in the right direction? I'm not at all scared of command-line utilities but I feel like I have no idea how to even get these to compile correctly. I was able to get Argyll to run the scanin command based on the instructions I found here (and after moving the chart reference files to the bin folder):

http://www.trumpetpower.com/photos/Exposure#Normalizing_exposure

But that's about it. DCamProf's instructions say "It should also be relatively easy to build on Mac OS X". I have no idea what to do with that information :/ All I want to do is generate a profile that's roughly as accurate as what the X-rite software can create for use in ACR but one that I can use in Capture One instead (so, ICC right?). I don't need to build a custom target or anything crazy (yet) just the most basic of camera calibrations.

Help? Please?
Logged

Redcrown

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 514
Re: DCamProf for dummies?
« Reply #1 on: September 03, 2015, 01:10:51 pm »

I'll add a ditto. I gave up on that other DcamProf thread. Too much effort, too much noise. Been waiting to see if anyone is willing to commercialize it. Also been waiting for someone to show the benefits - side by side comparisons?

I've always been curious why there has been relatively no improvement in camera profiling in years. Xrite is still on version 1 after several years. Adobe DNGPE is also mostly unchanged since initial release. And no camera makers offer a product, which is strange.

I assume the market for camera profiling is just too small. Looks like there are only about 15 people in the original DcamProf thread. There may be a few hundred, maybe even a thousand others out there who are potential users/customers.
Logged

AlterEgo

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1995
Re: DCamProf for dummies?
« Reply #2 on: September 03, 2015, 01:20:21 pm »

Would any of you who are smarter than me using Argyll and DCamProf on a Mac be willing to help me out with it, or point me in the right direction? I'm not at all scared of command-line utilities but I feel like I have no idea how to even get these to compile correctly.

ask for a build in the main topic = I share my builds for Windows : 0.9.4 build for Windows (mingw = dcamprof.exe + libgomp_64-1.dll + HTML & PDF manual / = copy of Torger's web page and the same converted to PDF /) : https://app.box.com/s/hxy4q0rzi59jhxdv4aqrefl5gd1p8xu1

somebody likewise can share for OSX
Logged

AlterEgo

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1995
Re: DCamProf for dummies?
« Reply #3 on: September 03, 2015, 01:23:11 pm »

Logged

professorbalrog

  • Newbie
  • *
  • Offline Offline
  • Posts: 11
Re: DCamProf for dummies?
« Reply #4 on: September 03, 2015, 01:35:05 pm »

I'll tell 'em you sent me. Thanks!
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf for dummies?
« Reply #5 on: September 03, 2015, 03:28:44 pm »

I've just released a tutorial of how to use DCamProf to make a camera profile:

http://www.ludd.ltu.se/~torger/photography/camera-profiling.html

I don't know if it can be called "DCamProf for dummies", as it's a quite comprehensive workflow, but it should make it easier to get into profiling than having only the reference documentation.

Concerning binaries I personally don't make or link them currently as I'm too lazy to maintain that. As seen there's some really helpful people in this forum though that makes builds.

When DCamProf reaches version 1.0 "general availability" I might make builds available directly on the home page. It's at version 0.9.5 now and I'm quite satisfied with the feature set, so I think it's really close to 1.0 now.
Logged

professorbalrog

  • Newbie
  • *
  • Offline Offline
  • Posts: 11
Re: DCamProf for dummies?
« Reply #6 on: September 03, 2015, 03:35:27 pm »

Awesome, thanks so much!! Really appreciate all the effort you've put into doing this. Looking forward to where you take it next
Logged

hk1020

  • Newbie
  • *
  • Offline Offline
  • Posts: 19
Re: DCamProf for dummies?
« Reply #7 on: September 03, 2015, 06:04:34 pm »

Would any of you who are smarter than me using Argyll and DCamProf on a Mac be willing to help me out with it, or point me in the right direction? I'm not at all scared of command-line utilities but I feel like I have no idea how to even get these to compile correctly. I was able to get Argyll to run the scanin command based on the instructions I found here (and after moving the chart reference files to the bin folder):

http://www.trumpetpower.com/photos/Exposure#Normalizing_exposure

But that's about it. DCamProf's instructions say "It should also be relatively easy to build on Mac OS X". I have no idea what to do with that information :/ All I want to do is generate a profile that's roughly as accurate as what the X-rite software can create for use in ACR but one that I can use in Capture One instead (so, ICC right?). I don't need to build a custom target or anything crazy (yet) just the most basic of camera calibrations.

Help? Please?

I've been in the same boat recently.  Over on the Capture One forum they started a thread about C1's colors and using dcamprof for C1.  I tried for the first time to use dcamprof (never heard of it before) to make a profile for C1 to be used with a Sony NEX-6.  Fortunately, there is spectral data for a Nex-5 so I don't need a test target (I don't have any equipment to measure colors).  My first attempts were quite promising and improved the C1 colors a lot.  Unlike what you read in many forums the C1 colors for Sony are actually pretty bad.  Everything's got a red/brown tint so much so that I don't like to use it.  The colors from the out of camera jpegs are much better and closer to reality. I am not alone with this.  See the thread at the C1 forum here: http://forum.phaseone.com/En/viewtopic.php?f=57&t=20609&sid=1f4cb522e0e7f561246906b1061f1fa4

I asked questions there concerning dcamprof as this other 40+ page thread here in the forum and the instructions are simply inpenetrable for a newbee.

Now the bad thing is I can't get the profile I made work with real images.  The colors are much better than anything C1 otherwise produces but the brightest parts get a red tint.  Also the colors shift if I change brightness in C1.  Has anyone any ideas what might have happened?  This is the list of commands I used:

dcamprof make-target -c nex5.json -p cc24 target.ti3
dcamprof make-target -X -f linear_DSC04159_C11.tif -p target.ti3 new-target.ti3
dcamprof make-profile -c nex5.json new-target.ti3 profile.json

dcamprof tiff-tf -f linear_DSC04159_C11.tif auto_DSC04159_C11.tif tone-curve.json
dcamprof make-icc -n 'nex5' -f auto_DSC04159_C11.tif -t tone-curve.json profile.json prelim-profile.icm
cp prelim-profile.icm /c/Users/michael/AppData/Local/CaptureOne/Color\ Profiles/
 

The tif files I used are from some random picture I took and exported from C1 as 16bit tiff, with embedded camera profile, the "no color correction" profile in C1 and with either linear or automatic tone curve (as indicated in the names above).

I'd be grateful if someone could tell me what I did wrong.  If desired I could show examples or more verbose reports.

I'd really like to get correct colors from C1.

And considering building dcamprof: I use Windows with cygwin. There it is quite straightforward. You need to have libcms2 installed via cygwin's setup.exe first. Then you unpack the dcamporf source and remove -fopenmp in the Makefile just as the instructions say. Then simply make and you are done. Leaves a dcamprof.exe in the build directory.  Move to /usr/local/bin/ or anything in your path, just standard Unix stuff.

Michael
Logged

Redcrown

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 514
Re: DCamProf for dummies?
« Reply #8 on: September 03, 2015, 08:08:34 pm »

no pain no gain, no ?


I see the pain, I don't see the gain. Maybe it's in that long thread somewhere?
Show me some images to compare. One converted with Adobe Standard, one with an Adobe DNGPE custom profile, maybe one with Xrite custom profile, and then one with DCamProf. Point out the differences and why one is better than another.

Maybe consider a service? I'll send you raw images of a 24 patch CC and $$. You send me profiles? Put a price on it.
Logged

Tim Lookingbill

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2436
Re: DCamProf for dummies?
« Reply #9 on: September 03, 2015, 08:32:20 pm »

Looking forward to seeing a side by side comparison with image samples between applying a DCamProf profile versus regular DNG profile method we've all been using with the CC chart Wizard or Xrite's CC Passport software.

Over 30K views on that long thread shows there's quite a few folks who would want to see this.
Logged

AlterEgo

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1995
Re: DCamProf for dummies?
« Reply #10 on: September 03, 2015, 11:54:40 pm »

I see the pain, I don't see the gain. Maybe it's in that long thread somewhere?
it depends on your definition of the gain - for me it was just the ability to approximate SSF/CMF for my camera and use that so that I can build a sufficiently neutral (sufficiently for me) reproduction-type kind of profile - nothing fancy, film-like or exotic... now if I am not getting any color sufficiently right for my liking after that starting point so to say then I know whom to blame and no that's not Adobe's of PhaseOne's profiles  :D ... so what kind of gain you were/are expecting ?
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf for dummies?
« Reply #11 on: September 04, 2015, 02:42:38 am »

There are a few pictures in the tutorial document:
http://www.ludd.ltu.se/~torger/photography/camera-profiling.html

And actually there are a few in the DCamProf reference documentation too:
http://www.ludd.ltu.se/~torger/dcamprof.html#tone_curves

You need a good screen, preferably calibrated, to be able to see differences well.

This is free open-source software so I'm not really into selling anything at this point. Those that want improvement and control over their colors will find the software. I do have some motivational speak in the tutorial introduction though:
http://www.ludd.ltu.se/~torger/photography/camera-profiling.html#why_own_profile

The difference between a custom camera profile and a bundled one is typically smaller than the difference between a custom printer profile and a manufacturer-provided. So one could say it's for "fine tasters" or "perfectionists" or "control freaks" or what you want to call us. I would say that it's "more pain than gain" for the majority of users, especially since camera profiling is a quite difficult task. It's much easier to make profiles for your printer or your screen.

It's very easy to show that Adobe Camera Raw have issues with color accuracy, just as Capture One. Those are the two reference converters I've looked at for DNG processing and ICC processing. It's not that they can't make profiles, it's just their design choice to apply strong looks. Raw converters today in a way simulate film photography and the profile is the film roll, complete with a contrast curve and a look. There is no raw converter today (at least among the big names) that do color the "scientific" way with a scene-referred colorimetric base profile and then do all the appearance modeling separate from the profile. To work with the current raw converters the profile must contain appearance modeling itself.

The question is then why would you want the manufacturer apply their look, when you can be in control of that yourself? I've been in contact with people that have big issues when they've changed camera brands because they can't really replicate the look they want any longer, as they became dependent on the look provided by their old brand. If you always do your own profiles that won't happen.

If you like to, DCamProf allows you to do your own subjective adjustments to the profile. This is the hardest part and does require that you have a decent eye for color, but it can be quite rewarding. The tutorial contains some guidelines of how to work.

I've used digital medium format a few years, and in that segment it's very clear how important well-designed profiles with refined looks are. Most users think it's all about their sensors, but really most MFD sensors are by today's measure mid range performance, but the profiles are really well-designed (and perhaps more importantly matching the taste of MFD users). I've been most impressed with Hasselblad which stay very close to a neutral realistic look, and that is how I make my own profiles using DCamProf.

DCamProf is mainly about making general-purpose profiles, that is one with a curve embedded. In that case there is no such thing as an objectively "better" profile, there are no established models for measuring that. It's all judged by eye. Sure I think my profile for my Hassy even exceeds Hassy's own, but that's because I've designed the profile along my taste. I'm sure there are users that prefer Hassy's look. If you have confidence in your own taste it can be very rewarding to make your own profile because then I'm quite sure you can exceed the quality of anything available at the market.

Still the differences will typically be subtle. If someone swaps the profile while I'm not looking on an image I haven't seen the original scene etc, I will probably not detect it. One part of having a custom profile is simply getting confidence in that you get sane color out of your camera.

When it comes to comparing various profiling software, in addition to looking at the image result you can simply look at the feature set and also disassemble the resulting profiles (for example by using DCamProf's dcp2json command). Few allows you to adjust contrast curve, few have an appropriate color appearance modeling to compensate for curve effects, few have flatfield correction and instead simply apply heavy smoothing or just ignore lightness axis, many have very coarse LUTs etc. The manufacturers don't use the consumer packages when they make profiles, they have their own in-house tools.

Comparing the results by eye only on unknown images is a little bit dangerous, as you don't know what you're looking at, you have no reference. For a particular image you may very well end up liking a matrix-only profile with a RGB curve on top the most. You should look at images you know, preferably ones you have very recently shot yourself, and you should have an idea of how you like colors to be reproduced.

I've written a guide of how to profile your screen, and then how to profile your printer. Then in February/March this year I was about to write the final article, that on how to profile your camera. Then I discovered that there actually were no software that could do it properly, so I had to write my own. I naively assumed it would be a quick hack job, but here I am six months and 20000 lines of code later.
« Last Edit: September 04, 2015, 03:24:47 am by torger »
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf for dummies?
« Reply #12 on: September 04, 2015, 03:48:58 am »

I'd be grateful if someone could tell me what I did wrong.  If desired I could show examples or more verbose reports.

I introduced some important fixes of the ICC LUT generation in 0.9.4 so if you didn't use the latest version, please re-run with the new.

Capture One is pretty messy to make a profile for, much more difficult than a DNG profile. What makes C1 difficult is the built-in pre-processing so the profile don't see the "raw" data.

The key to make a profile is that you need to have the RGB values the profile will be fed with, together with the XYZ reference values they correspond to. With DNG profiles the profile will see the "raw" data, which means that you can generate the RGB values from camera SSF. With Capture One you can't do that because they apply pre-processing.

I see what you try to do, the "dcamprof make-target -X -f linear_DSC04159_C11.tif -p target.ti3 new-target.ti3" does cancel out the curve preprocessing, but there is typically more, probably some pre-matrixing. This seems to differ between models, so maybe your model is without pre-matrixing, I don't know. The cameras I've tested in Capture One has some pretty heavy pre-matrixing which makes it impossible to make profiles from camera SSF. EDIT: the make-target -X .... command makes no sense in your workflow as we start with SSF, see post further down.

If one could find out the matrix (and any other additional pre-processing if any) one could revert that as well just as the curve, but DCamProf currently has no such functionality. It would require some more reverse-engineering work of C1. DCamProf can revert the curve so it can process color checker shots produced by C1, but you can't use SSFs, unfortunately. Unless your particular camera doesn't receive any pre-matrixing. The typical result if there is pre-matrixing is that your SSF profile will be over-saturated, colors can look sort of right anyway as whitepoint will be correct etc.

I suspect that the matrix embedded in many of the Lab-LUT ICC profiles C1 have actually is the pre-matrix, but I haven't verified if that is the case. Having a matrix in the profile makes no sense as only the LUT will be used, so I've figured that it's there for some sort of informational purpose. I've noted that many of the newer profiles don't have any matrix embedded, and when generating a new profile you don't need to include the matrix, so it's hard-coded. If the newer profiles don't apply any pre-matrixing or if they only do hard-coded I don't know. If the matrix is only hard-coded in C1 it will of course be very difficult for DCamProf to get it.

If anyone reading this knows anything about C1's pre-matrixing, please let us know. If it really is only a pre-matrix and it possible to figure out what it is, I'll gladly implement the feature in DCamProf. Reverse-engineering C1 is not my favorite task though, it's pretty frustrating spending a lot of time figuring out one specific software's inner workings just because there is no standard. The DNG pipeline is at least pretty well documented and has published open source reference code.

The problems you mention "the colors are much better than anything C1 otherwise produces but the brightest parts get a red tint.  Also the colors shift if I change brightness in C1" indicate some other issue though, possibly a bug in the LUT creation. It would be interesting to further investigate that. If you haven't generated your ICC with the latest version, please generate it again, and if the problem remains I'd love to see the profile so I can debug the problem.
« Last Edit: September 04, 2015, 10:29:50 am by torger »
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf for dummies?
« Reply #13 on: September 04, 2015, 04:28:33 am »

Sony NEX-6.

I just looked at the bundled Capture One Sony Nex-6 profile, it does not have any matrix elements in it, unlike for example P45+ profile (which I know has pretty strong pre-matrixing). If we're lucky this means that there is no pre-matrixing for Sony NEX-6, and thus reverting the curves would be all needed to make it work in your SSF workflow. If we're not lucky here is some hard-coded pre-matrixing or even worse some other non-linear pre-processing in addition to the curves which means we can't use SSF.

You can try make a profile without a curve first. If the colors seem to look right, the SSF workflow is hopefully working for this camera.

I'd recommend that you also try making a profile the traditional way. If your tint/shift issues disappear when making it the traditional way, there's probably some issue with matching actual camera raw values (as generated by the SSF processing) with the pre-processing C1 does. You're probably the first user that makes a SSF-based profile for C1, I haven't done it myself :).

If you don't have a CC24 target, you can download and process this shot in C1. Follow the work-flow described in the DCamProf reference doc. Use the CC24 target, as the CCSG target in this shot is full of glare.
http://www.imaging-resource.com/PRODS/sony-nex-6/NEX6hVFAI00100.ARW.HTM
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf for dummies?
« Reply #14 on: September 04, 2015, 10:23:01 am »

If we assume that C1 doesn't do any more pre-processing than the curve so we could use SSF processing, then you should do like this:

* dcamprof make-target -c nex5.json -p cc24 target.ti3

now we have a target.ti3 that contains linear raw RGB values simulated from the SSF curves, using D50 illuminant and reflectance spectra from the CC24 target, and the corresponding XYZ reference values. This is all DCamProf needs to make a profile.

We don't need to do the linearization step since we get linear data directly when we do it from SSF rather than getting values from C1.

* dcamprof make-profile new-target.ti3 profile.json

Now we have a profile, that can be converted to both a DNG and ICC. The key to remember here is that in order for DCamProf to generate a profile it must have the true linear raw RGB values, and when we use SSF we get that directly.

From here you go on with the C1-specific stuff, described at http://www.ludd.ltu.se/~torger/dcamprof.html#workflow_icc_c1 :
* dcamprof tiff-tf -f linear_DSC04159_C11.tif auto_DSC04159_C11.tif tone-curve.json (extract the tone curve)
* dcamprof make-icc -n 'nex5' -f auto_DSC04159_C11.tif -t tone-curve.json profile.json prelim-profile.icm (make preliminary profile)
* cp prelim-profile.icm /c/Users/michael/AppData/Local/CaptureOne/Color\ Profiles/
* run C1 and design the modifier curve, put it in modifier-curve.json
* dcamprof make-icc -n 'nex5' -f auto_DSC04159_C11.tif -t tone-curve.json -t modifier-curve.json profile.json profile.icc (make final profile with modifier curve)

Your described workflow happens to work anyway because dcamprof make-profile -c nex5.json new-target.ti3 profile.json will overwrite the existing RGB values with new generated from the SSF (since "-c nex5.json" is provided), so the incorrect values generated in the dcamprof make-target -X -f linear_DSC04159_C11.tif -p target.ti3 new-target.ti3 are replaced. That step should only be performed when target.ti3's RGB values come from C1 directly (and are thus non-linear), not from SSF (which are linear).
« Last Edit: September 04, 2015, 10:26:51 am by torger »
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf for dummies?
« Reply #15 on: September 04, 2015, 10:32:34 am »

Did some quick look at a number of C1 profiles. Some have a matrix embedded, some have not. The matrix is always the same. If we compare IQ260 (has matrix) and IQ280 (no matrix) which have roughly the same color response we can conclude that they use the same pre-processing. Did some quick tests by preprocessing with the matrix, no sane result. So I have no idea what that matrix is for.
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf for dummies?
« Reply #16 on: September 04, 2015, 06:10:31 pm »

Added a "for dummies" section :-), a way to make a perfectly okay profile without having to make any manual stuff, not even shooting anything if your camera is represented on Imaging Resource for example:

http://www.ludd.ltu.se/~torger/photography/camera-profiling.html#the_easy_way

Logged

professorbalrog

  • Newbie
  • *
  • Offline Offline
  • Posts: 11
Re: DCamProf for dummies?
« Reply #17 on: September 04, 2015, 06:38:33 pm »

OK so emphasis on the dummy over here...I'm making progress here but when I try to run step 4 (as far as I've gotten) in the ICC profile instructions, i get this error:

dyld: Library not loaded: /opt/local/lib/liblcms2.2.dylib
  Referenced from: /Applications/Argyll_V1.8.0/bin/dcamprof
  Reason: image not found
Trace/BPT trap: 5

I exported from Capture one (following your directions) so I assume this step is necessary...I'm so close...I tried downloading little cms and executing the configure file but i guess that doesn't work? so close.....
« Last Edit: September 04, 2015, 06:40:33 pm by professorbalrog »
Logged

AlterEgo

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1995
Re: DCamProf for dummies?
« Reply #18 on: September 04, 2015, 09:57:16 pm »

OK so emphasis on the dummy over here...I'm making progress here but when I try to run step 4 (as far as I've gotten) in the ICC profile instructions, i get this error:

dyld: Library not loaded: /opt/local/lib/liblcms2.2.dylib
  Referenced from: /Applications/Argyll_V1.8.0/bin/dcamprof
  Reason: image not found
Trace/BPT trap: 5

I exported from Capture one (following your directions) so I assume this step is necessary...I'm so close...I tried downloading little cms and executing the configure file but i guess that doesn't work? so close.....


you probably better off asking the author of the OSX build (Iliah) in the main dcamprof topic here...
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf for dummies?
« Reply #19 on: September 05, 2015, 06:29:54 am »

Added "the easy way" for Capture One users. The same workflow as in the reference doc, but with finished relax recipe and described with some more words:

http://www.ludd.ltu.se/~torger/photography/camera-profiling.html#the_easy_way_c1
Logged
Pages: [1] 2 3 ... 5   Go Up