Pages: 1 ... 66 67 [68] 69 70 ... 78   Go Down

Author Topic: DCamProf - a new camera profiling tool  (Read 719889 times)

howardm

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1984
Re: DCamProf - a new camera profiling tool
« Reply #1340 on: September 27, 2016, 11:33:17 am »

I may have to actually use dcamprof.  I have a Fuji XT2 and took some images of British Redcoats (and they are VERY red) and they really can use some help.

Tim Lookingbill

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2436
Re: DCamProf - a new camera profiling tool
« Reply #1341 on: September 27, 2016, 01:38:09 pm »

This thread certainly got more interesting.

From editing the AdobeStrong version of the red pants image in ACR 6.7 I can't help but think there's got to be some issues caused by the display profile (or video hardware) mucking up a camera profile's ability to properly visually map luminance/saturation to the display that may have nothing to do with color gamut spaces.

Since the red pants image is in sRGB (edited on an sRGB-ish display) I still could make the highlights appear brighter but noticed each slider in ACR 6.7 raw converter acted upon the preview in different ways where even point curve edits created a wack-a-mole response between increasing luminance vs saturation. I can see how a LUT based profile can cause issues as well. The point curve edits required a raised flat, mesa shaped (not arched) curve segment for brightening the red pants highlights in order to retain definition (override compression effect) and yet it's in low gamut sRGB.

And the Lab L* channel gives the same highlight readouts on all three versions posted. Clearly there's also a disconnect between how the numbers define a human perceptual appearance model and reality.

Good work, Anders, I really learned a quite a bit in this thread.
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #1342 on: September 29, 2016, 04:15:07 am »

Just released v1.0.3. I had accidentally broke curve parsing in v1.0.2 so this is a pure bugfix release. Without it it's not possible to complete a full Capture One workflow with custom curves, so it's an important fix.

http://www.ludd.ltu.se/~torger/dcamprof.html#download
Logged

markanini

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 58
Re: DCamProf - a new camera profiling tool
« Reply #1343 on: September 29, 2016, 04:23:14 pm »

I seems canons picture style format is well on it's way to getting hacked http://magiclantern.fm/forum/index.php?topic=16299.125 I would be awesome if you could use DCamProf generated profiles directly in camera.
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #1344 on: September 30, 2016, 02:55:47 am »

I seems canons picture style format is well on it's way to getting hacked http://magiclantern.fm/forum/index.php?topic=16299.125 I would be awesome if you could use DCamProf generated profiles directly in camera.

If the "LUT3D" field really provides a LUT and it's not only about simple adjustments on top of fixed base profiles it could be possible to load profiles there. However I wouldn't put any massive effort into it, but say if the magic latern project provides a way to load straight ICC profiles into that LUT so it's simple to generate a profile for it I could do it.

It sure would be interesting and it's really how cameras should work -- it's a mystery how closed cameras still are. That you really can't load third-party profiles into the camera is just odd, it should have been possible since cameras became digital but the camera industry is just so incredibly conservative.
Logged

Doug Gray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2117
Re: DCamProf - a new camera profiling tool
« Reply #1345 on: September 30, 2016, 03:34:49 pm »

Outstanding stuff Anders. And your explanations are both clear and show a deep understanding of color science. It's also very useful for those of us that need to do repro work from time to time.
Logged

scyth

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 584
Re: DCamProf - a new camera profiling tool
« Reply #1346 on: October 01, 2016, 12:37:06 am »

Just released v1.0.3. I had accidentally broke curve parsing in v1.0.2 so this is a pure bugfix release. Without it it's not possible to complete a full Capture One workflow with custom curves, so it's an important fix.

http://www.ludd.ltu.se/~torger/dcamprof.html#download

I have a question... I did not make pure matrix profiles for some (long) time and it seems I forgot/miss something obvious - why I can't get rid of "LookTable" in dcp profile unless I put '-h 1,1,1' as a parameter for 'dcamprof make-dcp' ? I think I remember that it used to be that '-L' parameter alone was enough, no ?

Thank you !
Logged

markanini

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 58
Re: DCamProf - a new camera profiling tool
« Reply #1347 on: October 01, 2016, 02:04:58 pm »

If the "LUT3D" field really provides a LUT and it's not only about simple adjustments on top of fixed base profiles it could be possible to load profiles there. However I wouldn't put any massive effort into it, but say if the magic latern project provides a way to load straight ICC profiles into that LUT so it's simple to generate a profile for it I could do it.

It sure would be interesting and it's really how cameras should work -- it's a mystery how closed cameras still are. That you really can't load third-party profiles into the camera is just odd, it should have been possible since cameras became digital but the camera industry is just so incredibly conservative.

I agree cameras are excessively closed down. I think these efforts could open a lot of doors for different workflows concerning repro work. RAW is still king but if you can give it up ML's silent shutter can be used to minimize shutter wear for timelapses and astro or increase the speed of motion detection. In conjunction with a DCamProf generated profile it make a real killer feature. Then there's video.
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #1348 on: October 02, 2016, 02:52:49 am »

I have a question... I did not make pure matrix profiles for some (long) time and it seems I forgot/miss something obvious - why I can't get rid of "LookTable" in dcp profile unless I put '-h 1,1,1' as a parameter for 'dcamprof make-dcp' ? I think I remember that it used to be that '-L' parameter alone was enough, no ?

Thank you !

Yes it should only be '-L', but that parameter has apparently become broken. I'll fix to next release. Meanwhile you can get the original behavior by adding '-L -o standard'.
« Last Edit: October 02, 2016, 03:00:15 am by torger »
Logged

sebbe

  • Full Member
  • ***
  • Offline Offline
  • Posts: 113
Re: DCamProf - a new camera profiling tool
« Reply #1349 on: October 02, 2016, 08:30:55 am »

Anders, thanks for your great work.

In my oppinion the handling of the brights and darks with a DCamprof C1-Profile was already very well. The new function may be useful in some situations. I will use it, when the other profile has issues.
Here are some comparisons.
(note: the (linear curved) C1-profile was made with an IT8/7.2 target in sunlight (d50, dEmax5.16, dEavg1.65)). Maybe there are differences in the gamut compression to other targets)
« Last Edit: October 02, 2016, 08:56:51 am by sebbe »
Logged

sebbe

  • Full Member
  • ***
  • Offline Offline
  • Posts: 113
Re: DCamProf - a new camera profiling tool
« Reply #1350 on: October 02, 2016, 08:31:41 am »

and two more...
I pushed the saturation a little bit on 3strong.jpg to be comparable to 3normal.jpg (except in the reds/yellows).
« Last Edit: October 02, 2016, 09:20:35 am by sebbe »
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #1351 on: October 02, 2016, 10:34:51 am »

Thanks for the example images Sebbe. For anyone that wants to compare I recommend to download the images and make a A/B swap in place, it's always hard to see smaller differences side by side.

What can be seen is that the main effect of the stronger gamut compression is darkening of certain high saturation colors, unless the color is clipped in raw, then there's not much darkening as the color is already clipped and there's nothing to recover. This can be seen in the out of focus green blob for example, it keeps the brightness while the yellow unclipped blobs are darkened.

The gamut compression also have a shadow lightening effect, which is most notable in the first example images with the deep dark blues, but if you look carefully you can see it in the sewing spools image especially on the purple threads.

One interesting effect I hadn't thought about is also seen in the purple spools -- the stronger gamut compression actually increases saturation in them slightly. The reason for this is that the HSV preprocessing step darkens/lightens the color so the follow-on CIECAM02 compression gets more room to operate on and needs to compress less. That is you get more of lightness compression than a chroma compression.

I'm pretty pleased myself with the result, but something tells me that it's not the last time I'll look into gamut compression algorithms.
Logged

scyth

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 584
Re: DCamProf - a new camera profiling tool
« Reply #1352 on: October 03, 2016, 03:22:45 pm »

Certainly not stupid question, I've also been through all THE INTERNET and it's a mess with all different compiler versions and distributions out there. I was actually surprised that it worked statically with MSYS2/MinGW64, it was not something I actively searched for, it just worked. My plan was to distribute the DLL together with the binary just as I do with OS X now, but it turned out to not be required as there's a libgomp.a with the MSYS2 distribution.

Makefile is the same as the distributed Linux makefile, but with the added "-static" option to LDFLAGS and added "-ljpeg -llzma -lz" on the libraries (DCAMPROF_LIBS), then built inside a mingw64_shell that comes with MSYS2 (if you install it using the package management pacman). I don't know exactly which packages I have installed as the machine used for compiling is used for other projects too, and MSYS2 was not installed for this.

btw static build for windows worked - I had only to mod the makefile from your distribution to account for the library names

$(CC) -o $@ $(LDFLAGS) $(DCAMPROF_OBJS) liblcms2.a libtiff.a libgomp.a -lm

why didn't I try before the static .a library that I had all the time in the tdm-gcc distribution just sitting near .dll beats me... well, that is what separates a practising software developer from somebody who was /at least tried to be/ once a long time ago  ;D
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #1353 on: October 09, 2016, 03:04:27 pm »

I'll make a new release tomorrow (I hope) which contains new matrix optimization refinement feature; it will make it much easier than before to control tradeoffs in patch matching, which may be desirable especially for matrix-only profiles. As a LUT when relaxed closes in on the matrix it may be useful also for LUT profiles, for the perfectionist.

I've attached an example for the common CC24 target. First the normal matrix, where DCamProf without any refinement parameters and thus generates a matrix as before, trying to make total sum of errors as small as possible. As you can see the D02 patch has 0 error, as D02 is the most neutral patch in a CC24 and is assigned "D50 white", and as DCamProf's matrices are always "white point preserving" it will always have 0 error, that cannot be changed. DNG profiles require it by definition, and it's a good idea in any case so it can't be turned off.

Then I've added manual refinements using the new feature, to make the following adjustments: exact match on the light skin-tone patch, make sure that blues are not darkened for more stable behavior, and make sure colors are rather pulled away from purples than towards it (reds rather orange than purple, blues rather cyan than purple). It's specified like this:

dcamprof make-profile -L -v A02 0 -v C01 0,2,-1,1,-2,0 -v C03 -3,3,-3,3,0,3 -r dump1 cc24.ti3 test-profile.json

A02 0: exact match, DE 0.
C01 0,2 lightness range (up to 2DE lighter, but no darker), chroma range -1,1 and hue rather counter-clockwise (towards cyan) and absolutely not towards purple, same for red.

The tricky thing with matrices is that everything is interconnected, so if you make one patch match better, someone else must become worse. Quite easily you refine too hard so there's no solution possible, and then DCamProf will fail with an error, so it's a trial and error process. A quite simple one though.
Logged

Doug Gray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2117
Re: DCamProf - a new camera profiling tool
« Reply #1354 on: October 09, 2016, 04:30:34 pm »

I'll make a new release tomorrow (I hope) which contains new matrix optimization refinement feature; it will make it much easier than before to control tradeoffs in patch matching, which may be desirable especially for matrix-only profiles. As a LUT when relaxed closes in on the matrix it may be useful also for LUT profiles, for the perfectionist.

Excellent! Like you were reading my mind!
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #1355 on: October 10, 2016, 04:14:38 am »

v1.0.4 is now released. The key new feature is better manual control of the matrix optimizer, especially useful for those making matrix-only profiles, but can also be useful to the perfectionist making LUT profiles as the matrix is the linear base the LUT is drawn to when relaxed, so it's always good to have the matrix as close as possible to the desired end result. As before -- for casual use the default automatic mode is good enough, so the manual controls are for advanced users.

News:
http://www.ludd.ltu.se/~torger/dcamprof.html#news

Downloads:
http://www.ludd.ltu.se/~torger/dcamprof.html#download
Logged

scyth

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 584
Re: DCamProf - a new camera profiling tool
« Reply #1356 on: October 11, 2016, 01:35:40 pm »

v1.0.4 is now released. The key new feature is better manual control of the matrix optimizer, especially useful for those making matrix-only profiles, but can also be useful to the perfectionist making LUT profiles as the matrix is the linear base the LUT is drawn to when relaxed, so it's always good to have the matrix as close as possible to the desired end result. As before -- for casual use the default automatic mode is good enough, so the manual controls are for advanced users.

News:
http://www.ludd.ltu.se/~torger/dcamprof.html#news

Downloads:
http://www.ludd.ltu.se/~torger/dcamprof.html#download

thank you and, this was, I believe, already asked - does it make any sense (I think it does, but) to allow a command line parameter that will make dcamprof to store the whole set of parameters (from that same command line + what was stored on the prev. step in case sequence like make-profile + make-dcp) inside some tags for make-profile (that shall be easy as it is your own json), make-dcp (may be in "ProfileCopyright" tag ?), make-icc ? so that later you can (if you wish) peek how did you create this profile of yours in the first place...
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #1357 on: October 12, 2016, 04:10:43 am »

thank you and, this was, I believe, already asked - does it make any sense (I think it does, but) to allow a command line parameter that will make dcamprof to store the whole set of parameters (from that same command line + what was stored on the prev. step in case sequence like make-profile + make-dcp) inside some tags for make-profile (that shall be easy as it is your own json), make-dcp (may be in "ProfileCopyright" tag ?), make-icc ? so that later you can (if you wish) peek how did you create this profile of yours in the first place...

Yes that sounds like a good idea, I'll consider it.
Logged

scyth

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 584
Re: DCamProf - a new camera profiling tool
« Reply #1358 on: October 12, 2016, 07:58:38 pm »

v1.0.4 is now released. The key new feature is better manual control of the matrix optimizer, especially useful for those making matrix-only profiles, but can also be useful to the perfectionist making LUT profiles as the matrix is the linear base the LUT is drawn to when relaxed, so it's always good to have the matrix as close as possible to the desired end result. As before -- for casual use the default automatic mode is good enough, so the manual controls are for advanced users.

News:
http://www.ludd.ltu.se/~torger/dcamprof.html#news

Downloads:
http://www.ludd.ltu.se/~torger/dcamprof.html#download

Anders, can you shed some light on the following :

1) assume we have some SSF (ssf.json file) - it does not matter whether it is a for a real camera or we just invented some curves resembling some good or so-so data posted wherever at this moment

2) let is make a very simple synthetic (as we have SSF) target (with further pure matrix profile in mind, hence CC24 - we don't really need tons of patches for a pure matrix profile) : dcamprof make-target -c ssf.json -i spectrum -C -p cc24 ssf.ti3  , where spectrum will be D65, then D50, then StdA... see below why.

3) let us make dcamprof profile (with further pure matrix profile in mind, hence -L for example) : dcamprof make-profile -c ssf.json -i spectrum -C -B -L -r .\report ssf.ti3 profile.json

now the question - why in the world when I move from D65 to D50 to StdA I am getting the results significantly worse and worse with each "lower-K" spectrum used for forward matrix (as reported by dcamprof) ? is it how is should be (like because StdA shape is very deficient energy-wise in "blue" part of the spectrum,  but why so noticeable difference between D50 and D65 ?) or is it some bug or is it because of a particular shape of SSF curves or am I doing something stupid here ?
Logged

Alexey.Danilchenko

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 257
    • Spectron
Re: DCamProf - a new camera profiling tool
« Reply #1359 on: October 13, 2016, 04:28:28 am »

let is make a very simple synthetic (as we have SSF) target (with further pure matrix profile in mind, hence CC24 - we don't really need tons of patches for a pure matrix profile)

From personal experience building pure matrix profiles from real camera SSFs, CC24 is not enough for a good pure matrix profile. I managed to generate a very good matrix profile with CC SG set plus some from Munsell set with slow matrix optimisation on (-s).
Logged
Pages: 1 ... 66 67 [68] 69 70 ... 78   Go Up