Pages: 1 ... 45 46 [47] 48 49 ... 78   Go Down

Author Topic: DCamProf - a new camera profiling tool  (Read 769385 times)

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #920 on: September 26, 2015, 04:24:22 pm »

I have now released 0.9.12, with improved smoothing of out of gamut ICC LUT entries, which had some issues before when a curve was used. The jaggies in the LUT curve in Fredric's message a some time back was probably such an effect.

In normal use there should be no difference, but if you have ICC LUT profiles I still recommend to re-render them using 0.9.12

I also added the possibility to control patch set merging in make-target with include/exclude text files (a feature request I got)
Logged

AlterEgo

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1995
Re: DCamProf - a new camera profiling tool
« Reply #921 on: September 26, 2015, 06:08:15 pm »

0.9.12 build for Windows ( mingw = dcamprof.exe + libgomp_64-1.dll + both manual & tutorial / = copies of Torger's web pages / in 3 formats : IE archive .mht, Mozilla archive .maff and regular .pdf ) : https://app.box.com/DCamProf
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #923 on: September 27, 2015, 04:07:11 am »

In normal use there should be no difference, but if you have ICC LUT profiles I still recommend to re-render them using 0.9.12

To clarify, you just need to re-run the last make-icc command of your workflow.
Logged

hk1020

  • Newbie
  • *
  • Offline Offline
  • Posts: 19
Re: DCamProf - a new camera profiling tool
« Reply #924 on: September 27, 2015, 11:34:46 am »

I just built my "standard" profile from the dummies thread again with 0.9.12.  While it is pretty good there are definitely still issues with blue hues. I see it in several pictures and just uploaded an illustrative example here: dcamprof_blue. In there is the raw, OOC jpeg, standard C1 output and one rendered with C1 and dcamprof 0.9.12. I used the preliminary profile without any additional curve.

Note the different blue hues.  The one from dcamprof is pretty much off whereas the other three renderings are fairly consistent. Everything in the blue/violet/magenta range is affected. I am aware the example is not really good at judging what is correct but I don't have something more appropriate at the moment. I hope this is enough to guide you.  If not I can certainly find other examples.
Logged

AlterEgo

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1995
Re: DCamProf - a new camera profiling tool
« Reply #925 on: September 27, 2015, 02:51:10 pm »

I just built my "standard" profile from the dummies thread again with 0.9.12.  While it is pretty good there are definitely still issues with blue hues. I see it in several pictures and just uploaded an illustrative example here: dcamprof_blue. In there is the raw, OOC jpeg, standard C1 output and one rendered with C1 and dcamprof 0.9.12. I used the preliminary profile without any additional curve.

Note the different blue hues.  The one from dcamprof is pretty much off whereas the other three renderings are fairly consistent. Everything in the blue/violet/magenta range is affected. I am aware the example is not really good at judging what is correct but I don't have something more appropriate at the moment. I hope this is enough to guide you.  If not I can certainly find other examples.

while there certainly might be bugs with DCamProf code - how good was your input ? you can find a lot of anecdotal cases of people shooting targets with dress throwing some color reflections on target or @ wrong illumination and/or camera angles to target or whatever... if you want to make the case I 'd suggest post the original source (raw of the target used to build a profile) too ... that's again not to say that you made an error, but if you expect the author to help the more details the better chances... my $0.02
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #926 on: September 27, 2015, 04:35:09 pm »

I just built my "standard" profile from the dummies thread again with 0.9.12.  While it is pretty good there are definitely still issues with blue hues. I see it in several pictures and just uploaded an illustrative example here: dcamprof_blue. In there is the raw, OOC jpeg, standard C1 output and one rendered with C1 and dcamprof 0.9.12. I used the preliminary profile without any additional curve.

Note the different blue hues.  The one from dcamprof is pretty much off whereas the other three renderings are fairly consistent. Everything in the blue/violet/magenta range is affected. I am aware the example is not really good at judging what is correct but I don't have something more appropriate at the moment. I hope this is enough to guide you.  If not I can certainly find other examples.

To be able to help I need the data to be able to make the profile from scratch.

If there's a color bug it's more likely to be related to the tone reproduction. If there's a color error in the profile without any curve, I'd say it's more likely some error in the profile making process (misdetected color values etc) rather than a bug.

I've downloaded the files and looked at them. Unfortunately I'm at my super-old laptop with the worst imaginable TN panel so I don't see colors that well. But I'd say what may be strange is a woman with back towards the camera with a blue jacket which has some red component in DCamProf and green in C1, and balanced in OOC. The pink bags are clipped badly, but that's probably normal, you could try the new gamut mapping feature if you want to compress high saturation colors.

C1 and many other profiles typically render deep blues much too light (see bottom right CC24 patch), probably as a form of gamut compression, so a large lightness difference is usually seen. C1 also gamut compresses more than most, seen on the pink bags which are shaded in white. With gamut mapping feature you can get the same look.

I have noted that deep blues are quite easily pulled into out-of-gamut positions. If something bad is happening there it could be that some colors are pulled out of gamut where there is estimation and smoothing going on. I'd test the same profile as DCP too. But I have much too little data to go on, I need all data and commands for generating the profile.
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #927 on: September 27, 2015, 04:43:28 pm »

I realize you (hk1020) has posted before with some workflow, profile from SSF right? Anyway it's very helpful to post all data each time or a link to an old message where the workflow is, I get quite a lot of "support requests" not only visible here also in private messages I can't really keep track of it all.

EDIT: found some old directory with stuff. I can have a look (probably) tomorrow to see if I can find some problem. There's a risk that it will end with that you need to find a real deep blue object, shoot it under a fixed light, and make a render and compare so you have a real reference to see which rendering that's most accurate. What I do know is that C1 probably is quite off, it usually is especially for saturated colors. OOC probably gamut maps too. I can agree though that there's more than expected going on, so I'll have a look.
« Last Edit: September 27, 2015, 05:00:24 pm by torger »
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #928 on: September 27, 2015, 05:28:15 pm »

As you're making with SSF, try using a much larger target, like this for example:

dcamprof make-target -c nex5.json -p cc24 -p munsell -p adobergb-grid target.ti3

I don't think there is a bug going on, but a CC24 profile will have limited precision in high saturation colors, and that coupled with unrelaxed LUT can pull high saturation colors into poor directions. The pink plastic bags seems to suffer from that, get too dark and too saturated. With a larger target and/or relax this is fixed.

The difference in dark blues I think can be higher accuracy in DCamProf. I'd suggest to make a fixed reference comparison.

It is a problem today that if you make a LUT of a CC24 without relax it the spline effect may continue far outside (towards higher saturation colors) and may mess up a bit. I may look into that, but all profiles should have some relax anyway.
Logged

hk1020

  • Newbie
  • *
  • Offline Offline
  • Posts: 19
Re: DCamProf - a new camera profiling tool
« Reply #929 on: September 27, 2015, 06:38:20 pm »

Thanks for looking into this. Unfortunately, I don't have time at the moment for a more detailed reply. Will do soon.
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #930 on: September 28, 2015, 09:02:52 am »

Looked at it a bit more, and I can see that with the nex5.json SSF the dark blue patch on the CC24 gets a bit more red in it than it should. Making a profile from a CC24 shot you don't get as much red in there, then it's very closely to Adobe's matrix result, and looking at a real CC24 it matches other cameras well too. There might still be a tiny bit too much red in there though, which I think can be that the camera filter response has a tendency towards pulling deep blues in that direction.

So what does this mean? I think the SSF curves you've got has some issues in the blue range. I think I've seen this with other SSF curves for my 5Dmk2 for example, that is you see that it's not as accurate on the deep blue patch of a CC24, and maybe this is due to some limitations in measurement when measuring short wavelengths? Or simply that there may actually be some difference between Nex5 and Nex6 so the SSFs is not actually matching? SSF may seem scentific and all, but a badly measured SSF curve set is certainly not better than a good measured CC24 target.

It may be possible to manually tune the SSF curves to make it match the blue better, but then you're on your own :). Personally I'd play a little bit with that, at least to start with. Otherwise you can base your profile on a SSF target, or make an ugly thing in this case when Adobe seems to have a colorimetric matrix -- steal their matrix :). You don't get a LUT correction but this sensor matches very well with just a matrix (see DxOMark's metamerism index, 85 for this). Stealing the matrix makes an almost exact match with your own CC24 target, but the Adobe matrix seems to be designed with some larger target as it's more stable on extreme saturation, like the pink bags.

But still it seems like the camera has some tendency towards getting a little more red into the deep blues, probably it would be nice to have a target with more deep blues than the CC24 has. It seems like Adobe's NEX5 profile is designed with a colorimetric base matrix, and then their look table reduces red component a bit (manual tuning?).

Conclusions:
 - The SSFs in nex5.json has an issue in the blue range, and cannot be used for good precision there.
 - Not using any relaxation/weighting at all is not recommended, as you can worsen things for colors with higher chroma than the target
 - With a CC24-based profile you get less problems with the blue, but possibly there is still some small amount. This is probably hardware related and you
   need a target with more deep blue and purple patches to get better precision
 - The issues is not due to any bug, but specific to these SSFs and this camera. That is no software changes required.

Note that C1 is known for adding in a good amount of yellow, making saturated reds orange (see the H&M text top of image), and most likely that deep blue is affected too, so I wouldn't use that as a reference.

Solution(s):
 - Use traditional CC24 target, add weighting/relax, residual problems are then small
 - Use a more saturated target like an IT-8 (preferably combine with CC24), better quality
 - Try to manually correct the SSF curves for better deep blue matching
 - Manually correct the blue range with look operators.
 - Steal matrix from Adobe Standard profile (by eye the base matrix seems to have good colorimetric properties), this is the easiest and most fool-proof in
   this case. Note that Adobe's profiles are designed differently profile-to-profile, the base matrix is far from always a colorimetric one like here.

Note that if you copy the Adobe matrix the look of the profile will not at all be like Adobe's profile, as the tone reproduction is totally different. I have attached a native profile with the Adobe D65 matrix you can play with, just run your make-icc commands on that.
« Last Edit: September 28, 2015, 09:13:34 am by torger »
Logged

AlterEgo

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1995
Re: DCamProf - a new camera profiling tool
« Reply #931 on: September 28, 2015, 09:30:56 am »

So what does this mean? I think the SSF curves you've got has some issues in the blue range. I think I've seen this with other SSF curves for my 5Dmk2 for example, that is you see that it's not as accurate on the deep blue patch of a CC24, and maybe this is due to some limitations in measurement when measuring short wavelengths? Or simply that there may actually be some difference between Nex5 and Nex6 so the SSFs is not actually matching? SSF may seem scentific and all, but a badly measured SSF curve set is certainly not better than a good measured CC24 target.
reading the other forum about building a monochromator setup it is clear that it is not that easy to measure SSF directly... so quite possible that source of that SSF (from RIT) didn't make a precise work in that part... somebody (Iliar Borg is a natural suspect) who did measure SSFs a lot can comment on RIT's SSFs - whether their blue curves are good or not
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #932 on: September 28, 2015, 09:40:08 am »

Maybe it's the red curve that's bad that causes too much red when blue is weak (a deep blue color means little green and red, but blue is also weak, so it's quite easy to throw off balance). I haven't really played much with tweaking SSF curves so I cannot provide a precise analysis on this particular color issue.
Logged

AlterEgo

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1995
Re: DCamProf - a new camera profiling tool
« Reply #933 on: September 28, 2015, 10:04:20 am »

Maybe it's the red curve that's bad that causes too much red when blue is weak
indeed... in any case somebody needs (as in it will be nice) to comment about SSFs from RIT vs some other measurments...
Logged

mgrayson

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 67
Re: DCamProf - a new camera profiling tool
« Reply #934 on: September 28, 2015, 11:32:49 am »

Anders,

First, thank you for your efforts! It's quite liberating to separate one's desired "look" from a particular hardware/software choice.

I successfully followed the easy method using the imaging resource target RAW to make a C1 profile for the A7II. The resulting profile looks very good. I'd like to do the same for Lightroom. When I tried it, though, this happened:

/Applications/dcamprof make-dcp -n "SONY ILCE-7M2" -d "My Profile" -t acr my-profile.json my-profile.dcp
Generating HueSatMap with 90x30 = 2700 entries...done!
9 (0.333%) of the HueSatMap entries have hue shift discontinuity with neighbors.
Hue shift discontinuity between LUT entry neighbors are not allowed. Aborting.

Are there parameters earlier in the process I should relax? Any help much appreciated!

Best,

Matt
« Last Edit: September 28, 2015, 11:55:40 am by mgrayson »
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #935 on: September 28, 2015, 12:46:35 pm »

First a comment on hk1020 test picture, while the blue stuff is no bug, the pink plastic shopping bags in the picture shows that I need to rework the recent gamut mapping algorithm... that will be my next priority.
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #936 on: September 28, 2015, 12:53:50 pm »

I successfully followed the easy method using the imaging resource target RAW to make a C1 profile for the A7II. The resulting profile looks very good. I'd like to do the same for Lightroom. When I tried it, though, this happened:

/Applications/dcamprof make-dcp -n "SONY ILCE-7M2" -d "My Profile" -t acr my-profile.json my-profile.dcp
Generating HueSatMap with 90x30 = 2700 entries...done!
9 (0.333%) of the HueSatMap entries have hue shift discontinuity with neighbors.
Hue shift discontinuity between LUT entry neighbors are not allowed. Aborting.

Are there parameters earlier in the process I should relax? Any help much appreciated!

I've actually never succeeding in triggering hue shift discontinuity without using look operators, so this is a first :). What this typically means is that there's some strong stretching going on in the LUT, and that's not good. Could it be that there's some problem with the target shot or patch matching or something? Large Delta E values on the matrix matches (say larger than 4 or so) when running make-profile is suspicious.

There's also a possibility that there's some bug that causes hue discontinuity being triggered despite there are no strong LUT stretches, in that case I would need all your data so I can re-run the workflow and trigger the problem myself.

I made a quick-and-dirty A7r-II profile earlier today, posted in this thread:
http://forum.luminous-landscape.com/index.php?topic=104195.0
Logged

mgrayson

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 67
Re: DCamProf - a new camera profiling tool
« Reply #937 on: September 28, 2015, 08:47:36 pm »

I thought I'd found my mistake, as I was pointed to a wrong file, but pointing to the right file didn't completely fix the problem:

5 (0.185%) of the HueSatMap entries have hue shift discontinuity with neighbors.
Hue shift discontinuity between LUT entry neighbors are not allowed. Aborting.

Ah well. I still have the A7II, not the A7rII, but your profile may be enough incentive to upgrade   ;D

--Matt
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #938 on: September 29, 2015, 04:41:15 am »

I thought I'd found my mistake, as I was pointed to a wrong file, but pointing to the right file didn't completely fix the problem:

5 (0.185%) of the HueSatMap entries have hue shift discontinuity with neighbors.
Hue shift discontinuity between LUT entry neighbors are not allowed. Aborting.

Ah well. I still have the A7II, not the A7rII, but your profile may be enough incentive to upgrade   ;D

If you can share the data in a dropbox or something I can have a look. It seems pretty strange that a normal profile would trigger hue shift discontinuity.

You can force make-dcp to output a profile though by adding the -H parameter.
« Last Edit: September 29, 2015, 07:37:57 am by torger »
Logged

mgrayson

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 67
Re: DCamProf - a new camera profiling tool
« Reply #939 on: September 29, 2015, 08:12:07 am »

Perhaps my error was using the linearized cc24.tif exported from C1 in the earlier workflow. I used Raw Therapee instead and have gotten further.

Alas - the resulting profile turns everything a bright green.

Here are dropbox links to the linear targets saved from C1 and Raw Therapee. They are clearly different. I am completely new to RT, so have no confidence in that sample. I clicked the "output TIFF for calibration" button.
 
https://www.dropbox.com/s/evvtmteueabeopn/C1cc24.tif?dl=0
https://www.dropbox.com/s/qnd4mvr7nleieob/RTcc24.tif?dl=0

As an aside, I had a few days access to a Leica S (007) and compared several dozen captures with the A7II and Canon 1DsII. The Leica and Canon colors were fairly close in LightRoom. The Sony quite yellow/green in comparison, but the use of some freely available profiles moved the Sony colors much closer to the other two. This, more than any amount of online discussion, has convinced me that it IS the profile that makes MOST of the difference between camera looks, optics and sensors being otherwise up to the task. (To be more precise, Leica S(007) with APO 120/2.5, Sony A7II with Minolta 100/2, Canon 1DsII with 24-105/4L , this was for color, not corner sharpness).

Best,

Matt
Logged
Pages: 1 ... 45 46 [47] 48 49 ... 78   Go Up