Pages: 1 ... 67 68 [69] 70 71 ... 78   Go Down

Author Topic: DCamProf - a new camera profiling tool  (Read 719890 times)

scyth

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 584
Re: DCamProf - a new camera profiling tool
« Reply #1360 on: October 13, 2016, 09:21:34 am »

From personal experience building pure matrix profiles from real camera SSFs, CC24 is not enough for a good pure matrix profile.

OK, but the core question is why we are getting significantly worse results for forward matrix (as reported by DCamProf) when we are moving illumination from D65 to D50 to StdA with each step ?


I managed to generate a very good matrix profile with CC SG set plus some from Munsell set with slow matrix optimisation on (-s).

just because there were more patches or more patches with specific hue/sat that were driving DCamProf into certain direction while making 3x3 matrix transform ? I did not see any reasonable improvement with "-s" with CC24 spectral data embedded in DCamProf code... and how you will define "good" in terms of numbers (d*-whatever = average, mean, max) in your example ?


attached dcamprof report (left w/o "-s", right "-s") - I'd not call "-s" beneficial in this particular case

PS: replaced .tif with .jpg

« Last Edit: October 13, 2016, 09:33:24 am by scyth »
Logged

scyth

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 584
Re: DCamProf - a new camera profiling tool
« Reply #1361 on: October 13, 2016, 09:26:18 am »

and ssf.json used attached
Logged

Alexey.Danilchenko

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 257
    • Spectron
Re: DCamProf - a new camera profiling tool
« Reply #1362 on: October 13, 2016, 10:57:04 am »

just because there were more patches or more patches with specific hue/sat that were driving DCamProf into certain direction while making 3x3 matrix transform ? I did not see any reasonable improvement with "-s" with CC24 spectral data embedded in DCamProf code... and how you will define "good" in terms of numbers (d*-whatever = average, mean, max) in your example ?

I tried -s with CC24 as well - it generally had better results. In that particular case I had a target ideal matgrix profile to compare it to (which was built by someone else and fine tuned through substantial database of photos) - the one I built from CC SG matched it almost perfectly (if comparing gamut). Every single one I built from CC24 patches alone had problems in violets/blues at least (I have not investigated all other differences in detail).
Logged

scyth

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 584
Re: DCamProf - a new camera profiling tool
« Reply #1363 on: October 13, 2016, 11:15:18 am »

In that particular case I had a target ideal matrix profile to compare it to (which was built by someone else and fine tuned through substantial database of photos)

one can assume that you were trying to match IB's matrix profile in RPP for your kodak back then (why 'd otherwise you invest time & effort in that exercise ?)... but then one can assume that IB did profile himself from (some copy of) CCSG target raws from a similar back and you built monochromator to measure CFA SSFs and then you reasonably shall get close to IB's matrix profile if you are using similar target data but with measured SSFs, no ?

Logged

Alexey.Danilchenko

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 257
    • Spectron
Re: DCamProf - a new camera profiling tool
« Reply #1364 on: October 14, 2016, 06:06:22 am »

one can assume that you were trying to match IB's matrix profile in RPP for your kodak back then (why 'd otherwise you invest time & effort in that exercise ?)... but then one can assume that IB did profile himself from (some copy of) CCSG target raws from a similar back and you built monochromator to measure CFA SSFs and then you reasonably shall get close to IB's matrix profile if you are using similar target data but with measured SSFs, no ?

Not in this particular case - the profile was for Kodak SLR/n.

To answer "why" - to see how easily I can build good quality profiles from SSFs. Iliah's profile in that case was built from CC24 + quite a few iterations rebuilding it, colour correcting the set of images (from what I understood).

I did a few other builds  for different SSFs I took - with pre-canned Kodak profiles, and without comparing it to any profiles but testing the outcome on a set of photos and was getting reasonably close with the patchsets substantially larger than CC24. Have not built satisfactory profiles from CC24 yet. I only use matrix for camera profiles (not really interested in LUTs) hence my interest in getting the best matrix I can from the SSFs.
« Last Edit: October 14, 2016, 08:37:25 am by Alexey.Danilchenko »
Logged

markanini

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 58
Re: DCamProf - a new camera profiling tool
« Reply #1365 on: October 14, 2016, 07:13:41 am »

Did anyone experiment with virtual targets other than CC24 for LUT profiles?
« Last Edit: October 14, 2016, 07:16:48 am by markanini »
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #1366 on: October 15, 2016, 05:16:48 am »

That StdA results get worse than D50 is expected. I don't have all reasons for that fully analyzed but I think one part is that camera SSFs are less suited to separate colors with low temperature illuminants, but I suspect that simply because the spectrum is so slanted colors get harder to match in general.

That D50 is worse than D65 I find surprising, but I need to see how much worse worse is. If it's 0.5 DE here and there I'd say it's about that particular cameras SSFs, if it's huge differences on specific colors I'd suspect some sort of error in the process or bug. I'm travelling for the moment so I don't have that good ability to do testing on my own.

Anders, can you shed some light on the following :

1) assume we have some SSF (ssf.json file) - it does not matter whether it is a for a real camera or we just invented some curves resembling some good or so-so data posted wherever at this moment

2) let is make a very simple synthetic (as we have SSF) target (with further pure matrix profile in mind, hence CC24 - we don't really need tons of patches for a pure matrix profile) : dcamprof make-target -c ssf.json -i spectrum -C -p cc24 ssf.ti3  , where spectrum will be D65, then D50, then StdA... see below why.

3) let us make dcamprof profile (with further pure matrix profile in mind, hence -L for example) : dcamprof make-profile -c ssf.json -i spectrum -C -B -L -r .\report ssf.ti3 profile.json

now the question - why in the world when I move from D65 to D50 to StdA I am getting the results significantly worse and worse with each "lower-K" spectrum used for forward matrix (as reported by dcamprof) ? is it how is should be (like because StdA shape is very deficient energy-wise in "blue" part of the spectrum,  but why so noticeable difference between D50 and D65 ?) or is it some bug or is it because of a particular shape of SSF curves or am I doing something stupid here ?
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #1367 on: October 15, 2016, 05:28:35 am »

I tried -s with CC24 as well - it generally had better results. In that particular case I had a target ideal matgrix profile to compare it to (which was built by someone else and fine tuned through substantial database of photos) - the one I built from CC SG matched it almost perfectly (if comparing gamut). Every single one I built from CC24 patches alone had problems in violets/blues at least (I have not investigated all other differences in detail).

"-s" is makes a brute force optimizer run, it's intended as a sanity checker or debug if one suspects that the normal matrix optimizer does some mistake. As it's brute force it searches the whole space and is thus extremely slow. I have yet not come across any case when the normal optimizer does any big mistake, but the brute force finder will likely reach at a slightly better match, mathematically speaking, but personally I don't think the difference is worth the wait. After the brute force run the normal optimizer is still used for refinement steps, where white point preservation is applied. Refinement means only small changes so it isn't any problem.

Matrix profiles often have problems with violet/blues due to clipping, limiting matrix negative factors with the -y parameter can help that. Possibly by using CCSG you get a similar balancing effect thanks to higher saturation colors, I would expect a bit worse matching of lower saturation colors though.
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #1368 on: October 15, 2016, 05:48:34 am »

Did anyone experiment with virtual targets other than CC24 for LUT profiles?

Yes I've worked with many target types and spectral databases, but it is a while ago as the recent features I've worked with haven't been related to SSF.

In general my view on targets is that for reproduction photography it's probably great to have special-made targets that have patches from the artwork you should copy, but when it comes to making a profile for general-purpose photography I see little need to have large special targets. If you have SSF and a rich spectral database experimenting is free though, there's no harm.

General-purpose photography has so large variability in illuminants, cameras are relatively limited in matching capability, and there's always a need to consider smoothness (that is not over-correct), so larger targets isn't a recipe for "better" profiles. With a LUT profile much of the look sits in the curve and tone operator, gamut compression, and subjective look operators if any.

When I've got feedback about hue "errors", there's almost always been huge errors (5+ DE), and it's not really about some target mismatch or something, but that the person giving feedback has a subjective idea what the color should be, which is not matching the colorimetric fact. So it's not been a target issue. DCamProf supports subjective adjustments too, but it makes it much more difficult to make a profile.

There's an exception range, and that's extreme saturation colors, where cameras get difficulties matching and all sort of technical issues happen in the color pipeline. If one has matching needs there one may need to use special targets and do hand-tuning etc.

This brief target simulation testing can shed some light on what large tartgets contribute:
http://www.ludd.ltu.se/~torger/photography/camera-profiling.html#target_eval
Logged

AlterEgo

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1995
Re: DCamProf - a new camera profiling tool
« Reply #1369 on: October 19, 2016, 05:25:39 pm »

camera SSFs are less suited to separate colors with low temperature illuminants
interesting to see what SSFs imitating something like some CIE 1931/2 Observer will yield in this case
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #1370 on: December 15, 2016, 03:41:42 am »

I've now come far enough in this project to know that it's not going to be vaporware so I thought I could tell about it, although I can't mention a release date. Hopefully Q1 2017, depending how swamped I'll be with other projects.

The project is "Lumariver Digital Camera Profiler", which basically is DCamProf with a GUI. It will be a commercial product, but won't affect DCamProf command line tool which will continue to be free and open-source. With a GUI I hope that DCamProf can reach a broader audience. It's designed to be simple for the basic use, just place the grid on the image and press "Render" and you have a profile ready to use, just as easy as any other profile maker. However there's also lots of tuning possibilities for the advanced user.

Developed with QT it will run on Mac and Windows, and Linux too if anybody except me wants it. The first version will be DNG Profile and CC24 only. If interest proves to be good, I'll probably follow up with a version that supports ICC and custom targets. Development time is substantially shortened with those two limitations though. Originally I didn't intend to make a GUI for look operators, but there will be that too. You can't make all things you can in the DCamProf JSON text format, but most of it.

Here's a feature list (may change!)
  • Profile maker for single or dual-illuminant DNG profiles using the CC24 target
  • Well-picked defaults to make it easy for casual use
  • Built-in reference spectra, or load your own for target and illuminant
  • Glare compensation, flatfield correction
  • Visual manual tuning of reference colors
  • Visual manual tuning of matrix optimization
  • Visual manual tuning of LUT optimization
  • Visual manual tuning of tone curve
  • Choose tone reproduction operator, gamut compression, base look (built-in or load DCamProf look format)
  • Visual look adjustment editor, like a "color editor on steroids"
  • Own project file format to store your profile projects, export to DNG profile as needed
  • Profile comparison mode to make A/B testing on images of your choice
Logged

G*

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 69
Re: DCamProf - a new camera profiling tool
« Reply #1371 on: December 15, 2016, 04:09:36 am »

If that one produces anything I can use with C1, Iím your customer.
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #1372 on: December 15, 2016, 04:29:57 am »

If that one produces anything I can use with C1, Iím your customer.

It probably won't be C1 in the first release, as C1 requires a special form of ICC support, a separate rendering pipeline etc. However DCamProf command line tool can do C1 so it's not unlikely that it will appear later, just like custom target support and maybe some SSF features. First release's first though, much is done, but still much work to do.
Logged

scyth

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 584
Re: DCamProf - a new camera profiling tool
« Reply #1373 on: December 15, 2016, 12:17:31 pm »

It will be a commercial product

how much $ ?
Logged

ErikKaffehr

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 11311
    • Echophoto
Re: DCamProf - a new camera profiling tool
« Reply #1374 on: December 15, 2016, 12:35:17 pm »

Hi Anders,

You have a buyer here!

/Erik

I've now come far enough in this project to know that it's not going to be vaporware so I thought I could tell about it, although I can't mention a release date. Hopefully Q1 2017, depending how swamped I'll be with other projects.

The project is "Lumariver Digital Camera Profiler", which basically is DCamProf with a GUI. It will be a commercial product, but won't affect DCamProf command line tool which will continue to be free and open-source. With a GUI I hope that DCamProf can reach a broader audience. It's designed to be simple for the basic use, just place the grid on the image and press "Render" and you have a profile ready to use, just as easy as any other profile maker. However there's also lots of tuning possibilities for the advanced user.

Developed with QT it will run on Mac and Windows, and Linux too if anybody except me wants it. The first version will be DNG Profile and CC24 only. If interest proves to be good, I'll probably follow up with a version that supports ICC and custom targets. Development time is substantially shortened with those two limitations though. Originally I didn't intend to make a GUI for look operators, but there will be that too. You can't make all things you can in the DCamProf JSON text format, but most of it.

Here's a feature list (may change!)
  • Profile maker for single or dual-illuminant DNG profiles using the CC24 target
  • Well-picked defaults to make it easy for casual use
  • Built-in reference spectra, or load your own for target and illuminant
  • Glare compensation, flatfield correction
  • Visual manual tuning of reference colors
  • Visual manual tuning of matrix optimization
  • Visual manual tuning of LUT optimization
  • Visual manual tuning of tone curve
  • Choose tone reproduction operator, gamut compression, base look (built-in or load DCamProf look format)
  • Visual look adjustment editor, like a "color editor on steroids"
  • Own project file format to store your profile projects, export to DNG profile as needed
  • Profile comparison mode to make A/B testing on images of your choice
Logged
Erik Kaffehr
 

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #1375 on: December 15, 2016, 01:45:37 pm »

how much $ ?

More than Adobe DNG Profile Editor, less than BasICColor Input ;-). Haven't really decided on price yet, but I want to make it reasonable.
Logged

jrp

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 317
Re: DCamProf - a new camera profiling tool
« Reply #1376 on: December 15, 2016, 02:40:54 pm »

Anders, this looks very promising.

A 1-click profile generator with well-picked defaults would be a very good start.

The technical manual tuning would be nice to have, but to make this more than a single-use tool (once you have built a profile for your camera you never use it again) it would be good to make it something that builds a community around it -- eg, people can contribute look operators -- and that doesn't require a deep understanding of illuminants and gamut compression.

The Adobe DNG editor has some useful features, but you can't see what difference your tweaks make and so the workflow with Lightroom / ACR is cumbersome.  Finding a clever visual way of making comparisons between profiles or individual adjustments is the key to success.  For example, being able to tune the DCampProf Natural or Natural+ profile generated profile (relative to the Adobe Standard, say) would be a good start.  (So being able to pick up the profiles installed in your system without copying them and then using them to tune your own profiles would be a good start.  Put another way, concentrate on easy workflow, rather than raw capability would be good.

My 1/2p.  Happy to offer to B test at an appropriate time.
Logged

scyth

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 584
Re: DCamProf - a new camera profiling tool
« Reply #1377 on: December 15, 2016, 02:51:24 pm »

More than Adobe DNG Profile Editor, less than BasICColor Input ;-). Haven't really decided on price yet, but I want to make it reasonable.

consider to make a deal libraw people to bundle that with rawdigger profile edition ...
Logged

jwlimages

  • Full Member
  • ***
  • Offline Offline
  • Posts: 115
Re: DCamProf - a new camera profiling tool
« Reply #1378 on: December 15, 2016, 07:38:24 pm »

This sounds great! Very exciting, Anders. I like the look of the feature list, especially the notion of using my own readings of my CC Passport target. Good luck with getting it up & available soon.
And it's good to hear you may not need to charge as much as BasICColor. ;-)
John
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #1379 on: December 16, 2016, 03:20:40 am »

As a business case, this type of niche software is not so good. It's high complexity in development (requires lot of time to do), and even if you sell well the volumes is not going to be huge. That I make the GUI a commercial product is not really to make it a bread-winner for my company, but to allow me to put in a few extra daytime hours and not just spare time. I like to work, but I no longer have the capacity I had when I was 25.

I think BasICColor has about the only niche that makes sense commercially -- professional reproduction photography. For a pro photographer Ä500 for an efficient workflow tool is not unreasonable (just think about the cost of all hardware you need, and salaries), and for reproduction you really need it. It's low volume market, so the high price is necessary, and software engineering in Germany is not cheap. I don't think it's particularly good at making general-purpose profiles though, which instead is the focus of my product.

There's a reason Adobe doesn't care to sell their profile editor, that X-Rite's profile maker is not actively developed, and that QPCard, which is actually a Swedish company like my own, from the public numbers does not seem profitable enough to even finance one software developer. Camera profile making is not a great niche if you want to make a bunch of money. It is a great niche if you want an enjoyable software development challenge though :-)

Knowing it won't be a great financial success gives me some freedom though, I can make the software the way I like it personally and not put effort into making it sell as much as possible. In other words I won't make the software in a way that I personally don't like even if I think it's a good idea commercially. BasICColor has done the right thing(tm) in terms of design for selling as much as possible -- their software has almost no settings, and that's what the general user wants, something than looks clean and easy and something that just works. While my software will do great profiles with "one click" too, there will still be massive amount of settings and it will look quite technical and that will scare some people away regardless of how simple it will be to make a profile with default settings.
« Last Edit: December 16, 2016, 03:33:10 am by torger »
Logged
Pages: 1 ... 67 68 [69] 70 71 ... 78   Go Up