DCamProf - a new camera profiling tool

[sTi]

A question about including gamut compression in the generated profile.

With Lightroom is it preferable to

(a) include prophotoRGB gamut compression and then get Lightroom to compress down further to sRGB or your printer's RGB profile when exporting/printing; or
(b) include sRGB gamut compression in the profile itself?

Given how large ProPhoto is, it would hardly be a "compression"
The default seems to be compression to AdobeRGB, and this is what I use with my profiles (for Raw Therapee). I also compared it to the same profiles without compression, the differences are visible e.g. for very saturated reds (flowers etc.), which look a bit tamer and less saturated with Adobe RGB compression.

[jrp]

Ok. Thanks. But do they look better if you use prophoto / no gamut compression and use Lightroom to produce AdobeRGB output?

[sTi]

Ok. Thanks. But do they look better if you use prophoto / no gamut compression and use Lightroom to produce AdobeRGB output?

I guess it depends what you mean by better Also, I don't have Lightroom, I only use Raw Therapee, but I guess the trade-offs with using compression are similar in both programs.
Usually I stick to the gamut compression with AdobeRGB because it is better at avoiding oversaturation and blowing out of colors. On the other hand, I also had some images which relied on very saturated, wild colors, these looked better with compression disabled.  I guess it's good to have both options and use them based on the artistic results you want to achieve.  For a general purpose profile, I'd probably go with the "safer" AdobeRGB gamut compression.

[sebbe]

It depends on your hardware, workflow and output. If you only work on a sRGB-capable monitor and process for an sRGB-jpg at the end, it may be good to start with a profile that fits. On the other hand, when you use an expensive wide gamut display and do prints, why compressing?

[jrp]

To be clearer:

Suppose that I am aiming for sRGB for the final JPEG.

I have a choice of compressing the gamut early (in the DCP)  or late (as part of the final generation of the JPEG).

is one route clearly better than the other?

[sebbe]

...
Suppose that I am aiming for sRGB for the final JPEG.
...

If you never ever print and only go for sRGB, there is no reason to wait until the end.

(We're a little off topic btw)

[torger]

To be clearer:

Suppose that I am aiming for sRGB for the final JPEG.

I have a choice of compressing the gamut early (in the DCP)  or late (as part of the final generation of the JPEG).

is one route clearly better than the other?

If the raw converter has a good gamut compression algorithm it would be best if the DCP didn't compress at all. However most raw converters expects that the DCP has some compression in them, as their own compression isn't too good at handling large compression ranges. Targeting the profile for AdobeRGB and outputting sRGB is usually fine, I'd recommend that for LR.

Note that the gamut compression in DCamProf is not "mathematically exact", in it's standard configuration it allows quite some clipping as it looks better in practice, a sunset is a good test scenario. If compressed mathematically exact it doesn't become as bright as it should but flat and dull.

[jrp]

Thanks, Anders.  That's helpful.

[torger]

I finally got around to make my own Windows and Mac OS X builds, now available for download:

http://www.ludd.ltu.se/~torger/dcamprof.html#download

It's a new version too, version 1.0.1, but the code changes are really minimal so if you have 1.0.0 already running there's no hurry to upgrade.

I'd like to thank Howard and AlterEgo for making builds up to now. I plan to do my own builds from now on for Windows 64 bit and Mac OS X, so there's no need to make third-party builds for those platforms, but it's allowed of course as the software is free.

[howardm]

First off, you're welcome   I'm glad to help the community where possible.

Second, I've been building mine fully static & without OMP.  I see your new binary................

HJM-MBA:~/Desktop/dcamprof-1.0.1-macosx] howardm% otool -L dcamprof
dcamprof:
   /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)
   /usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.5)
   /usr/local/opt/llvm/lib/libomp.dylib (compatibility version 5.0.0, current version 5.0.0)

and the dreaded dylib linker error:

HJM-MBA:~/Downloads] howardm% ../Desktop/dcamprof-1.0.1-macosx/dcamprof
dyld: Library not loaded: /usr/local/opt/llvm/lib/libomp.dylib
  Referenced from: /Users/howardm/Downloads/../Desktop/dcamprof-1.0.1-macosx/dcamprof
  Reason: image not found
Trace/BPT trap


I'm not sure its reasonable for 99.9% users to have to install entire 'port' or 'fink' systems just to have libomp.
Is there enough of a runtime performance 'win' to really care about using OMP?

I suppose the libomp.dylib could be distributed w/ dcamprof.
I haven't looked into the whether libomp can be built as a old-time .a library for inclusion during build.

[torger]

Oh, my mistake in building, thanks for reporting. I'll look into it. I'll pull the Mac OS X build for the moment until I've come up with a suitable solution.

[scyth]

Is there enough of a runtime performance 'win' to really care about using OMP?

a "hoi polloi person" 'd assume that if there is matrix optimization -s used then it is, no ?

PS: does Win64 version include OMP statically linked inside binary executable ?

[torger]

a "hoi polloi person" 'd assume that if there is matrix optimization -s used then it is, no ?

PS: does Win64 version include OMP statically linked inside binary executable ?

Yes, unless I've made some mistake there too, the OpenMP should be statically linked in that case. I'm generally working in Linux so I'm a bit of a beginner messing around on Windows and Mac linking/building.

OpenMP is not strictly necessary no, but some aspects of the software run significantly faster. The standard Clang has had OpenMP support for a while, but unfortunately Apple have rolled their own Clang version which doesn't have OpenMP, so it's a bit of a mess to get working there.

[howardm]

Honestly, I have yet to actually use the tool to generate a profile but if the common usages of it run N seconds longer w/o using OMP, at what value of N does the pain become significant (esp considering the tool itself isn't run that often)??

To the users of my earlier builds, does the runtime seem excessive?

[torger]

When you start generating spectra for SSF simulations it can be nice with a 8-fold speedup, as runtime can be minutes. Otherwise the speedup is in most cases fairly cosmetic as it's fast enough anyway. I as a developer sure wants my parallel coding to be used though when I've done it, so when I make an own build I want OpenMP to be in. So far it does seem like one have to distribute libomp.dylib with the package though if to stay with standard procedures, that's how others do it.

Apple has their own "GCD" for parallel coding, I've coded a bit of that too, but I think it's a pain to use different proprietary technologies to make parallel coding, OpenMP is the best standard way out there, which unfortunately Apple has been slow to adapt despite that the standard Clang package has it.

[howardm]

I would leave it w/ OMP.

Assuming there isn't a sane way of creating libomp.a...............

Perhaps:
1. distribute libomp.dylib and have user install it (or provide script) in /usr/local/lib
2. compile w/ -R or -runpath to force the linker to look there w/o env. vars.

[scyth]

Yes, unless I've made some mistake there too, the OpenMP should be statically linked in that case.

how do you manage that ? I am not a software developer at all and when I tried to search "THE INTERNET" I always saw using libgomp***.dll (means non static) - if you don't mind to comment on this (what was the toolchain and the makefile used to build the static version on windows) ? I am sorry if it is a stupid question

[torger]

I've put back out a mac build, hopefully it works as it should now. There is that extra file libomp.dylib in the package, I figured if you can copy "dcamprof" file to "/usr/local/bin", one should manage to copy "libomp.dylib" to "/usr/local/lib". If one can't get past that, DCamProf is probably too complicated to use anyway

It's technically possible to make a libomp.a and link statically, but then you need to make it yourself from source, which is a mess. The reason there typically is no static version of the OpenMP lib in most distributions is because if more than one library in your application depend on it, there will be performance issues and possibly errors, as there can only be one OpenMP instance in a binary. It would be no problem in this case, but I'm not in charge of the Clang toolchain... hopefully Apple will eventually include OpenMP in their own Clang and then libomp.dylib will most likely become a standard library, just as in Linux.

That it can be static in Windows is because I'm using the MSYS2 distribution and they have made an exception to this rule and provided a static option also for OpenMP. I'm using MSYS2/MinGW64.

[torger]

how do you manage that ? I am not a software developer at all and when I tried to search "THE INTERNET" I always saw using libgomp***.dll (means non static) - if you don't mind to comment on this (what was the toolchain and the makefile used to build the static version on windows) ? I am sorry if it is a stupid question

Certainly not stupid question, I've also been through all THE INTERNET and it's a mess with all different compiler versions and distributions out there. I was actually surprised that it worked statically with MSYS2/MinGW64, it was not something I actively searched for, it just worked. My plan was to distribute the DLL together with the binary just as I do with OS X now, but it turned out to not be required as there's a libgomp.a with the MSYS2 distribution.

Makefile is the same as the distributed Linux makefile, but with the added "-static" option to LDFLAGS and added "-ljpeg -llzma -lz" on the libraries (DCAMPROF_LIBS), then built inside a mingw64_shell that comes with MSYS2 (if you install it using the package management pacman). I don't know exactly which packages I have installed as the machine used for compiling is used for other projects too, and MSYS2 was not installed for this.

[scyth]

Certainly not stupid question, I've also been through all THE INTERNET and it's a mess with all different compiler versions and distributions out there. I was actually surprised that it worked statically with MSYS2/MinGW64, it was not something I actively searched for, it just worked. My plan was to distribute the DLL together with the binary just as I do with OS X now, but it turned out to not be required as there's a libgomp.a with the MSYS2 distribution.

Makefile is the same as the distributed Linux makefile, but with the added "-static" option to LDFLAGS and added "-ljpeg -llzma -lz" on the libraries (DCAMPROF_LIBS), then built inside a mingw64_shell that comes with MSYS2 (if you install it using the package management pacman). I don't know exactly which packages I have installed as the machine used for compiling is used for other projects too, and MSYS2 was not installed for this.

I shall experiment w/ that tonight, thank you !