Pages: 1 ... 65 66 [67] 68 69 ... 78   Go Down

Author Topic: DCamProf - a new camera profiling tool  (Read 767371 times)

Alexey.Danilchenko

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 257
    • Spectron
Re: DCamProf - a new camera profiling tool
« Reply #1320 on: August 18, 2016, 11:38:33 am »

Yes, unless I've made some mistake there too, the OpenMP should be statically linked in that case. I'm generally working in Linux so I'm a bit of a beginner messing around on Windows and Mac linking/building.
OpenMP itself does not recommend static linking since a few things can go wrong. Depending how you compiled it for the mac, you may be able to statically link it as well  - I did this before for gcc 4.7 build with support for OMP for my own QT based projects but abandoned the approach in favour of bundled shared OMP library.
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #1321 on: August 19, 2016, 06:58:41 am »

OpenMP itself does not recommend static linking since a few things can go wrong. Depending how you compiled it for the mac, you may be able to statically link it as well  - I did this before for gcc 4.7 build with support for OMP for my own QT based projects but abandoned the approach in favour of bundled shared OMP library.

Yes I know, and that recommendation has caused most distributions to not include a static OpenMP lib. There's a million things in programming that can go wrong though, this is just one thing. If you know that none of your dependencies are using OpenMP you can link statically, and that's not too hard to find out, so I think it's a bit unfortunate that they are so stiff about it. I agree that eventually it should be dynamic, the real problem is that Microsoft and Apple are slow to adopt portable standards (as they like proprietary solutions better to lock in developers in their own platforms) so they don't provide any OpenMP lib in their platforms, but as soon as they do (if ever) dynamic is the right thing. Meanwhile being stiff about not making static libs just causes additional irritation for us developers.

Oh well, to be fair Microsoft actually have OpenMP support in their compilers. It's very difficult to make portable software that builds with Microsoft compilers in general though (especially if you have advanced stuff like inline asm and such), (both Linux and Mac OS X are Unix-based and Clang/GCC has very similar behavior, while Windows and their compilers are all different) that's why MinGW is used which brings Windows much closer to the other two.

Anyway, using GCC has become much more cumbersome in later OS X versions (don't remember the exact reason, but there's something about linking I think), so I'm using Clang. I'm awaiting that Apple should include OpenMP in their own Clang distro so I can just run with the default Apple toolchain, but so far one have to use a third-party standard Clang to get OpenMP. I've installed via "brew", so it's easy to get going there's no longer need to compile your own compiler (which I had to do a few years ago on the Mac).
« Last Edit: August 19, 2016, 07:05:10 am by torger »
Logged

Alexey.Danilchenko

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 257
    • Spectron
Re: DCamProf - a new camera profiling tool
« Reply #1322 on: August 19, 2016, 11:33:16 am »

Anyway, using GCC has become much more cumbersome in later OS X versions (don't remember the exact reason, but there's something about linking I think), so I'm using Clang. I'm awaiting that Apple should include OpenMP in their own Clang distro so I can just run with the default Apple toolchain, but so far one have to use a third-party standard Clang to get OpenMP. I've installed via "brew", so it's easy to get going there's no longer need to compile your own compiler (which I had to do a few years ago on the Mac).

Same here. I have not used brew - actually gotten the Clang officially build one for OS X and made it into a toolchain so it plugs nicely to XCode. With that I am able to use it just like standard bundled Apple version of clang with all the bells and whistles...
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #1323 on: September 20, 2016, 04:50:54 am »

I'm working on an update, which this time actually adds a new feature :-). Over the past year I've got some user feedback of various kinds which all I can translate to that gamut compression is not as strong as people are used to from the bundled commercial profiles. The current effect of DCamProf's gamut compression is indeed quite mild, which cause some scenes to clip quite badly compared to what they do with a typical commercial profile. I've always thought that it's bad design to have static gamut compression in the profile, it should instead be dynamic and handled by the raw converter, but well, that's not the way it is so I just have to deal with it.

So I have studied how to make a really strong compression that you see for example in Capture One's profiles. Quite early on I found a good way to strongly compress colors, but making the effect gradual towards the gamut edges without hurting gradients or bleeding too much into low saturation colors turned out to be very difficult.

I've come about 80% to the goal though. I think. Sometimes when you start testing more broadly you discover that the approach doesn't hold up and you need to get back to the drawing board. We'll see. If all goes well I should be able to make a new release within a couple of weeks.
Logged

scyth

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 584
Re: DCamProf - a new camera profiling tool
« Reply #1324 on: September 20, 2016, 10:34:22 am »

I'm working on an update, which this time actually adds a new feature :-). Over the past year I've got some user feedback of various kinds which all I can translate to that gamut compression is not as strong as people are used to from the bundled commercial profiles. The current effect of DCamProf's gamut compression is indeed quite mild, which cause some scenes to clip quite badly compared to what they do with a typical commercial profile. I've always thought that it's bad design to have static gamut compression in the profile, it should instead be dynamic and handled by the raw converter, but well, that's not the way it is so I just have to deal with it.

So I have studied how to make a really strong compression that you see for example in Capture One's profiles. Quite early on I found a good way to strongly compress colors, but making the effect gradual towards the gamut edges without hurting gradients or bleeding too much into low saturation colors turned out to be very difficult.

I've come about 80% to the goal though. I think. Sometimes when you start testing more broadly you discover that the approach doesn't hold up and you need to get back to the drawing board. We'll see. If all goes well I should be able to make a new release within a couple of weeks.

may be a stupid question - is it a good idea (or possible usability-wise) to have a parameter controlling which "colors" (shall be some notation allowing to indicate which colors do you mean) to compress "more" or "less" ?
Logged

Doug Gray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2197
Re: DCamProf - a new camera profiling tool
« Reply #1325 on: September 20, 2016, 12:10:02 pm »

I'm working on an update, which this time actually adds a new feature :-). Over the past year I've got some user feedback of various kinds which all I can translate to that gamut compression is not as strong as people are used to from the bundled commercial profiles. The current effect of DCamProf's gamut compression is indeed quite mild, which cause some scenes to clip quite badly compared to what they do with a typical commercial profile. I've always thought that it's bad design to have static gamut compression in the profile, it should instead be dynamic and handled by the raw converter, but well, that's not the way it is so I just have to deal with it.

So I have studied how to make a really strong compression that you see for example in Capture One's profiles. Quite early on I found a good way to strongly compress colors, but making the effect gradual towards the gamut edges without hurting gradients or bleeding too much into low saturation colors turned out to be very difficult.

I've come about 80% to the goal though. I think. Sometimes when you start testing more broadly you discover that the approach doesn't hold up and you need to get back to the drawing board. We'll see. If all goes well I should be able to make a new release within a couple of weeks.

Until it gets to the RAW converter how would it even know what the target colorspace gamut is? What kind of compression does C1 do and what assumptions re gamut does it make?
Logged

scyth

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 584
Re: DCamProf - a new camera profiling tool
« Reply #1326 on: September 20, 2016, 02:04:50 pm »

What kind of compression does C1 do

icc/icm camera profiles used by C1 naturally guide the color transfrom (somewhere at the end of the internal pipeline) from camera's digital numbers to coordinates in a proper PCS (cie***/D50, *** = Lab or XYZ, OEM profiles are Lab, but C1 will digest profiles with *** = XYZ too) - so he certainly can see what LUTs inside icc/icm container do.... and then it is from that PCS to the selected  colorspace for the output /unless we are doing output for profiling/... no ?
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #1327 on: September 20, 2016, 02:35:13 pm »

Until it gets to the RAW converter how would it even know what the target colorspace gamut is? What kind of compression does C1 do and what assumptions re gamut does it make?

Exactly, that's why I don't think it's particularly good design to make it in the profile, but that's what's "everybody" does, and raw converters have as a result limited control of large gamuts and thus makes it harder to work with profiles that doesn't compress a lot, when you shoot high saturation subjects.

The assumption seems to be that target color space is something similar to AdobeRGB, but there's not really a fixed target gamut, it's more like "with proper handling of clipping it should look good in AdobeRGB sized gamuts". Compressing camera gamuts is quite different from screen-to-printer gamut mappings, as with the camera you don't really know what you're going to get. Few "real" colors trigger a raw response of 0% R, 100% G and 0%B for example, so the outer limit of what the camera will deliver is sort of unknown. It's popular to say "camera's don't have a gamut", and this is one of the reasons.

Another aspect is that cameras can deliver very saturated colors, far outside AdobeRGB. The amount of compression required is typically much larger than regular gamut-mapping tasks, so you need to apply different much more subjective methods rather than just using colorimetric formulas.

I can't describe exactly how Capture One does its gamut compression, as I don't know. I haven't really studied it in detail, I try to come up with the best method I can do based on own research. I do look at Capture One and Hasselblad etc for sanity checking and compare performance. One thing I can say though is that to achieve higher compression you don't only work along the saturation axis, but also lightness, darken saturated colors to fit in gamut, and lighten saturated shadows (as colors can clip to zero too, not just max value). Evenso you must still consider scenarios when the camera does clip, like sunsets. "Mathematically accurate" gamut compressions make sunsets and other scenarios involving clipping look dull.

Making pleasing gamut compression for cameras is a little bit like making pleasing skin tones. It's not so much science, but more trial and error and judging the result by eye. It's not about putting the camera into a fixed gamut range, but to dampen the outer range so it behaves a bit "nicer", so when you shoot that deep purple flower you don't immediately need to pull lots of sliders in the raw converter to reduce clipping into reasonable levels, but instead it provides a pleasing output without adjustments.

As I've mentioned many times, camera profiles are still today intended to work as a "film roll", it should provide a well-behaved look within a reasonable gamut without requiring further adjustments of the raw converter, automatic or manual. It's not the way I'd like it to be, but it's legacy from the analog days and it's not going away soon.
« Last Edit: September 20, 2016, 02:41:19 pm by torger »
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #1328 on: September 20, 2016, 02:46:22 pm »

may be a stupid question - is it a good idea (or possible usability-wise) to have a parameter controlling which "colors" (shall be some notation allowing to indicate which colors do you mean) to compress "more" or "less" ?

Yes it's probably going to be some hue-control, although few will change the default setting. I've tried to used equal compression over the whole hue range, but it seems now that's not going to yield the best results. For example it seems that yellows cannot be compressed as much as reds for the most pleasing result. I'm investigating these aspects just now.
Logged

Hening Bettermann

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 945
    • landshape.net
Re: DCamProf - a new camera profiling tool
« Reply #1329 on: September 20, 2016, 03:25:38 pm »

Hi Anders, I hope the old way stays available as an option? The tech stuff is over my head, but I have more confidence in you than in what "everybody" does just for historic reasons, that is without good reason. - I highly appreciate the way you document your programs.
Best regards - Hening.
« Last Edit: September 20, 2016, 05:46:22 pm by Hening Bettermann »
Logged

jrp

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 322
Re: DCamProf - a new camera profiling tool
« Reply #1330 on: September 20, 2016, 03:52:35 pm »

I don't profess to understand the pipeline either, but I do find that on the Sony A7r II and Leica SL, I need to add saturation to get a pleasing picture (which I am happy to do, rather than reducing saturation after if it comes overdone).
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #1331 on: September 20, 2016, 03:53:17 pm »

Hi Anders, I hope the old way stays available as an option? The tech stuff is over my head, but I have more confidence in you than in to what "everybody" does just for historic reasons, that is without good reason. - I highly appreciate the way you document your programs.
Best regards - Hening.

Yes the old way will be there, you will still be able to make profiles entirely without gamut compression, with the old milder gamut compression, and then a new option to make this new stronger compression.
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #1332 on: September 20, 2016, 04:03:03 pm »

I don't profess to understand the pipeline either, but I do find that on the Sony A7r II and Leica SL, I need to add saturation to get a pleasing picture (which I am happy to do, rather than reducing saturation after if it comes overdone).

Indeed this is often the case. Gamut compression I'm talking about here is in the "extreme color" range, colors you find in say deep red flowers, artificial colored lights, artificial colored toys etc. In most "normal" scenes you don't come across colors that needs compression, so it's a quite narrow case. In my own photography I have really never had problems with over-saturated colors. I would post examples to make it clearer, but now my test images are mostly "problem images" that users have sent me which I can't post. Maybe when I'm a bit more finished with the stuff I can post some before/after examples.
Logged

Alexey.Danilchenko

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 257
    • Spectron
Re: DCamProf - a new camera profiling tool
« Reply #1333 on: September 21, 2016, 04:18:37 am »

Exactly, that's why I don't think it's particularly good design to make it in the profile, but that's what's "everybody" does, and raw converters have as a result limited control of large gamuts and thus makes it harder to work with profiles that doesn't compress a lot, when you shoot high saturation subjects.
If I understand you correctly, this compression is an attribute only for LUT profiles. In this case I think it probably meant to have less errors as LUT attempt to "fix" some of the colour problems and have smoother interpolation. My understanding is that matrix profiles do not compress so I would not really attribute this to all raw converters as default behavior.
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #1334 on: September 21, 2016, 07:42:51 am »

If I understand you correctly, this compression is an attribute only for LUT profiles. In this case I think it probably meant to have less errors as LUT attempt to "fix" some of the colour problems and have smoother interpolation. My understanding is that matrix profiles do not compress so I would not really attribute this to all raw converters as default behavior.

Yes it's only applicable to LUT profiles. By nature matrix profiles are 100% linear and thus put responsibility on the raw converter concerning tone reproduction if you apply a curve, and gamut compression if you need that. The big name raw converters like LR and C1 are not designed for that, but expect the profile to apply a tone curve, make subtle look adjustments, and then some gamut compression, and for all that a LUT is required. They will of accept matrix profiles, and yes then you are correct that no such adjustments will then be applied. Personally I think the rendering then falls short, so I don't think matrix-only profiles are a very good to use in either LR or C1, but it's a matter of taste.
Logged

Alexey.Danilchenko

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 257
    • Spectron
Re: DCamProf - a new camera profiling tool
« Reply #1335 on: September 21, 2016, 03:29:34 pm »

Personally I think the rendering then falls short, so I don't think matrix-only profiles are a very good to use in either LR or C1, but it's a matter of taste.

Depends on a camera. Works perfectly with Kodak SLR/n and Proback. I personally find LUT profiles less predictable and of questionnable quality so always try to build matrix ones.
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #1336 on: September 27, 2016, 04:29:30 am »

Here's an example. I've borrowed one of the test images I've got from a user, cropped out a detail so I hope it's okay to show it here.

There are three images, first without gamut compression (no-gamut), then with the weaker old compression (adobergb-old) and then with the upcoming newer stronger compression (adobergb-strong). The compression is configurable so one can tune it, but I expect most to use my presets either "adobergb" if you don't need to strong compression, or the new "adobergb-strong" if you want stronger compression, more similar to what you find in bundled profiles.

I'm doing some final testing and hope to get a release out within the next few days.

The new strong compression is just adding a pre-processing step to the old compression. In the pre-processing the (Prophoto) RGB max and min values are compressed (=HSV Value and Saturation), so you see a darkening effect of highlights and lightening of shadows of the saturated colors. The strength of compression is controlled by how far the inner gamut (AdobeRGB in this case) is from the outer gamut (ProPhoto) for each hue, plus some configurable parameters. The hard part was finding suitable amount of compression, making smooth compression curves and blending it in smoothly to not hurt gradients.

For this particular image I've compared with Capture One's compression, and the "adobergb-strong" compression is in the same ballpark concerning strength.

Note that this is not pure gamut mapping, there's some loseness and clipping required as one doesn't actually know what the outer gamut limit will be, and one need to leave some clipping to avoid dullness, so I've tested through a bunch of images to find a "suitable amount" of compression.

The attached images are sRGB, and indeed even if the output is going to be sRGB compressing towards AdobeRGB is usually the best idea, and let even more saturated colors clip. If you need even stronger compression there are sRGB presets though.
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #1337 on: September 27, 2016, 07:01:42 am »

I've now released v1.0.2, with the new stronger gamut compression, which is optional. The old is still in and works the same.

What's new:
http://www.ludd.ltu.se/~torger/dcamprof.html#news

Downloads (win, mac, source):
http://www.ludd.ltu.se/~torger/dcamprof.html#download
Logged

howardm

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1984
Re: DCamProf - a new camera profiling tool
« Reply #1338 on: September 27, 2016, 08:41:48 am »

A v1.0.2 fully static binary for Macintosh is available below (from box.com, the MS OneDrive has not)

markanini

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 58
Re: DCamProf - a new camera profiling tool
« Reply #1339 on: September 27, 2016, 10:34:45 am »

Its working great on my personal red-blowout shots. :D
Logged
Pages: 1 ... 65 66 [67] 68 69 ... 78   Go Up