Advice on how to manage color spaces from raw to print

[digitaldog]

Yes, well it would seem that using a custom working space might not be so hard (for example modifying AdobeRGB using IccXML).

You can build simple matrix profiles in Photoshop itself (Custom RGB in Color Settings).

Photoshop CC 2014 has a new Export CLUT where one can now create 3DL,CUBE,CSP, or ICC profiles.

[digitaldog]

That way any editing would always be within the monitor gamut. At least then we would have WYSIWYG ... on the monitor, at any rate (printing would still have the same issues .

A big issue. This is a big reason Photoshop divorced your specific display and how you edit the numbers and made working (editing) space agnostic (but display-like) color spaces back in Photoshop 5.
You're allow ignoring colors that could be out of that display gamut but defined from the raw data by virtue of the working space you select at the time of rendering from raw. It's why Adobe raw converters have a big honking and to some degree dangerous processing space.

[digitaldog]

Here is the image with the Lightroom OOG warning:

On this end (wide gamut display but color managed browser), I can't see how that overlay has any real relationship to either image. It shows a huge portion of solid red which seems rather alarming but the two images are rather similar. Further, the biggest visual difference I see immediately is the 'sharpness/contrast' reported.
The difference I see visually remind me of the recent discussions about sending PPI to the Epson and again, I just didn't see anything with the naked eye that made a non subjective difference. If I saw both images below individually I'd have no problem with either. I kind of like the sharper image better, totally subjective.

Looking closer I take that back, I'm seeing 'noise' and edges I don't like compared to the upper image. Subjective and I'm sure hugely influenced by display technology, calibration etc. Fun to go look on the iPad or iPhone...

[Tim Lookingbill]

Here is the image with the Lightroom OOG warning:...


At first sight there doesn't seem to be all that much difference.  However, whether you prefer (or dislike one version over the other, I think it's clear on closer examination that the Lightroom export is generally flatter. For example, the right petal in the Photoshop version has clearer hues and tones in the reds/pinks, and the top petal is also better in the reds and also in the yellows; the yellows of the leaves at the bottom right are more clipped in the Lightroom export than in the Photoshop version; the top left leaf is better in the Photoshop version; contrast is generally better in the Photoshop version.  Mostly, I don't see anywhere where the Lightroom export is better. BTW, the Photoshop version looks sharper, but I didn't apply any sharpening in Photoshop; however I think the down-sampling may have used Bicubic Nearest Neighbour (hard edges) ... not paying attention!..., so that might account for it.

So, although this is a sample of one, and so is hardly conclusive, my own feeling is that conversion to the destination color space and finishing-off processing in Photoshop is likely to show improvements in the image over a straight export/conversion from Lightroom.  With further work more improvements are possible in Photoshop as it has better selective editing facilities - and, more importantly, this editing can be done in the destination color space.

But, sRGB is a clean working space, unlike a paper profile ... so whether the same possible benefit applies to conversion and editing to a paper profile is not so clear (although in the tests I've tried out so far it does seem to be so).

I would be interested in your views!

Robert

 

Between the audio engineering blog discussion I participate on pitch changes related to loudness wars remastering and this OOG discussion, I see so many similarities on what is getting missed.

You want to make it loud at the digital source? (yellow flower? Rhino remaster of Chicago V album?) Learn to know when you've screwed it up and then learn what tools to use to bring it back at the Raw source. You have to hear, think and feel the sound or image as it should look or sound on output be it a print or as CD audio. I was taught that early on as a child musician. I finally saw the similarities to this in digital imaging later on.

I've captured flowers shot brilliantly lit by direct sunlight and can get no clipped data (visually) on both my sRGB screen display and glossy paper print and I use "Printer Manages Color" on a $50 Epson "All In One". Gorgeous prints of flowers brightly lit full of detail. Your yellow flower is shot in overcast light. Why are you blaming OOG on why you can't enhance it? Almost every image I work on in ProPhotoRGB space and attempt to edit in order to make it loud and brilliant ALWAYS shows clipped data converting to sRGB. Loudness Wars! OOG! It's all the same.

You're problem is not OOG. It's that you haven't learned to use the editing tools and how they work and behave in relation to your perception to get you the look you want to convey as you remember the scene which is rarely ACCURATE. It's not your fault. It's the limits of how the software is designed to make it easy to use for those that don't have the time to figure this out.

It took me almost 5 years to figure out PV2003 advantages and disadvantages of the slightly linear behavior of the Exposure slider vs the behavior of the Brightness slider and to finally realize that these two don't always map the not so linearly recorded sensor data of my camera the way my mind sees it.

I started using the Parametric curve to build a curve to remap linearity back into both the sensor data and how the Exposure slider and Brightness slider act on the preview to push brightness while maintaining local contrast in highlights just as I purchased LR4 for its PV2012 which is no longer linear at all and is now quite difficult for me to figure out what's affecting what. It's like starting over from scratch. I see you're having the same difficulties in your LR vs PS edit of the yellow flower. It's a dull, dim and flat looking image and you want to give it some pop. OOG concerns are not where to look.

[samueljohnchia]

Hi,

That approach is in principle not much different to using the Beta RGB colorspace that was designed by Bruce Lindbloom as a workingspace. Comfortably larger than AdobeRGB in the right places, and not as gigantic as ProPhoto RGB.

Cheers,
Bart

Hi Bart, Beta RGB and the Dcam spaces do have significant differences in their design approach. Both Bruce and Joseph took the same approach of looking at many examples of where real world colors lie and designed their color spaces to encompass as much of them as possible. That is as far as they are similar. Bruce compared the colors from film based input, but also from color charts and printing gamuts. Which makes it both input and output centric. It is still one big honking color space (not as big as ProPhoto RGB as you pointed out), one size fits all. Unnecessary quantization will still occur in small gamut input.

Joseph looked at many examples of digital images made with digital cameras, and then edited them in a variety of ways and looking hard at how colors are moved around by such massaging. He remained entirely input centric in the approach to the design of his master Dcam spaces, and specific to digital camera input. There are 5 of them in total, from Dcam 1 (tiny gamut) to Dcam 5 (absolutely gigantic), so the discerning photographer may select the smallest possible space that encompasses all the colors of any given image. For example, a foggy grey picture would require Dcam 1, while Robert's picture would be better served by Dcam 3 at least. Joseph also designed a unique 1024 tone response curve to better match the perceptual linearity of the way the human eye sees, which coincidentally is a better match to a profiled inkjet printer's TRC, minimizing quantization errors during the conversion. He rejected gamma curves for being too flat near black.

Since we are talking about saturation here, I should say that Joseph has also designed what he calls "chroma variants", which I find to be far superior to any method of dealing with the colorfulness of my pictures. They exhibit almost no tonal shifts and hue shifts, unlike the saturation tools found in PS and LR, especially for dealing with yellow greens, where increasing saturation there always causes yellow to shift toward green and drives colors toward clipping far too easily.

[samueljohnchia]

I'm assuming you don't have some technique to use the selective color adjustment layer to create a saturation mask?
Robert

Of course I do.

There are probably 5 or 6 ways to create what others call a "saturation mask" in Photoshop. I prefer the term "chroma masking". All of them do not create accurate chroma masks, except for the selective color method.

The selective color method is as follows:

1. New Selective Color Adjustment Layer
2. Absolute mode
3. In the color selection dropbox, all colors set black slider to 0%
4. All neutrals set black slider to 100%
5. Generate the mask from any one of the channels in the channels panel.

[Robert Ardill]

A big issue. This is a big reason Photoshop divorced your specific display and how you edit the numbers and made working (editing) space agnostic (but display-like) color spaces back in Photoshop 5.
You're allow ignoring colors that could be out of that display gamut but defined from the raw data by virtue of the working space you select at the time of rendering from raw. It's why Adobe raw converters have a big honking and to some degree dangerous processing space.

Yes, actually I'm not too pushed about this as Adobe RGB is really close to my monitor gamut, so I would need to be working with very saturated colors before I would risk not seeing them (which is a point you made in an earlier post).  I would really be more interested in limiting my working space so that it's within my printer/paper space, so that, to the extent possible, I print what I see and I see what I print.

But again, this is really a learning exercise for me ... I'm just trying to understand what is and what isn't possible so that I don't make stupid mistakes.

Robert

[Robert Ardill]

You want to make it loud at the digital source? (yellow flower? Rhino remaster of Chicago V album?) Learn to know when you've screwed it up and then learn what tools to use to bring it back at the Raw source. You have to hear, think and feel the sound or image as it should look or sound on output be it a print or as CD audio.

Why are you blaming OOG on why you can't enhance it? Almost every image I work on in ProPhotoRGB space and attempt to edit in order to make it loud and brilliant ALWAYS shows clipped data converting to sRGB. Loudness Wars! OOG! It's all the same.

You're problem is not OOG. It's that you haven't learned to use the editing tools and how they work and behave in relation to your perception to get you the look you want to convey as you remember the scene which is rarely ACCURATE. It's not your fault. It's the limits of how the software is designed to make it easy to use for those that don't have the time to figure this out.

I see you're having the same difficulties in your LR vs PS edit of the yellow flower. It's a dull, dim and flat looking image and you want to give it some pop. OOG concerns are not where to look.

Hi Tim,

As I said when I posted the image of the flower, I didn't pick it because I think it's a beautiful image or a well-processed image (actually, I think it needs to be binned .  I also think it can be simply improved just by adjusting the white balance, which, as has been pointed out, is much too warm.  The reason I picked it is that it does have some quite out-of-gamut colors and I am interested in OOG because that is one of the areas that messes up the output.  OOG causes Perceptual to shift all the colors and it causes Relative to clip them to the nearest in-gamut color (as we all know).  How exactly this is done is out of my personal control because it depends on the CMM and the output profile.  So, I do think that it is a good thing to keep the colors in gamut if possible, or at least to be aware of what can happen if you leave them out of gamut.  Of course, soft-proofing helps a lot here and if it shows that all is well, then fine, there's no need to sweat the small stuff.

At the end of the day, what I may get from all of this is simply an awareness of what I may be doing when I, for example, use the Camera Raw Filter in Photoshop: The ACR image (I assume) will now be in ProPhoto (or equivalent), whereas my image might have been in Adobe RGB.  I might then increase the luminance of the yellow, all seems well in ACR ... but unbeknownst to myself I have now pushed the image out of gamut.  

Your points are well made though: I do agree that having a 'feel' for the image is the most important thing, and I also agree that it takes a long time for this to mature (assuming it ever does .  But I also think that part of getting this 'feel' is understanding what, as you point out, things like the Exposure slider, White slider, Highlights sliders do and how they differ from each other.  In other words I think we need to understand our tools.

Finally, if an artist wants to print highly saturated colors ... well then, that's his prerogative, and whether you or I like his work is essentially irrelevant (to my mind, at any rate).  What he needs to understand is how to get the look he wants - and why his print just doesn't match what he sees on his monitor.

Robert

[Bart_van_der_Wolf]

Of course I do.

There are probably 5 or 6 ways to create what others call a "saturation mask" in Photoshop. I prefer the term "chroma masking". All of them do not create accurate chroma masks, except for the selective color method.

The selective color method is as follows:

1. New Selective Color Adjustment Layer
2. Absolute mode
3. In the color selection dropbox, all colors set black slider to 0%
4. All neutrals set black slider to 100%
5. Generate the mask from any one of the channels in the channels panel.

Hi,

At first glance, I fail to see how this specifically addresses the clipping caused by profile conversions, other than allowing to reduce total saturation for a few colors.

Cheers,
Bart

[Robert Ardill]

Hi,

Just to show that I do sort of understand that there are different ways to improve images, and not all require getting them into Photoshop (, here is a modified version of the flower I posted:



compared to the previous version:


All I did here was to crop, adjust the white balance and darken and desaturate the background, soften the green bits (by reducing the Clarity) and desaturate them a bit.  I also reduced the saturation in the yellows and reds in the flower and increased the overall contrast of the image.  I did all of this in Lightroom.  

I agree (before I get jumped on ... that it still isn't beautiful ... this photo never will be (at least not in my hands, if for no other reason than it isn't worth wasting time over IMO).

But, I think you will, hopefully, agree that the flower appears more saturated/vibrant, even though it is less so.  In fact the Lightroom image is pretty much fully inside sRGB and there is nothing to be gained by taking into Photoshop (well, not unless we want to do more things with it that can't be done in Lightroom, of course).

So, as the old saying goes, there's more way than one to skin a cat

Robert

[samueljohnchia]

At first glance, I fail to see how this specifically addresses the clipping caused by profile conversions, other than allowing to reduce total saturation for a few colors.

Just to clarify - I mentioned this technique of creating a saturation mask only because Robert mentioned that he creates them using the HSL plugin, which is a less accurate method of doing so. He wasn't aware of the details of this technique so I elaborated there. Of course a mask on its own cannot address clipping, it must be used in conjunction with other edits. To target specifically OOG colors, your method of creating an OOG mask is certainly a good way.

But, I think you will, hopefully, agree that the flower appears more saturated/vibrant, even though it is less so.

This attempt is much better! I believe that relative colorfulness is vital to making things appear more saturated than they are, and also proper white balancing. Now the centre yellow bit of the flower is not competing with the previously overly warm surroundings, so it looks cleaner and brighter and more colorful now. The greens also look much more natural.

I am also a believer that getting a feel for your tools and developing a refined taste for tone and color is more important. I would not worry so much about working with colors that may not be displayed. There's no way that a red would shift from red to green for example. What you should be afraid of is clipping colors in your working space of choice. You end up with featureless details, they are gone forever. A judicious photographer would never go completely crazy on the saturation slider anyway. Soft, natural looking renditions actually appear much cleaner and easy on the eyes. Like in your example, they can even cause the illusion of being more strongly coloful too.

Having excellent printer profiles with top notch gamut mapping to deal with OOG colors is preferred for me. I wouldn't want to work within the gamut boundaries of the printer, simply because that gamut changes all the time. Finding a better media setting, changing the ink load, changing the paper, changing your method of building your custom profiles etc all changes the output gamut. Printing technology advances over time too.

[Robert Ardill]

Soft, natural looking renditions actually appear much cleaner and easy on the eyes. Like in your example, they can even cause the illusion of being more strongly coloful too.

Having excellent printer profiles with top notch gamut mapping to deal with OOG colors is preferred for me. I wouldn't want to work within the gamut boundaries of the printer, simply because that gamut changes all the time. Finding a better media setting, changing the ink load, changing the paper, changing your method of building your custom profiles etc all changes the output gamut. Printing technology advances over time too.

Yes, I completely agree Samuel.

Regarding printer profiles ... I have a Z3100 which has a built-in spectrometer and it seems to calibrate well and create quite good profiles.  I've also used ArgyllCMS with an i1Pro2 and it seems to get a better black point and wider gamut than either the Z3100 or the canned profiles from Canson (I've only tried this on the Canson Platine and Photo High Gloss - which I use for acrylic-mounted pictures).  I'm also going to try these two papers using i1Profiler.  I check out the profiles using GamutVision (Imatest).

Do you or does anyone have experience of the pros-and-cons of the different profiling tools for different papers ... and what can be done to improve the profiles (for example less/more patches, FWA compensation or not for papers that use optical brighteners (like the Canson Photo High Gloss) ...)?

Thanks!

Robert

[bjanes]

Keep in mind, OOG is a fact of life and either you or the profile (or both) have to fit a gallon of water into a half gallon container (or less). If there are significant color shifts or important colors that block and lose detial, that would be an issue. If you start with a big container, at least you've stored the data until you funnel it into the smaller and/or different output space. If you look at the shapes of RGB working space and then output spaces, there's a lot of disconnect there too. Round holes that need to be fit into square peg shapes. I'd be a bit less obsessive for awhile, see how the output comes out. You might end up being a tad less obsessive since there isn't a lot you can do. If you find blocked up colors with no detail or a color shift, yes, pay attention and see if you can fix it, either manually (good luck) or with better profiles, a better selection of rendering intents, a different output device with a larger gamut and so forth.

Here's an example of my attempts, perhaps not well done, to 'fix' the OOG colors in LR instead of just letting a good profile do the job:
http://digitaldog.net/files/LR4_softproof2.mov

With many of my images, my experience with letting the CMS manage out of gamut colors is similar to that of the DigitalDog. As an example, i have a still life image of some flowers rendered into ProPhotoRGB. Many of the colors are out of the gamut of my Epson 3880 with Epson Premium Glossy paper.

The gamut warning in Photoshop (or LR) is not particularly helpful, since there is no indication of the extend to which the colors are out of gamut--just a little or by a wide margin.



ColorthinkPro can show pseudocolor gamut map, but the process is laborious. One must downsize to small images because of the slow performance of the program, convert to a color list and then active the DeltaE facility. Results for the image in consideration is shown.



The Gamutvision program by Norman Koiren, the author if Imatest, can create a pseudocolor mapping of the extent of the out of gamut colors directly and the results are here. I would encourage interested readers to look at this program, which is modestly priced in comparison to ColorThinkPRo.



It would be helpful if LR and Photoshop would create such out of gamut displays. However, in this case, I get the best results by printing the image with relative colorimetric and letting the out of gamut colors clip. They do not appear blocked up in the print. The colors are out of gamut to the extent that perceptual rendering makes little difference.

Bill

[digitaldog]

We have to pay very close attention to the display, the weak link, as we edit. We might be altering OOG numbers that exist that we can't see. Here's where an overlay in three straights with opacity control (let me see the detail that's OOG, don't cover it with a Red stain) would be really useful. And I but Eric could do all this. But how many customers would care or aid considering the engineering other costs? I still hear more complain about dark prints, that's a bigger fish to fry IMHO

Going back to OOG, there might be a way to do this using ColorThink and Photoshop....

[bjanes]

We have to pay very close attention to the display, the weak link, as we edit. We might be altering OOG numbers that exist that we can't see. Here's where an overlay in three straights with opacity control (let me see the detail that's OOG, don't cover it with a Red stain) would be really useful. And I but Eric could do all this. But how many customers would care or aid considering the engineering other costs? I still hear more complain about dark prints, that's a bigger fish to fry IMHO

Going back to OOG, there might be a way to do this using ColorThink and Photoshop....

Such an overlay would be nice, but that fact that it is not available may be that most of are satisfied with what we have and have been getting good results without laborious attempts at manual gamut mapping, which often make the image look worse. The display is often the weak point, as  you point out, but there is a gamut shape mismatch between the output space of the display and printer. As an example, here is a Colorthink plot of my printer (solid color), my display (wireframe), and the previously discussed image . At high luminance values, the display actually has a larger gamut than the printer. The printer excels in low luminance saturated greens, but such colors are not present in the current image and the extent to which ones perception of these low luminance colors is sensitive is in question. Intuitively, I would think that high luminance colors would likely attract most notice with observers.

[Robert Ardill]

The Gamutvision program by Norman Koiren, the author if Imatest, can create a pseudocolor mapping of the extent of the out of gamut colors directly and the results are here. I would encourage interested readers to look at this program, which is modestly priced in comparison to ColorThinkPRo.

Bill

Yes, GamutVision is really excellent (can't compare with ColorThink Pro as I don't use it).  In fact, one way of getting a good saturation map is to use the 'Read Image for Analysis' feature with the analysis set to Delta-C*ab with a Grayscale color map. 



The grayscale saturation map can then be copied into a mask and resized to fit.  A little laborious, but not so bad ... and certainly using GamutVision to check out the image does give a load of very interesting information.

For example, this 3D plot of Input to Output (in this case the image from sRGB to print) shows that the OOG pixels are mostly in the red and green, but at quite low luminance (the lines ending in circles show the transform, with the size of the circles giving an indication of the frequency of the color):


Robert

[Robert Ardill]

We have to pay very close attention to the display, the weak link, as we edit. We might be altering OOG numbers that exist that we can't see. Here's where an overlay in three straights with opacity control (let me see the detail that's OOG, don't cover it with a Red stain) would be really useful. And I but Eric could do all this. But how many customers would care or aid considering the engineering other costs? I still hear more complain about dark prints, that's a bigger fish to fry IMHO

Going back to OOG, there might be a way to do this using ColorThink and Photoshop....

Hi Andrew,

You've mentioned, a couple of times, the problem of potentially not being able to see OOG colors because the monitor gamut is smaller than the working space.  However, when I suggested modifying the working space to fit the monitor gamut (effectively using the monitor gamut as the working space) you seemed quite against the idea.  I can see why using non-standard working spaces is generally not a good idea, but I don't understand what the danger is if it's done with awareness.

If we do the final edit in ProPhoto we're at great risk of being OOG both on the monitor and printer.  So surely it's better to do the final edit in Adobe RGB; and by the same token even better to do it in the monitor color space, because then we only need to worry about the print OOG.


As Bill points out, the paper gamut is often bigger in places than the monitor gamut.  Here, for example you can see that the printer gamut is a bit bigger in the higher luminance orange-greens and lower luminance green-blues ... but everywhere else the monitor gamut is much bigger (and this is a really good paper!).



So I would have thought that the final edit would be much better done with the image converted to the printer profile, with the awareness that there are some colors, as in the plot, that are potentially dangerous.  I understand that there are issues with this (in the same way that soft-proofing should be used with caution because the eye doesn't adapt chromatically to a monitor as it does to paper).  But I don't understand what else could be a problem.  If you have the time I would really appreciate an explanation.

Robert

[digitaldog]

As Bill points out, the paper gamut is often bigger in places than the monitor gamut. 

Exactly!

[Bart_van_der_Wolf]

Exactly!

Meaning that it only matters if the image has very saturated colors in the C/M/Y axes, for which the printer has specific inks and the display only has R/G/B primaries ...

Cheers,
Bart

[Robert Ardill]

Hi Robert,

I use something much better than a simple in/out-of gamut mask. I create a Hue/Saturation adjustment layer based on exactly how much the colors are OOG. I created a Photoshop Action to create the selection of OOG colors, which then is applied as a mask to the Hue/Sat adjustment layer, so only OOG colors are affected.

The basis of the action is, make a duplicate of your document (flattened), and convert the new document to to the destination colorspace (relative colorimetric intent). Copy that converted/clipped image and paste it as a layer in the original document (preserve appearance), in Difference blending mode.

Now you can use that to create an intensity mask for the degree of being OOG. Now creatively apply that as a mask to a Hue Saturation adjustment layer where you can set Saturation to -100% if necessary, or target specific colors. Because it is a layer, you can adjust the opacity if you only want to apply a subtle change.

Cheers,
Bart

Hi Bart,

I wanted to go back to this technique because it's really quick and easy using an action - and it makes sense to me.  You are specifically targeting OOG colors (so it's not just a saturation map as with the HSL filter technique).  As I mentioned, I thought I had invented this technique myself ... ah well, Lamark must have felt like this when Darwin published 'The Origin' 

Anyway, what I found is that the technique doesn't just give OOG colors, but also dark-point clipping, so there's a need to do a small levels adjustment on the mask to remove the dark shadow (but now that I say that I see that I'm being ridiculous because these dark areas won't be printed anyway .

What I would like to understand (and from a post I saw on another thread I can see that you know your maths) is what happens exactly when we use this technique.  I'll try to explain what I think happens and perhaps you would be good enough to set me straight if I've got it wrong.

OK, when we convert from the working space to the paper profile using Relative, every OOG pixel will be shifted to the nearest in-gamut color.  What that color is depends on the CMM and the profile, but it could include saturation, hue and lightness shifts.  So lets say that a particular pixel gets shifted from red (Lab 50,80,70) to orange (Lab 50,50,70) (could happen, right?).

Then, when we convert back to the working space, the CMM and reverse profile may or may not shift that color again.  Let’s say that this particular color is inside the gamut of the print but outside the gamut of the working space.  Then there could be a shift to, say, Lab 50,60,70, to bring it back into the working-space.  Correct?

If we then use the difference between the round-trip image and the original, we’ll get a difference … but the difference won’t be quite correct.

I assume, and I could be absolutely and entirely wrong here (!!) that in the round-trip, the white point shift that occurs in the working-space to print profile direction will be reversed correctly in the print profile back to working-space conversion. But if it’s not, then there are a whole load of colors that will show a difference even though they aren’t OOG at all.  Presumably what actually happens is down to how good the CMM and profile are … but do you know what is supposed to happen?

At any rate this doesn’t mean that the mask that’s generated isn’t 90% correct (and if it is it’s good enough for me!).  But I would really like to understand what actually happens (or is supposed to happen) when we convert from one profile to another … and then back again.  I would seriously appreciate an explanation … either from you or from anyone who knows and has the time to give it!

Robert