Advice on how to manage color spaces from raw to print

[digitaldog]

If we then use the difference between the round-trip image and the original, we’ll get a difference … but the difference won’t be quite correct.

Most likely not. All the CMM understands is what it is feed, numbers in Lab. It has no idea what source color space built that set of Lab numbers.

[Bart_van_der_Wolf]

Hi Bart,

I wanted to go back to this technique because it's really quick and easy using an action - and it makes sense to me.  You are specifically targeting OOG colors (so it's not just a saturation map as with the HSL filter technique).

Hi Robert,

Yes, I'm targeting OOG colors because they tend to be pushed to the 255 (in 8-bit) boundary in the smaller gamut colorspace. However, we're not getting only those OOG colors, but also smaller differences e.g. due to converting back and forth between different illuminants and some other color shifts. But the more extreme differences will indeed allow to address the OOG colors.

Anyway, what I found is that the technique doesn't just give OOG colors, but also dark-point clipping, so there's a need to do a small levels adjustment on the mask to remove the dark shadow (but now that I say that I see that I'm being ridiculous because these dark areas won't be printed anyway .

Spot on! We are altering colors, so it is more about taming the magnitude of difference, and the relevance of those colors.

The fact that we also get some colors with smaller differences will allow a sort of Perceptual (I'm using this very loosely) adjustment of adjacent colors that were still more or less in gamut. Afterall, we cannot only wrestle the OOG colors into gamut submission, because that would essentially be similar to a Relative Colorimetric intent, and would lose color differentiation/detail.

What I would like to understand (and from a post I saw on another thread I can see that you know your maths) is what happens exactly when we use this technique.  I'll try to explain what I think happens and perhaps you would be good enough to set me straight if I've got it wrong.

OK, when we convert from the working space to the paper profile using Relative, every OOG pixel will be shifted to the nearest in-gamut color.  What that color is depends on the CMM and the profile, but it could include saturation, hue and lightness shifts.  So lets say that a particular pixel gets shifted from red (Lab 50,80,70) to orange (Lab 50,50,70) (could happen, right?).

Then, when we convert back to the working space, the CMM and reverse profile may or may not shift that color again.  Let’s say that this particular color is inside the gamut of the print but outside the gamut of the working space.  Then there could be a shift to, say, Lab 50,60,70, to bring it back into the working-space.  Correct?

If we then use the difference between the round-trip image and the original, we’ll get a difference … but the difference won’t be quite correct.

You've got it. It won't be absolutely correct, but it will be somewhat weighted towards the magnitude of the absolute difference.

I assume, and I could be absolutely and entirely wrong here (!!) that in the round-trip, the white point shift that occurs in the working-space to print profile direction will be reversed correctly in the print profile back to working-space conversion. But if it’s not, then there are a whole load of colors that will show a difference even though they aren’t OOG at all.  Presumably what actually happens is down to how good the CMM and profile are … but do you know what is supposed to happen?

Correct again, however, the largest differences will still be in the clipped colors.

At any rate this doesn’t mean that the mask that’s generated isn’t 90% correct (and if it is it’s good enough for me!).

That's it. It won't be 100% correct, but it will have a weighting towards the OOG colors.

But I would really like to understand what actually happens (or is supposed to happen) when we convert from one profile to another … and then back again.  I would seriously appreciate an explanation … either from you or from anyone who knows and has the time to give it!

Solutions like Gamutvision and Colorthink will allow to get much closer, however it remains to be seen how that accuracy in converted into a practical solution. Identifying a problem is nice, identifying it very accurately is nicer, but not being able to solve the issue is worthless.

Based on the OOG weighted initial mask, we can adjust the mask, and we can adjust the corrective layer on which that mask will be used. We can make an overall desaturated layer and apply the (adjusted) mask to that, but we can also choose to do something different.

Cheers,
Bart

[Robert Ardill]

Hi Bart,

Many thanks for taking the time to answer my questions!

Here is a comparison between an error mask created using GamutVision (dE94) and Bart's OOG technique (BOOG ... previously called ROT for Robert's OOG Technique .

The top mask is BOOG and the bottom one is GV:



As you can see they are very close.  

The only areas that might need adjustment in this particular image are the ones that are very bright, actually, so if I was adjusting an image with these masks I would apply a levels adjustment to the BOOG mask, like this:



and probably also apply a bit of blurring.

So really good on this sample of one!

I might write a script to make it easier to use (than actions) with different profiles (or perhaps picking the profiles automatically from the working space and Soft-Proof selection).

Robert

[Bart_van_der_Wolf]

Here is a comparison between an error mask created using GamutVision (dE94) and Bart's OOG technique (BOOG ... previously called ROT for Robert's OOG Technique .

The top mask is BOOG and the bottom one is GV:

[...]

As you can see they are very close.

Hi Robert,

Thanks for that comparison with GV. While not perfect, my suggestion is apparently not all that bad, especially for a free solution that can be scripted/'action'ed and also works with older versions of Photoshop.

Cheers,
Bart

[Robert Ardill]

Hi Bart,

Yes, it's pretty good.  But ... the question is what do you do once you have the mask!  I find that Hue/Saturation does a pretty bad job of it. Vibrance seems a lot better. Selective Color is also not so bad.  For this particular image I found that a combination of Vibrance (which fixes the yellows pretty well) and Selective Color (which finishes off the OOG greens) is good.

After all, there's not much point in using this sort of technique if the CMM does a better job all on its own!

This could be why Adobe hasn't bothered with an OOG mask: it's interesting, but is it useful?  The jury's still out for me, although I suspect that it has its place, used with discretion .

Robert

[Slobodan Blagojevic]

... This could be why Adobe hasn't bothered with an OOG mask: it's interesting, but is it useful?...

I think Adobe realized the problem has been solved long time ago. More precisely, when Shakespeare wrote a play about it: "Much Ado About Nothing"

[Robert Ardill]

I think Adobe realized the problem has been solved long time ago. More precisely, when Shakespeare wrote a play about it: "Much Ado About Nothing"

Could be .  I think like Socrates I am becoming wise ... because I edging towards knowing that I do not know .

[Robert Ardill]

I thought I would try to summarise what I've learnt here about OOG issues.

1.   When we work in Lightroom we are working in a huge gamut (Melissa RGB), much bigger than the web or monitor or printer.  So it’s very easy to push the image OOG for the intended destination.

2.   When we import the image into Photoshop from Lightroom, we should import into ProPhoto to ensure that there is no clipping, as ProPhoto has the same gamut as Melissa RGB.  

3.   Before conversion to a destination profile (or to another working space like Adobe RGB) it’s very important to do a soft-proof to see how the conversion alters colors.  The OOG warnings in Lightroom and Photoshop are useful as a pointer where to look, but not for much more as they don’t give an idea of how much or how little individual pixels are OOG.

4.   In general, the CMM does a better job of converting OOG colors to in-gamut colors than attempting manual adjustments.

5.   If manual adjustments are necessary because the CMM has not done a proper job, then:
   a.   The first thing is to look at the image to see if an alternative adjustment could yield a better result that would not have the same OOG problems (for example, simulating highly saturated colors by having low-saturation areas nearby).
   b.   Less is more: so if possible, it should be done selectively, only where the CMM has failed.
   c.   An OOG mask can be produced using GamutVision (possibly ColorThink Pro also?) or using the OOG technique explained by Bart in this thread, or even manually.
   d.   Using the OOG mask, combinations of Hue/Saturation, Vibrance and Selective Color adjustments (and maybe others!) can be used selectively to corral the pixels back into gamut.  All that’s needed is to bring them reasonably back into gamut (again, less is usually more) … the CMM should be able to do the rest of the job.  The idea in general is to fix those areas that look badly flattened (or clipped) in the soft-proof: these are likely to be fairly large areas of fairly uniform color.

6.   Where it is not possible to process the image as desired without ending up with badly OOG regions, an alternative is to convert the image to the destination space (making sure that the image is mostly within gamut before the conversion) and then to finalise the adjustments in the destination space.  This will ensure that there are no OOG colors in the output file and it will also make it possible to see visually how colors are being squashed towards the gamut boundaries.  One thing to bear in mind here is that the monitor gamut may be smaller than the destination space, so it may not always be possible to see what’s happening.  In this case it may be useful to soft-proof to the monitor profile – the Photoshop OOG warning will give an indication of where we can no longer see the changes we’re making.

7.   Using a tool like GamutVision or ColorThink Pro is really useful as it shows where there are mismatches between the source and destination profiles, how good the profiles are, where to watch out for OOG problems … and a lot more.

Please correct me if I’ve left things out or if I’ve said things that are not true!

And thanks for your help and advice!!

Robert

[Bart_van_der_Wolf]

I thought I would try to summarise what I've learnt here about OOG issues.
[...]

Hi Robert,

Seems like a fair summary, it's mostly how I also see things. The only exception is with point 4, where Photoshop uses a Relative Colorimetric intent (which simply clips to gamut boundaries) even when Perceptual is selected. That's not a good job, that's brutal.

Cheers,
Bart

[Robert Ardill]

Hi Robert,

Seems like a fair summary, it's mostly how I also see things. The only exception is with point 4, where Photoshop uses a Relative Colorimetric intent (which simply clips to gamut boundaries) even when Perceptual is selected. That's not a good job, that's brutal.

Cheers,
Bart

Thanks Bart - it's good to know I'm not too far off-course.  

My understanding is that any conversion from, say ProPhoto or Adobe RGB to, say sRGB (actually I think it's to any matrix-based profile) is always Relative (even if Perceptual is specified) ... so there's no point in trying out one or other intent in conversions with these types of profile.  As you say, the clipping in that case is going to be whack, bye-bye!  So, actually, converting an image for the web is one of the places one has to be the most careful (now that I think of it ... thanks for pointing it out!).

However with profiles that use LUT tables (which would include all print profiles, I would think), the conversion is done by whatever CMM is picked (Adobe ACE, for example) using the profile tables.  Then what happens is not necessarily just a clipping ... there could also be a hue and lightness change, as well as a saturation change (at least it seems like that when I've compared an image before and after conversion). In that case the clipping will depend on the conversion intent used ... and also on the CMM and tables ... and if these are well implemented the clipping shouldn't be too brutal.  Do you agree?

Robert

[digitaldog]

1.   When we work in Lightroom we are working in a huge gamut (Melissa RGB), much bigger than the web or monitor or printer.  So it’s very easy to push the image OOG for the intended destination.

To clarify, Melissa RGB is the color space for the histogram and numbers you see outside of soft proofing, it's ProPhoto RGB primaries with an sRGB like tone curve. The internal color space (more important) that is used for the processing has no name. It's also ProPhoto RGB primaries but uses a 1.0 TRC. It isn't what you're seeing in the Histogram or RGB numbers. What the OOG colors use as a source space, which IMHO is moot, it's not accurate, is something maybe Eric can define.

[Bart_van_der_Wolf]

Thanks Bart - it's good to know I'm not too far off-course.  

My understanding is that any conversion from, say ProPhoto or Adobe RGB to, say sRGB (actually I think it's to any matrix-based profile) is always Relative (even if Perceptual is specified) ... so there's no point in trying out one or other intent in conversions with these types of profile.  As you say, the clipping in that case is going to be whack, bye-bye!  So, actually, converting an image for the web is one of the places one has to be the most careful (now that I think of it ... thanks for pointing it out!).

That's exactly why I mentioned it, in particular conversions for Web publishing (typically to the small gamut sRGB colorspace) need a second look. Loss of feature detail may, or may not, be important for certain image content. Small tweaks (saturation, or even hue, or lightness) in LR, based on the Proof preview, may already be enough. The goal is not to prevent clipping all together, that would be detrimental to overall image quality. All we need to do is mitigate the detrimental effects, if any. Going from a wide gamut to a narrow one is supposed to challenge the gamut boundaries if we start with a colorful image.

However with profiles that use LUT tables (which would include all print profiles, I would think), the conversion is done by whatever CMM is picked (Adobe ACE, for example) using the profile tables.  Then what happens is not necessarily just a clipping ... there could also be a hue and lightness change, as well as a saturation change (at least it seems like that when I've compared an image before and after conversion). In that case the clipping will depend on the conversion intent used ... and also on the CMM and tables ... and if these are well implemented the clipping shouldn't be too brutal.  Do you agree?

Yes.

Cheers,
Bart

[bjanes]

I thought I would try to summarise what I've learnt here about OOG issues.

4.   In general, the CMM does a better job of converting OOG colors to in-gamut colors than attempting manual adjustments.

Robert,

Thanks for an excellent analysis. Current CMMs are dumb--that is, with perceptual rendering, they do not look at the actual colors in the image, but merely perform a predetermined amount of compression to the color gamut whether or not such a compression is needed. This is explained well by Mike Chaney in a nearly 10 year old post. Contrary to what some believe, the gamut of the image is not compressed to the gamut of the printer, and clipping of the most out of gamut colors may occur.

What is needed is an intelligent CMM that looks at the colors actually in the image in order to determine the amount of compression needed and which colors need to be compressed. This ICC post mentions the possibility of an intelligent CMM--see the link V4 raison d'etre... As far as I know, such an intelligent CMM is not available for general use, but there is some hope that such a CMM may be available sometime in the future.

Regards,

Bill

[Robert Ardill]

To clarify, Melissa RGB is the color space for the histogram and numbers you see outside of soft proofing, it's ProPhoto RGB primaries with an sRGB like tone curve. The internal color space (more important) that is used for the processing has no name. It's also ProPhoto RGB primaries but uses a 1.0 TRC. It isn't what you're seeing in the Histogram or RGB numbers. What the OOG colors use as a source space, which IMHO is moot, it's not accurate, is something maybe Eric can define.

Enough to give anyone a headache!!  I suppose there has to be some rationale to all of this, but it seems that it's piling complexity on top of complexity ... and one wonders how the poor CMMs (not to mention us) can hope to see the wood from the trees!

Robert

[bjanes]

Enough to give anyone a headache!!  I suppose there has to be some rationale to all of this, but it seems that it's piling complexity on top of complexity ... and one wonders how the poor CMMs (not to mention us) can hope to see the wood from the trees!

Robert

There is a rationale for the Melissa space for the LR readouts and histograms. Editing in a linear space has many advantages, but a main disadvantage is that a linear space is not perceptually uniform. A small change in the sliders would have a much greater impact on the darker tones than the highlight tones so that the sliders would be touchy (overly sensitive) in the shadows. The Melissa response for the sliders and histograms enables one to edit with a linear tone curve, but with the appearance to the user that he/she is using a more perceptually uniform space with a sRGB tone curve.

Bill

[Robert Ardill]

What is needed is an intelligent CMM that looks at the colors actually in the image in order to determine the amount of compression needed and which colors need to be compressed.

Thanks Bill.

I can't help feeling that what we currently have is a solution that has grown (slowly and painfully) to fix a problem (or rather, lots of problems) and what may be needed is to clear the decks and start again.  If you think about it: we have an input device that's wrong; a display device that's wrong; an output device that's wrong ... and so what we do is to try to fix the wrongness in all of them so that they can sort of talk to each other. It's like UN translators using Esperanto as an intermediate language!

Graeme Gill of ArgyllCMS said this to me today: "Figuring out what the applications that use the profiles are actually doing is the hard part because typically the software vendors give you no clue, and hide behind inscrutable "user friendly" buttons such as 'Simulate Paper Color'". 

He's a guy who's a giant in this business, and you can just hear the frustration in his words!

I can't see any particular reason why all devices couldn't have the intelligence to understand a common language directly ... and for it to be their problem to follow the orders (to the best of their ability, taking into account their inherent limitations).

But I suppose that would be akin to getting the world to speak one language ... maybe your intelligent CMM has better chances of seeing the light of day!  It remains to be seen how 'intelligent' it manages to be

Robert

[Robert Ardill]

There is a rationale for the Melissa space for the LR readouts and histograms. Editing in a linear space has many advantages, but a main disadvantage is that a linear space is not perceptually uniform. A small change in the sliders would have a much greater impact on the darker tones than the highlight tones so that the sliders would be touchy (overly sensitive) in the shadows. The Melissa response for the sliders and histograms enables one to edit with a linear tone curve, but with the appearance to the user that he/she is using a more perceptually uniform space with a sRGB tone curve.

Bill

Thanks for that explanation Bill.  So effectively we can ignore what happens below Melissa RGB, right?  That's just the internals ... and it doesn't introduce any risks that we need to be aware of?

Robert

[Jim Kasson]

Enough to give anyone a headache!!  I suppose there has to be some rationale to all of this, but it seems that it's piling complexity on top of complexity ... and one wonders how the poor CMMs (not to mention us) can hope to see the wood from the trees!

I think the rationale for programs like Lr is to provide a simpler user experience by hiding the complexity. Ps takes the other approach for the most part, exposing it completely (if you know where to look). The advantage of the Lr approach is that it provides an easier-to-learn alternative. The disadvantage is that advanced users sometimes can produce better results when they know what's going on under the covers (or inside the black box), and the Lr approach is much less transparent than the Ps approach.

If you want a more transparent raw developer, you might look at Iridient. Only runs on the Mac, though, and doesn't have the organizer/printing/publishing capabilities of Lr.

Jim