Hand Made CGATS Chart

[Brad P]

Thanks a lot Samuel.  Highly interesting and useful explanations for both dynamics. 

[samueljohnchia]

Brad, you're welcome. Glad it was useful to you.

For years I've been fussing over the same infinitesimally small details, and building and measuring and evaluating patch sets of all kinds. To save you a lot of grief and time, I can assure you that the improvements from such optimizations are tiny and non-visible in most prints if you're printing regular photographs and not synthetic targets designed to show every flaw in your setup.

You certainly don't need anywhere near 4000-5000 samples to profile your HP Z3200. Be careful that measurement errors from the Z3200's spectrometer isn't throwing off your final evaluations. Most differences between profiles made from finer vs coarse sampling are limited to the darkest shadow separation mostly, and of course subtle differences in grayscale smoothness and neutrality and perhaps better separation in the gamut boundaries, assuming your printer is mostly well-behaved across its gamut. And all these differences are miniscule.

The quality of the gamut mapping is far more important and you'll notice it in the majority of your prints. It can be well over an order of magnitude in difference. Ensure that the tone curve of your profile is not introducing an S-curve shape and watch the way out-of-gamut colors are mapped. For perceptual rendering beware that all profiling software other than RT Kit and Argyllcms (with the right commands) are not capable of preventing further compression of the in-gamut colors other than the movement required to map absolute white and black to paper white and ink black. That is, you'll notice that in-gamut colours for Perceptual are not mapping like they would be with Relative Colorimetric.

[Mark D Segal]

Hi Mark.  Apologies for not replying earlier.  Neither my equipment nor my profiling experience is as good or extensive as yours to be sure, and I do respect your opinion on these things even if I haven’t replied. 

..............

Anyway, typing all this out makes me agree with you that it probably is helpful to at least understand where I’m coming from and what I maybe still foolishly intend to do.

Hi Brad,

We're probably more aligned on much of this than may meet the eye. There is ever more to know and learn and to push the frontiers of our own limitations of technical knowledge, mathematical skill and experience. In that sense, regardless of what's been done before I have no hesitation whatsoever in revisiting, or seeing others revisit anything where there may be something different and innovative that could be helpful. It's that kind of field. However, in so doing, my sense is that the missiles should be well-guided to come to a successful landing and that's what prompted my questions. Serendipity can of course play a role - we can poke around here and there on a hunch and come up with pleasant surprises, but I was just wondering about the place for guidance from first principles, bodies of experience and especially knowledge of how the software develops conversion algorithms from the patches that could guide individual efforts. I don't pretend for a moment to know the intricacies under the hood - I work with the materials at hand to understand what they do and don't do and learn from useful experience of others.

Let's take the argument about the number of patches for example. We understand that this is one variable amongst others affecting profile quality, but perhaps not the most determinative. For example, based on Ethan Hansen's work, I have noticed that I achieved greater linearity and neutrality of the Luminance scale (but not by a sea-change) using X-Rite's 2371 patch set rather than the 1870. Years ago I was learning by working along with a true colour management expert when we tried to determine between numbers of patches and alternative profiling packages what combination landed on the "best" profiles in respect of accuracy and smoothness of tonal gradations. He had some sophisticated test chart images for visualizing especially the latter. We tested up to 6000 patches because he had an iSis and it turned out that we could see little difference moving much beyond a couple of thousand patches, but the choice of profiling software made the much more obvious differences between outcomes. Since then I've retained the notion that the math under the hood is perhaps more important than how we configure patch sets, and if that's the case, have we pretty much reached the limits to what more can be achieved with the latter?

[Doug Gray]

Hi Brad,
He had some sophisticated test chart images for visualizing especially the latter. We tested up to 6000 patches because he had an iSis and it turned out that we could see little difference moving much beyond a couple of thousand patches, but the choice of profiling software made the much more obvious differences between outcomes. Since then I've retained the notion that the math under the hood is perhaps more important than how we configure patch sets, and if that's the case, have we pretty much reached the limits to what more can be achieved with the latter?

I agree, There is not much to be gained, especially visually, with more patches beyond a few thousand. My particular problem relates to the 9800's lumpy neutral rendering where smaller dE00's are more critical.

The most intractable printing areas though are in low luminances. Here, the problem is that small L* changes put one in a different set of a*,b* grid areas in the BtoA tables and the results can be somewhat ragged. When printing with large areas of deap shadow these problems can become visible. It's not just the printer or more patches. It's a fundamental limitation of the BtoA tables.

As one increases the number of patches, especially in the darker regions, one can get better colorimetric tables on the AtoB side but this only improves soft proofing with normal color management engines.  However, it's possible to use the more accurate AtoB tables, which don't suffer from the intrinsically inaccurate mapping of the BtoA tables, to construct better printing. Essentially replacing the BtoA table lookup with a reverse interpolation.

I have a recently made a Matlab program that does this but it's slow. Each pixel in Lab takes about 50ms to find the optimal device RGB to render it's Lab value. Great for improving individual patches where but intolerably slow for fixing up an image with millions of pixels.  It should be possible to do do so with a C program and something like Marti's LCM within a reasonable time. Perhaps something like this exists? I don't see any other solution. More patches doesn't do a lot to improve profiles with their intrinsic clipping in the BtoA RelCol tables.

[Mark D Segal]

.........

The most intractable printing areas though are in low luminances. Here, the problem is that small L* changes put one in a different set of a*,b* grid areas in the BtoA tables and the results can be somewhat ragged. When printing with large areas of deap shadow these problems can become visible. It's not just the printer or more patches. It's a fundamental limitation of the BtoA tables.

.............

That's really interesting. I'm getting good linearity of the luminance scale and very low hue shifts along that scale with the Epson SC-P5000.

Have you tried using a RIP that bypasses the Epson driver, ImagePrint for example, to see whether your particular problem is attenuated by using their own profiles and driver for that printer? I ask because one of their main selling points is what they say to be the higher quality of their profiling and driver algorithms.

[Alan Goldhammer]

That's really interesting. I'm getting good linearity of the luminance scale and very low hue shifts along that scale with the Epson SC-P5000.

Have you tried using a RIP that bypasses the Epson driver, ImagePrint for example, to see whether your particular problem is attenuated by using their own profiles and driver for that printer? I ask because one of their main selling points is what they say to be the higher quality of their profiling and driver algorithms.

Of interest is whether subtle mathematical differences lead to visual ones.  Of course that's problematic in its own way because it is very hard to standardize 'blind' viewing experiments. 

[Doug Gray]

That's really interesting. I'm getting good linearity of the luminance scale and very low hue shifts along that scale with the Epson SC-P5000.

Have you tried using a RIP that bypasses the Epson driver, ImagePrint for example, to see whether your particular problem is attenuated by using their own profiles and driver for that printer? I ask because one of their main selling points is what they say to be the higher quality of their profiling and driver algorithms.

What I'm referring to are intrinsic interpolation errors due to the design of the ICC tables and applies as much to RIPs and the OEM drivers because it's a property of the ICC profile structure. The AtoB tables are broadly populated and printed colors stretch between 0-255 on each of the 3 channels. The I1Profiler max quality interpolation tables have a cube with about 50k internal points so the AtoB tables can be quite accurate. OTOH, the BtoA tables span Lab from L*=0 to 100, and a*-b* ranges from -128 to 127. Thus, in gamut colors, which are far more limited, only utilize a tiny portion of the cube space used for interpolation. There simply is more error when interpolation BtoA than AtoB as a result. This is why device link profiles. Which convert from one device space to another, are used instead of converting to Lab then to the second device space.

Normally, this isn't even close to a visible problem and introduces errors that are less than other errors from the printer/measurements/paper/ and time changes. But there are a few places it can show up. Mostly along low luminance gradients in hues that are perceptually sensitive. The area most sensitive is the violet/magenta axis. Printer gamuts in this region typically have gamuts extending well into the magenta. And they can increase their reach as much as 30 units with only a few units increase in L* since blue/magenta has only a small impact on L*. This can create a scalloping effect when a gradient runs along the boundaries of these cubes.

Here's an example of a Rel Col gradient from Lab(12, -8, 2) to Lab(20, -35, 8 ) showing the effect. The first two charts are the round tripped Lab values along the gradient (solid lines). The second chart is just an expanded region of the first that shows the scalloping. The dashed lines in both charts are the interpolated Lab values using only the AtoB tables. Since the colors are all in gamut and are interpolated from the requested Lab values using AtoB tables, they match the requested Lab values within < .1 dE.

The third chart is the dE00 between the requested Lab values and what would be printed using the BtoA tables for conversion to device space and the expected Lab values based on the much larger AtoB table. Pretty significant difference.

The fourth table is a way to visually bring out the scalloping effect. It's the difference between device RGB values  from each point on the gradient to the next which is about 0.3 saturation unit increases. The steps occur when moving between cell boundaries in the BtoA grid.

These affects are from the interaction of printer gamuts, particularly the magenta at low luminance, and the way ICC profiles are constructed. They occur on all printers but are worse in different areas depending on the printer's gamut.

BTW, this profile interpolation issue is almost entirely something that occurs in color areas. And only then when in a critical perception area. It's not significant in black and white with RelCol or Perc. And only a slight factor in Abs. Col.

This is also different from the way profiles handle OOG colors on the low end. While RelCol is defined for in gamut, it isn't for those that are out of gamut and is up to the designer. There is a legitimate issue with the way I1Profiler deals with the black point and below rendering. But within gamut it's pretty good for pure neutrals.

[Mark D Segal]

Of interest is whether subtle mathematical differences lead to visual ones.  Of course that's problematic in its own way because it is very hard to standardize 'blind' viewing experiments.

I think Doug goes some way to address this question in his reply just above.

I'd only add that in principle, one dE(00) is supposed to be a "just noticeable difference", so measured dE values <1.0 are presumably not noticeable; but I have also observed differences >1.0 that are also not noticeable. So it's not an exact science working between formulas and visual perception, but it's the best we have and on the whole I think not too bad.

[GWGill]

However, it's possible to use the more accurate AtoB tables, which don't suffer from the intrinsically inaccurate mapping of the BtoA tables, to construct better printing. Essentially replacing the BtoA table lookup with a reverse interpolation.

To some degree that's what you get using ArgyllCMS "collink -G" mode :- the device link is constructed from the A2B of the source space and inverting the A2B of the destination. There's still interpolation in the resulting link of course, but it's in terms of the source colorspace, which typically is better conforming to the destination gamut shape than an L*a*b* grid.

Reversing the A2B accurately is pretty slow, and any sort of sophisticated clipping is also likely to be slow.

[ My code breaks the grid cells into simplexes and caches the inversions of the simplex interpolation equations using as much physical memory as is available. There's a lookup table to do a rough inversion to get a list of candidate cells, and then they are searched. You do get all possible inversions (i.e. if the A2B is many to one), and so may have to choose which one to return. For CMYK you get the locus of solutions with different black inkings. ]

[Doug Gray]

To some degree that's what you get using ArgyllCMS "collink -G" mode :- the device link is constructed from the A2B of the source space and inverting the A2B of the destination. There's still interpolation in the resulting link of course, but it's in terms of the source colorspace, which typically is better conforming to the destination gamut shape than an L*a*b* grid.

Reversing the A2B accurately is pretty slow, and any sort of sophisticated clipping is also likely to be slow.

Tell me about it

[ My code breaks the grid cells into simplexes and caches the inversions of the simplex interpolation equations using as much physical memory as is available. There's a lookup table to do a rough inversion to get a list of candidate cells, and then they are searched. You do get all possible inversions (i.e. if the A2B is many to one), and so may have to choose which one to return. For CMYK you get the locus of solutions with different black inkings. ]

Thanks Graeme.