New Approach for Generating Optimal Profile Patch Sets

[Doug Gray]

Mark Segal,

Excellent results with the 2.3k patch set.  As for the gamut size differences, I consider that irrelevant as it occurs at the outer boundary. The 1% difference should be looked at as a .2% difference in reality since the outer boundary differences would be visually strongly attenuated by the dE00 formula. Also, it's attenuated by an additional factor of 2 due to geometrical aspects (area increases twice as fast as linear dimensions).

The neutral patch results are interesting. One of the thing's I've noticed is that deviations along a* are much more sensitive in dE00 than b* while L* deviations are less sensitive. But L* and a* effects are magnified more than dE00 suggests in areas of low luminance as MHMG discusses from the perspective of I*.

Edit: a* and b* were reversed. a* is about 50% more sensitive than b*.

[Mark D Segal]

Mark Segal,

Excellent results with the 2.3k patch set.  As for the gamut size differences, I consider that irrelevant as it occurs at the outer boundary. ...........

Yes, sure. Not only that, a high proportion of our photos don't need gamuts of those sizes in the first place, but it depends also on the gamut shape relative to the more saturated colours in the photo.

[Doug Gray]

Yes, sure. Not only that, a high proportion of our photos don't need gamuts of those sizes in the first place, but it depends also on the gamut shape relative to the more saturated colours in the photo.

Yep, that too. As I've often said, I rarely look at gamut size except for looking at different printer's capabilities and even then it's less significant than it appears.

[Doug Gray]

I've been concentrating on making the smallest sets of patches that produce good quality results for the Pro1000 with glossy type papers.

There are two issues:

The most significant is that dE00 is much more sensitive to Lab changes along the neutral axis where it is sometimes magnified than in colors with any significant degree of saturation where it's attenuated. This causes average dE00 on neutrals to exceed average dE00 on color patches. The opposite is observed with delta E 1976 (dE76) where neutrals have less variance than color patches.

The other issue is variations in Lab on when the same color is printed in multiple places. The basic problem is that the variation between the same color patches printed in different locations are nearly as large as how well the profiles. To mitigate the matter effect and bring out variations due to the profile itself I have made a known color patch set that is repeated 5 times and randomized.

The patch set contains 2 collections of Lab values. The larger one, with 149 patches, contains evenly distributed colors that are in gamut for glossy type papers on all three of my printers. The smaller one contains 42 evenly spaced L* neutrals. This makes it easy to compare things using the exact same set of data for any of my printers and glossy/luster paper. And it provides a reasonably broad set of color patches that are independent, which is critical, of the set used to create the profile.

This creates a single, letter size iSis page which is scanned. Each of the 5 duplicates are averaged to reduce the variations described above. This reduces same patch variation by about 60% and makes it easier to see the intrinsic profile performance.

Here we compare two profiles. The first is the default, US letter size 957 patch set profile. The second is also a 957 patch profile. It's created by using the N=8, packed grid with added tracked neutrals. The tracked neutrals were created by filling the remaining 957-(512+343)  with RGB values derived from evenly spaced L* values. Unlike the added neutrals in the posted collections, these RGB values differ from each other as they are based on what the default profile calculates is needed to produce neutrals using Rel. Col. This makes a printer/paper tailored patch set that does an excellent job reducing dE00 on neutrals.

Here's the results of the 149 color and 37 neutral, in-gamut, patches for the default profile:

Profile: iSis Default
Mean dE00, Color patches: 0.31  Worst 10%: 0.59
Mean dE00, Neutral patches: 0.54  Worst 10%: 1.69

Note how the dE00 is worse on the neutrals. This is unlike the dE76 which is simple and still commonly used. Here's the dE76 results:

Profile: iSis Default
Mean dE76, Color patches: 0.50  Worst 10%: 0.98
Mean dE76, Neutral patches: 0.51  Worst 10%: 1.44


And here's the results from the packed grid with tracking profile. The color patches are slightly better but the neutrals are much better.

Profile: N8 and Adapted Neutrals
Mean dE00, Color patches: 0.28  Worst 10%: 0.54
Mean dE00, Neutral patches: 0.27  Worst 10%: 0.54

For comparison using delta E 1976

Profile: N8 and Adapted Neutrals
Mean dE76, Color patches: 0.46  Worst 10%: 0.88
Mean dE76, Neutral patches: 0.27  Worst 10%: 0.62


Overall, pretty good indication that the packed grid combined with a paper/printer specific patch set, yields significant improvements. Especially in the neutrals.

Attached chart is the dE00 histogram of the 37 L* neutral patches for the default profile and the packed grid profile /w tracked neutrals. Both on 957 patch iSis letter size charts

[rasworth]

Doug,

I don't understand "The other issue is variations in Lab on when the same color is printed in multiple places."

Is this a variation for horizontal location, or print to print, or what?

Richard Southworth

[dehnhaide]

Unlike the added neutrals in the posted collections, these RGB values differ from each other as they are based on what the default profile calculates is needed to produce neutrals using Rel. Col. This makes a printer/paper tailored patch set that does an excellent job reducing dE00 on neutrals.

Dear Doug,
Indeed the measured improvement in dE00 for the neutrals is outstanding but I don't get it how should I mofifuythe neutral values in your publish sets to accomplish the same. Care to explain, or maybe attach a modified say N8 for this purpose?

[Doug Gray]

Doug,

I don't understand "The other issue is variations in Lab on when the same color is printed in multiple places."

Is this a variation for horizontal location, or print to print, or what?

Richard Southworth

Both. The relative affect of each varies with printers. And patches can have varied readings just reading the same ones a second time though those are tiny. The bigger one is the horizontal location a patch is printed. It's affected by tracking head height variations which depends on the physical design. There are also warm up changes, and how long it's been since an image was last printed. Also, over a longer term, there are bigger changes as the ink volume decreases and it gradually gets more dense as the as fraction of the solvent slowly evaporates. For testing profiles my protocol is to duplicate and throw away the first print.

Easy, if tedious, way to test it is just duplicate color patches in the same or random locations and measure them.

[Doug Gray]

Dear Doug,
Indeed the measured improvement in dE00 for the neutrals is outstanding but I don't get it how should I mofifuythe neutral values in your publish sets to accomplish the same. Care to explain, or maybe attach a modified say N8 for this purpose?

Unfortunately, this is unique to each printer/paper type and somewhat hard. It also requires an existing, initial profile.   One approach is to create a set of neutral Lab patches in Photoshop. Then convert to printer device space. Add those RGB values at the end of a CGATs file. Really easy to do in Matlab. I wouldn't be surprised if Argyll has a tool to convert a Lab CGATs file to RGB in device space.

[rasworth]

Doug,

Not intending to divert your thread, I believe this is relevant.

I dissected an i1Profiler 915 auto generated patch target input file.  In addition to the 9 X 9 X 9 RGB color triplets there were 20 R=G=B neutrals, and 60 near neutrals.  The 60 RGB near neutrals were in sets of three, each entry with one of the three channels slightly different than the other two.  I attached an excerpt from the patch generation file for clarity.

I assume the intent is to gain accuracy for the profile neutrals.  What is your opinion as to the value of this technique?

Richard Southworth

[GWGill]

I wouldn't be surprised if Argyll has a tool to convert a Lab CGATs file to RGB in device space.

Currently there is no automatic way, although it is possible using icclu and some manual editing. I have added a new option to targen for the next release of ArgyllCMS that allows generation of neutral axis wedge steps using the pre-conditioning profile, which will make this approach much more convenient. (Of course the -N parameter already allows control of the neutral axis patch density for the full spread patches generation, but generating patches right on the neutral axis may be a more efficient supplement.)

[Doug Gray]

Currently there is no automatic way, although it is possible using icclu and some manual editing. I have added a new option to targen for the next release of ArgyllCMS that allows generation of neutral axis wedge steps using the pre-conditioning profile, which will make this approach much more convenient. (Of course the -N parameter already allows control of the neutral axis patch density for the full spread patches generation, but generating patches right on the neutral axis may be a more efficient supplement.)

Graeme,

I've also tried patch sets with narrow spreads around the neutral axis. It takes more patches and the results weren't better so I reverted to just a simple set of neutral axis values as determined by the BtoA1 tables. It worked quite well so I halted some experiments along these lines.

There is one area where the application may be determinative. Rel. Col. v Abs. Col. Selecting either the Rel. Col neutral axis which tracks the paper white, or selecting the D50 neutral axis which forces a*=b*=0 might be application specific. With M2 profiles these are close enough that I don't think there is any practical difference but M0/1 with high OBAs might be another matter.

In terms of what folks can do w/o Matlab, I believe PatchTool has the ability to run a CGATs file through the BtoA1 profile LUTs and create scannable I1Pro charts. It also formats and creates printable charts from fractional (16 bit)  CGATs files. It's not what I use so I'm unable to provide a recipe but it looks like it would work.

[Doug Gray]

Doug,

Not intending to divert your thread, I believe this is relevant.

I dissected an i1Profiler 915 auto generated patch target input file.  In addition to the 9 X 9 X 9 RGB color triplets there were 20 R=G=B neutrals, and 60 near neutrals.  The 60 RGB near neutrals were in sets of three, each entry with one of the three channels slightly different than the other two.  I attached an excerpt from the patch generation file for clarity.

I assume the intent is to gain accuracy for the profile neutrals.  What is your opinion as to the value of this technique?

Richard Southworth

It's interesting. The neutrals help with the dE00 sensitivity. At least as long as the printer doesn't deviate too far from the actual neutral in device space. I used a smaller space between neutrals because the 9800 is very lumpy, the Pro1000 has larger deviations but is less lumpy. Using more neutrals helped for both of these.  However, using RGB "tracking neutrals" from the BtoA1 profile tables produces patches that are close to actual neutral and hence interpolation errors are less.

As for the 60. It has an interesting pattern of increasing the spacing but maintaining a fairly constant ratio of about 20% for the larger value as it walks up. Interesting concept. At the larger values it's not going to be very effective for getting close dE00 tracking along the neutrals. But it might work well at low L*.

[Alan Goldhammer]

Currently there is no automatic way, although it is possible using icclu and some manual editing. I have added a new option to targen for the next release of ArgyllCMS that allows generation of neutral axis wedge steps using the pre-conditioning profile, which will make this approach much more convenient. (Of course the -N parameter already allows control of the neutral axis patch density for the full spread patches generation, but generating patches right on the neutral axis may be a more efficient supplement.)

I look forward to trying out the new release!  You also describe my workflow.  I use a 924 patch set for the original profile (two letter size sheets) and then use that as the pre-condition for the second one with the -N parameter along with an extra set of B/W patches as well.  The second patch set is 1848 (four letter size sheets).  I have not done the exhaustive type of study of the profiles (Epson 3880) that Doug has for his Canon printer.

[Alan Goldhammer]

I agree that a printed, continuous gradient is a good way to view smoothness and that's where eyeballs are much better than spectros. The latter is excellent for determining accuracy but not so good at smoothness unless you scan tens of thousands of patches. And one can enhance the latter by scanning or photographing the gradient and increasing contrast. How far contrast can be increased and still retain visual smoothness (noise aside), would be a good metric. At least for that particular gradient. Analyzing the smoothness of the entire gamut is a very hard problem.

When you print out your neutral patch set for profile analysis, what settings do you use?  I've finally got my i1Pro set up with Win10 and want to look at the Moab Entrada profile that I created using a two step process with Argyll.  I have a 100 step B/W generated patch set that is randomized.  I don't want to go beyond this right now.

[Doug Gray]

When you print out your neutral patch set for profile analysis, what settings do you use?  I've finally got my i1Pro set up with Win10 and want to look at the Moab Entrada profile that I created using a two step process with Argyll.  I have a 100 step B/W generated patch set that is randomized.  I don't want to go beyond this right now.

My current profile validation reference patch set consists of 149 color patches that are in gamut and spread evenly across all printer glossy gamuts in Lab space as well as a 42 patch set of neutrals evenly spread from L*=0 to 100. OOG neutral patches, which occur near L=0 and 100, are not counted in the statistics.  Each of these 191 patch sets is duplicated 5 times and randomized to fit on a single iSis page. The duplication provides more precise info and cuts the error related to printing in different locations by about 60% while not changing error related to the profile accuracy. It also provides independent info on the error associated with printing the same color in different locations.

I have a bunch of automated scripts that suss out the info of interest to compare printers/paper. Generally the 9500 II is less accurate and varies more than the 9800 and Pro1000. The 9800 produces the most repeatable colors printed in different locations but profile accuracy isn't as good with as gradients are lumpier over short color gradients. The Pro1000 produces the best overall color accuracy and slightly smoother neutrals. However, both the 9800 and Pro1000 are fairly close to each other overall. The principal traits of the Pro1000 are larger gamut and lower L* mins.

[Alan Goldhammer]

My current profile validation reference patch set consists of 149 color patches that are in gamut and spread evenly across all printer glossy gamuts in Lab space as well as a 42 patch set of neutrals evenly spread from L*=0 to 100. OOG neutral patches, which occur near L=0 and 100, are not counted in the statistics.  Each of these 191 patch sets is duplicated 5 times and randomized to fit on a single iSis page. The duplication provides more precise info and cuts the error related to printing in different locations by about 60% while not changing error related to the profile accuracy. It also provides independent info on the error associated with printing the same color in different locations.

Thanks, I don't think I can create a small color patch set from your parameters using Argyll though I think Graeme is coming up with a newer version that might be able to do that.  I was more interested in you Photoshop setting for printing regarding the rendering intent or whether that makes a difference at all.

[Doug Gray]

Thanks, I don't think I can create a small color patch set from your parameters using Argyll though I think Graeme is coming up with a newer version that might be able to do that.  I was more interested in you Photoshop setting for printing regarding the rendering intent or whether that makes a difference at all.

I believe Graeme mentioned a technique that would work with the current Argyll programs but would require adding in the additional patches and editing a CGATs file. Makes sense to me but the Matlab tools I have pretty much automate things so I haven't looked at it.

I print 16 bit tiff verification charts using Abs. Col. intent selecting the profile being tested. I print charts used to create profiles using the so called "null profile trick" where I assign an untagged chart with an arbitrary printer profile. This is then printed using the same profile. The actual printer profile used makes no difference. In this case only, rendering intent also has no effect. Photoshop sees this as the RGB values already being in device space so prints them directly ignoring rendering intent. This technique has always worked in Windows with all Photoshop versions since they discontinued the ability to print w/o color management. Canon's plugin also allows printing w/o color management producing identical results but is limited to current canon printers and doesn't even work on the 9500 II.

[samueljohnchia]

Hi Doug, I was just wondering how much of the thinking behind creating optimal patch sets for RGB printer profiling can be applied to CMYK profiling. I have been working with a company who has an HP Indigo printer to make some prototype books of late and will be going to offset to make the full run soon. At the moment I've resorted to i1Profiler's Smart Generator targets rather than customizing my own. I'm wondering how one would figure out an efficient way to sample the printer's CMYK space. Especially how to sample the neutral spine adequately. Its not clear what CMYK values will result in a neutral-ish output. And certainly many combinations of CMYK will. There is of course more mind-boggling options about black generation and total ink limits etc. I have tried the ArgyllCMS method of using a "pre-conditioning" profile to assist targen in more efficiently exploring the CMYK printer space. Unfortunately it crashes all the time when attempting larger patch sets around 3000 or so, and when I look at the sampling points in 3D graphing software, ironically they look less equidistant in printer space than i1Profiler's auto generated targets. Most unexpected since for RGB profiling, Argyll's target certainly is better than i1Profiler at this. Grame has on several occasions pointed out this advantage of Argyll to me, in fact also mentioned that Argyll's profile building math is more optimised for equidistant sampling exploration of printer space rather than the (i1Profiler) method of equidistant orthogonally in (arbitrary) RGB cube space.

I want to make the final profiles in Argyll to take advantage of the luminance-preserving gamut mapping which Graeme coded at my suggestions. I'm digressing a bit more now, but for anyone else interested in creating top-notch CMYK profiles, you'll be a little frustrated with Argyll as the black generation controls, while more sophisticated than i1Profiler in general, don't work as expected. It's a bit buggy in its behaviour and the black ink curve is resistant to being shaped in certain important ways, wanting to always have a convex shoulder at the shadow turning point. Another weird bug is sometimes you see is one gets 100% for K ink in perceptual rendering at dmax but only say ~98% or less in Relative Colorimetric rendering. A more serious issue is with how Argyll computes the CMYK ink combinations. The profile (observed in ColorThink Pro's neutral rendering curves) nearly reaches it's total ink limit at L*10, then reduces the ink limit as we proceed towards L*0 before gradually building up to the total ink limit again. This results in crazy looking ink curves. But actual printed result looks OK for some reason! Nonetheless this behaviour is annoying and certainly not beautiful. At normal profile table resolutions, Argyll profiles develop strongly visible Mach banding (as described by Graeme), while i1Profiler and another software I tried is able to render gradients smoothly despite the LUT resolution being smaller. I do know about the suggestion to use device-link profiling for far more precise conversions. But to be brutally honest, the differences in precision are not really visible in conversions of regular photographs for printing, and thus not worth the trouble (same goes for smart mapping workflows in Argyll) apart from Argyll's own buggy profiles. But when some "magic" combination(s) of commands produce(s) a beautifully rendering profile from Argyll, I must say it does work out quite well.

[Doug Gray]

Hi Doug, I was just wondering how much of the thinking behind creating optimal patch sets for RGB printer profiling can be applied to CMYK profiling. I have been working with a company who has an HP Indigo printer to make some prototype books of late and will be going to offset to make the full run soon. At the moment I've resorted to i1Profiler's Smart Generator targets rather than customizing my own. I'm wondering how one would figure out an efficient way to sample the printer's CMYK space.

I don't see a way to do this. With CYMK. let alone more ink versions, there is an infinite set of CYMK values that create the same Lab color. With "rgb" printers the algos are fixed so a device rgb value corresponds to only one Lab color and vice versa for in gamut Lab values.

 Especially how to sample the neutral spine adequately. Its not clear what CMYK values will result in a neutral-ish output. And certainly many combinations of CMYK will.

Exactly.

There is of course more mind-boggling options about black generation and total ink limits etc. I have tried the ArgyllCMS method of using a "pre-conditioning" profile to assist targen in more efficiently exploring the CMYK printer space. Unfortunately it crashes all the time when attempting larger patch sets around 3000 or so, and when I look at the sampling points in 3D graphing software, ironically they look less equidistant in printer space than i1Profiler's auto generated targets. Most unexpected since for RGB profiling, Argyll's target certainly is better than i1Profiler at this. Grame has on several occasions pointed out this advantage of Argyll to me, in fact also mentioned that Argyll's profile building math is more optimised for equidistant sampling exploration of printer space rather than the (i1Profiler) method of equidistant orthogonally in (arbitrary) RGB cube space.

I want to make the final profiles in Argyll to take advantage of the luminance-preserving gamut mapping which Graeme coded at my suggestions. I'm digressing a bit more now, but for anyone else interested in creating top-notch CMYK profiles, you'll be a little frustrated with Argyll as the black generation controls, while more sophisticated than i1Profiler in general, don't work as expected. It's a bit buggy in its behaviour and the black ink curve is resistant to being shaped in certain important ways, wanting to always have a convex shoulder at the shadow turning point. Another weird bug is sometimes you see is one gets 100% for K ink in perceptual rendering at dmax but only say ~98% or less in Relative Colorimetric rendering.

Perceptual mapping is undefined but Rel. Col. is.  It should maintain the CIExy values of the media white point down to the darkest color achievable with the same CIExy. Requesting a darker color than that is often mapped to a lower L* but with a shift in CIExy. But this is out of gamut mapping and is also undefined by the ICC.

A more serious issue is with how Argyll computes the CMYK ink combinations. The profile (observed in ColorThink Pro's neutral rendering curves) nearly reaches it's total ink limit at L*10, then reduces the ink limit as we proceed towards L*0 before gradually building up to the total ink limit again. This results in crazy looking ink curves. But actual printed result looks OK for some reason! Nonetheless this behaviour is annoying and certainly not beautiful. At normal profile table resolutions, Argyll profiles develop strongly visible Mach banding (as described by Graeme), while i1Profiler and another software I tried is able to render gradients smoothly despite the LUT resolution being smaller. I do know about the suggestion to use device-link profiling for far more precise conversions. But to be brutally honest, the differences in precision are not really visible in conversions of regular photographs for printing, and thus not worth the trouble (same goes for smart mapping workflows in Argyll) apart from Argyll's own buggy profiles. But when some "magic" combination(s) of commands produce(s) a beautifully rendering profile from Argyll, I must say it does work out quite well.

I have zero experience with RIPs and CYMK so I'm just commenting from what's defined by the ICC and the intrinsic math involved.

[samueljohnchia]

With "rgb" printers the algos are fixed so a device rgb value corresponds to only one Lab color and vice versa for in gamut Lab values.

I would like to point out that profile building software also do not produce random combinations of C,M,Y for the same black generation settings (fixed K). After black generation is decided upon, and total ink limits, the recipe for C,M,Y is consistent for each profiling software's programmed logic. I.e., you are not going to get a different profile each time you re-run the build. In this regard, it is not really different from how "rgb" printer drivers work.

This is giving me an idea of how to get it to work. One needs a preliminary CMYK profile built, with one's final choice of black generation settings. Convert a patch set of neutral and near-neutral RGB samples into CMYK printer space using this preliminary profile. Those CMYK combinations will then be sampling and bracketing the neutral spine of the printer when re-purposed as profiling target patches. Then combine with i1Profiler or Argyll's targen auto-generated target for CMYK.

What do you think about this method?

But this is out of gamut mapping and is also undefined by the ICC.

My main concern is with out of gamut mapping. It is undefined by the ICC and it is also the greatest source of grief for me. No profiling solution available right now exists which does the right kind of mapping. ArgyllCMS comes closest to a 20 plus year old software which I used as inspiration to advise and suggest to Graeme a much better way of performing gamut mapping. The mapping in Argyll is still not ideal yet, but it is all I have now given that I've got no coding knowledge.