New Approach for Generating Optimal Profile Patch Sets

[Doug Gray]

I would like to point out that profile building software also do not produce random combinations of C,M,Y for the same black generation settings (fixed K). After black generation is decided upon, and total ink limits, the recipe for C,M,Y is consistent for each profiling software's programmed logic. I.e., you are not going to get a different profile each time you re-run the build. In this regard, it is not really different from how "rgb" printer drivers work.

This is giving me an idea of how to get it to work. One needs a preliminary CMYK profile built, with one's final choice of black generation settings. Convert a patch set of neutral and near-neutral RGB samples into CMYK printer space using this preliminary profile. Those CMYK combinations will then be sampling and bracketing the neutral spine of the printer when re-purposed as profiling target patches. Then combine with i1Profiler or Argyll's targen auto-generated target for CMYK.

What do you think about this method?

Seems like a good approach. Certainly worth a shot.

My main concern is with out of gamut mapping. It is undefined by the ICC and it is also the greatest source of grief for me. No profiling solution available right now exists which does the right kind of mapping. ArgyllCMS comes closest to a 20 plus year old software which I used as inspiration to advise and suggest to Graeme a much better way of performing gamut mapping. The mapping in Argyll is still not ideal yet, but it is all I have now given that I've got no coding knowledge.

I don't often encounter materially OOG images but the idea of a luminance preserving approach seems good. If I needed to do that I'd approach it from the perspective of preprocessing the image to map each OOG pixel into the closest in-gamut value that preserves L* as well as hue. The problem, of course, is that this will create banding at the gamut boundary. OTOH, letting the hue vary while fixing L* is problematic too. Something has to give. One could create a transitioning region to prevent banding but this would reduce the saturation near gamut edges. In effect one would be creating a Perceptual mapping prior to color managed colorimetric printing. I also think this would be best done in the profiling s/w. Since I have little need in this area I haven't looked at Graeme's code. Graeme certainly would have the best insight on this.

[Doug Gray]

I'm currently working on a C++ program (Windows only executable but I'll put the source code in github for anyone that wishes to port it to Macs.) that produces packed RGB profiling CGATs sets. It uses Marti Maria's open source C LittleCMS to generate a tracking neutral RGB set from an existing profile. I've already verified the guts of it and am now cleaning it up to make it easier for people to use with I1Pro. I expect to finish it in a day or so.

After some experimentation, I've found the best results on the neutrals, which is critical for B&W using color profiles, is just a large number of tracking, actual neutrals, rather than staggered, near neutrals when there is a good, existing profile. When there isn't, such as making the first profile of a new paper, staggered, near neutrals over device space produces the best profiles. So the program will allow for both of these.

Also, I've found that the Pro1000 printer colors shift slightly after printing the first page and subsequent pages are much closer. Thus, my profiling (and testing) process now includes printing duplicates of the first page and discarding the initial print. This warm up effect is, surprisingly, a source of error that becomes quite significant with patch sizes over about 1K where the effect becomes larger than the improvement in accuracy from more patches.

[Doug Gray]

Have an operational program. It has two main modes. Initial mode is generating a patch set that has packed grids as well as near neutrals which does a pretty decent job of printing along the neutral axis.

The second mode uses a profile created from the first set to generate patch sets with tracking neutrals and near neutrals. This set can also use a programmable, larger number of packed grids for more accurate color rendition. I've also added options to duplicate and scramble patches to reduce the intrinsic error associated with printing the same patch in different locations.

I'm going to test this on the 9800 as well since it's rather lumpy on the neutral axis and I have extensive prior work optimizing its neutrals. I'm optimistic based on current P1000 tracking results.

Rather interesting coding this in C++. Debugging is a bit harder but the tools have really progressed. It requires a bit of a different mindset with a compiled program v Matlab which is interpreted and highly interactive.

[Alan Goldhammer]

Have an operational program. It has two main modes. Initial mode is generating a patch set that has packed grids as well as near neutrals which does a pretty decent job of printing along the neutral axis.

The second mode uses a profile created from the first set to generate patch sets with tracking neutrals and near neutrals. This set can also use a programmable, larger number of packed grids for more accurate color rendition. I've also added options to duplicate and scramble patches to reduce the intrinsic error associated with printing the same patch in different locations.

How many patches are in the set?  Can it run on Argyll for profile target generation and profile construction?

[vikcious]

Have an operational program. It has two main modes. Initial mode is generating a patch set that has packed grids as well as near neutrals which does a pretty decent job of printing along the neutral axis.

The second mode uses a profile created from the first set to generate patch sets with tracking neutrals and near neutrals. This set can also use a programmable, larger number of packed grids for more accurate color rendition. I've also added options to duplicate and scramble patches to reduce the intrinsic error associated with printing the same patch in different locations.

I'm going to test this on the 9800 as well since it's rather lumpy on the neutral axis and I have extensive prior work optimizing its neutrals. I'm optimistic based on current P1000 tracking results.

Rather interesting coding this in C++. Debugging is a bit harder but the tools have really progressed. It requires a bit of a different mindset with a compiled program v Matlab which is interpreted and highly interactive.

Good news! Really anxious about this. looking forward for a beta when available!

[Doug Gray]

Here's the program executable. Unzipped there are 3 files, ICCProfilePatches.exe and LCMS2.DLL. Keep both in the same directory. Here's the readme.txt:

Program ICCProfilePatches

There are two modes of operation. For those uncomfortable with commmand line programs it operates interactively. Just start the program in a command prompt window by typing ICCProfilePatches. It asks whether you want an initial, small patch set to characterize the neutral axis. If you already have a good profile you can skip this step. If not, answer (y)es and it will create a small, 232 patch set, with an outer grid of 5x5x5 and inner of 4x4x4 along with 50 added device rgb neutrals. It always prints the same RGB set so you only need to run this once to create a training profile. It's not very good for colorsbut has a pretty large set of near neutrals and will do a decent job printing B&W. A profile from this set is then used to generate near neutrals along the actual neutral curve in the next step. This only needs to be done once for any given paper type and printer. You can use the same profile in the future when making the larger tracking profile so long as the printer and paper type (MK v PK) are the same.

The other option is used to create the larger patch set with printer tracking neutrals. You have an option to enter the grid size plus the number of tracking, near neutrals. The grid size should be no less than 6 which will make under 400 patches with 70 neutrals. Adjust for your instrument. I use 8 or 10 for 1 or 2 iSis letter size pages. You will be asked to enter the profile to track to which can be from the first step or any other good profile you have for that printer and paper type.

That's it. You get CGATs RGB patches that can be loaded into I1Profiler or Argyll to create images and profiles. Feel free to experiment with different grid sizes and neutral counts to see what the totol patch size comes out to.

This is a copy of the two interactive modes I recently ran.

ICCProfilePatches
ICCProfilePatches V0.1
Create small patch set to generate profile for initial tracking? (y/n): y
RGB patches in outer grid:125
RGB patches in grids:189
RGB patches in grid after removal of neutrals:182
Using device neutrals
Added tracking neutrals:50
Total Patches:232

ICCProfilePatches
ICCProfilePatches V0.1
Create small patch set to generate profile for initial tracking? (y/n): n
How many values are on a major edge? (suggest between 6 to 12): 8
How many Tracking neutrals do you wish to insert? (suggest between 30 to 200): 60
Enter name of profile to create Rel. Col. neutral tracking from: SmallTargetPatches_25-03-19.icm
RGB patches in outer grid:512
RGB patches in grids:855
RGB patches in grid after removal of neutrals:842
Added tracking neutrals:60
Total Patches:902


For the brave, here's the options to runt the program directly from a command. The most useful feature not found in the interactive mode is the "-r Dups" which duplicates all the patches "Dups" times. It also randomizes the patches since the location of a patch can and does alter the actual Lab color when it's scanned. I see a standard deviation of about .20 dE00 from printing patches at different locations while just printing the same pattern on separate sheets can have half that variation. When the patches are randomized, an index file is printed out that can be loaded into Excel and used to sort measurement CGATs files to de-random,ize them. The duplicate patches are averaged by I1Profiler and provide slight improvements. As the grid size increases, the improvement from averaging increases. Particularly on good, smooth, printers like the Pro1000. Basically, there is a tradeoff between the variation of same patch color v linearity of the grid which improves as the grid points increase.

See readme file for options.


This was the result on my 9800 which is more "lumpy" than the Pro1000. It was quite good, only slightly worse on color patches than the 3828 patch set I was using on the 9800. However, it was slightly better on the neutrals. Getting the  neutrals dE00 significantly below the color dE00s is rare since dE00 exaggerates magnifies Lab differences on or near the neutrals to better indicate where vision is more sensitive to small tints while suppressing differences once the colors become even slightly saturated and vision is less sensitive.

Profile: 9800 Two stage, optimized
Mean dE00, Color patches: 0.45  Worst 10%: 0.90
Mean dE00, Neutral patches: 0.38  Worst 10%: 0.63

[Alan Goldhammer]

Doug, thanks for posting the program.  I have a couple of new papers to profile and I'll see how this works in Argyll (I'll need to figure out how to paste the generated CGATs file into the appropriate Argyll 'ti' file.

[samueljohnchia]

Hi Doug, thank you for sharing so generously. I hope to try this out for myself soon. Quick question first, how are you arriving at the near neutrals tracking sampling from the preliminary profile? This sounds kinda like the approach I mentioned with CMYK earlier. Just wondering.

Getting the  neutrals dE00 significantly below the color dE00s is rare since dE00 exaggerates Lab differences on or near the neutrals while suppressing differences once the colors become even slightly saturated.

Doesn't this correlate better with human perception? Don't we tend to notice colour casts in neutrals more easily than in colours?

[Doug Gray]

Hi Doug, thank you for sharing so generously. I hope to try this out for myself soon. Quick question first, how are you arriving at the near neutrals tracking sampling from the preliminary profile? This sounds kinda like the approach I mentioned with CMYK earlier. Just wondering.

What the program does is create a series of evenly spaced steps in L* from the black point to 100 (in Re. Col.) with a* and b* set to 0. It's then runs these through the supplied profile to generate RGB values that correspond to this. Then the RGB values are adjusted slightly with each channel in turn increased a small amount while the other two are decreased a smaller amount. This produces staggered near neutrals much like what the I1Profiler's patch generator does with near neutrals except that these more closely track the actual neutrals.

Doesn't this correlate better with human perception? Don't we tend to notice colour casts in neutrals more easily than in colours?

Yes, you are quite correct. I was trying to say dE00 typically magnifies Lab differences and "exaggerates" implies it increases dE00 inappropriately. I'll fix that post.

[rasworth]

This may help some visualize the results of Doug's program.

I picked a very small color set and 50 neutrals to emphasize the neutrals treatment.  I "trained" Doug's patch generator with an existing profile, generated the patch set, brought it into ColorThink Pro, attached the printer training profile, and graphed it in Lab space.  The image clearly shows the new neutrals hugging the L axis in the training profile space, and the three sets of near neutrals now closer to the axis.

Richard Southworth

Added by edit - For clarity's sake, the next step would be to use the new patch set to generate another printer profile, taking advantage of the nearer neutrals.

[rasworth]

I just discovered a "feature" of Doug's program - if one gives it an invalid training profile the program reverts to untrained neutrals.  The attached image shows the results, with the untrained neutrals bending away from the L axis.

Richard Southworth

Modified to insert more informative image.

[Doug Gray]

I just discovered a "feature" of Doug's program - if one gives it an invalid training profile the program reverts to untrained neutrals.
Richard Southworth

Yep. That's intentional and it prints out a comment that it couldn't find the icc profile. I felt it best to default to using device space neutrals rather than just abandoning things. Still makes a pretty good patch set with all the extra neutrals even if they aren't tracking since it couldn't find the reference profile.

[rasworth]

You may want to insert a pause or two, to keep the program from exiting before messages are read.

I invoked your executable directly, without first establishing a command window, and all worked fine except the program exited with no chance to read the final messages.  I realize the proper method is to bring up a command window before any execution, but it is very convenient to double click the executable directly.

Richard Southworth

[dehnhaide]

You may want to insert a pause or two, to keep the program from exiting before messages are read.

I invoked your executable directly, without first establishing a command window, and all worked fine except the program exited with no chance to read the final messages.  I realize the proper method is to bring up a command window before any execution, but it is very convenient to double click the executable directly.

Richard Southworth

Not exactly what would be expected as norm from command line executable. Otherwise it would mean that the Doug's executable would have to first call on the "cmd" and than execute it's code... That would be lame. 99% percent of Windows binaries that require interaction need be called from a cmd prompt.

[rasworth]

Doug has created a program that runs in two modes, either interactive or full command line.  I double clicked his executable, it fired up and started asking me questions, i.e. full interactive mode.  IMO it's a perfectly logical approach to using the program, works like any other W10 window, just needs a little more work to make it fully interactive, i.e. don't finish the program and disappear without giving the user a chance to read the messages.

And I will wager 9 out of 10 who first try this program will start it the first time with no command window.

Richard Southworth

[rasworth]

Doug,

Doing some boundary value checking - entering "0" for edge value works ok, gives only neutrals.  However entering "1" gives an invalid rgb triplet for the first entry.

"2" is sort of a sweet spot, gives 0-0-0 and 255-255-255 plus the 6 fully saturated rgb values.  It's the right setting for generating neutrals targets.

Ok, I'll quit trying to break your program, it's a significant contribution to the profile maker's toolset.

Richard Southworth

[Doug Gray]

Doug,

Doing some boundary value checking - entering "0" for edge value works ok, gives only neutrals.  However entering "1" gives an invalid rgb triplet for the first entry.

"2" is sort of a sweet spot, gives 0-0-0 and 255-255-255 plus the 6 fully saturated rgb values.  It's the right setting for generating neutrals targets.

Ok, I'll quit trying to break your program, it's a significant contribution to the profile maker's toolset.

Richard Southworth

Richard,  These are useful suggestions and there is definitely is a use case for creating only staggered neutrals so people can add it to a good performing patch set they already have.

Thanks for exploring things and making these suggestions.

[Doug Gray]

The program has been modified per Richard's suggestions. Thanks, Richard!

[rasworth]

Doug,

You're welcome, version 2 runs great!  Thanks again for your contribution.

Richard Southworth

[Alan Goldhammer]

The program has been modified per Richard's suggestions. Thanks, Richard!

Where is Version 2 posted?