New Approach for Generating Optimal Profile Patch Sets

[Doug Gray]

I've been considering a different approach to generating optimal profiling patch sets and have done some preliminary testing that is highly promising. It goes like this. Rather than generating evenly spaced interval RGB sets, making a two step approach.

1. Generate an evenly spaced RGB grid. For instance a 8 point, 3D grid with 8^3 RGB values.
2. Then generate a 7 point, 3D grid with the same spacing as the 8 point grid. This is then offset so each point in this grid is centered in each "cube" of the 8 point grid.

This makes a kind of packed structure that significantly improves both the average minimum distance and maximum distance of random RGB points which should produce better profiles compared to similar patch counts generated by I1Profiler. And, initial testing indicates it does! So far I've tested this on a single page chart with this dual grid set (512+343) plus adding additional neutrals to fill it out to the iSis 957 single page set.

And there is another cool effect. One can easily identify areas of the printer's RGB device space where non-linearities are largest. One can compare each of the centered RGB points to an average of the 8 corners it is surrounded with on the larger grid. If perfectly linear, the deltaE between the averaged 8 surrounds and the centered point would be 0. And this works. The distribution of these deltaE values turns out to be a great way to compare printer "lumpiness."  I've noted that my old 9800 is quite a bit more lumpy than the 9500 and especially the Pro1000 and I now have a way to create a mechanism to characterize this lumpiness. For the 9800 the average dE for the above case is .95 while the Pro1000 came in at .47. So this is a good explanation why the 9800 needs a lot more patches to reach the same level of color accuracy as the Pro1000.

This suggests further optimization can be done.

Make a patch set 12^3 grid points and add in a centered 11^3 grid. Print the patches and measure. Now take the original set of 12^3 (1728), sort the patches against the dE (which shows which patches are making a difference) , and add the worst dE sets back into the original 1728 until the total reaches a 2 page iSis set of 1914 patches.

I'm pretty sure this would make a better profile patch set for each of my printers. And I'd only have  to go through the process twice for each printer to cover the two main paper types. After that I'd be able to make great profiles for normal paper/ink and aging changes without printing and scanning bunches of pages each time.

[aaron125]

Very interesting indeed! Do you have a script or something which you could share so others may give this a try?

One thing I’m misunderstanding is how the 7^3 grid can have the same spacing as the 8^3 grid? Also, how does one centre the smaller 7^3 grid in between the cubes created by the 8^3 grid nodes?

What possessed you to come up with such an intriguing concept for this patch layout? Have you tried it with any other profile creation software? A friend of mine has access to BasICColor Print RGB and in the past I’ve sent him measurement files which he then processes through Print RGB with various settings (that app has an incredible array of settings and options which one can alter/play around with to produce profiles which alter a printer’s output in ways simply not possible to reproduce with i1Profiler). One great thing about that app is that it accepts any and all patch sets equally, afaik. I’ve not tested this but it essentially states this in the user manual and one need not use a specific or even a BasICColor produced app to create the charts/patch sets which are used to creating profiles with Print RGB.

One thing you haven’t mentioned is how do the profiles you’ve produced using this patch layout do with real world images and how is the smoothness of the transitions in said profiles? Accuracy is great, but for those of us in the photography and fine art markets, accuracy isn’t of great importance if transitions, especially how smooth they are, specifically in images which have (for example) blue skies, large areas of skin tones and other areas of this nature, in which smoothness of the transitions are of utmost importance.

Anyway, I’m certainly most interested in following the progress you have with this intriguing and innovative patch set layout procedure.


Sent from my iPhone using Tapatalk

[Doug Gray]

Very interesting indeed! Do you have a script or something which you could share so others may give this a try?

All my works uses scripts/functions I wrote in Matlab. They need the standard Matlab package with the image workshop which has a lot of useful color management and image related routines. I've made a number of tools that add on to this and do things like profile conversions, CGATs reading and writing. Matlab is extremely powerful and efficient and can do pretty much anything I've been able to imagine. The only exception so far was dealing with the issue of scanner wide area crosstalk. Even there I used  Matlab for analysis and investigation but it was too slow to use for correcting scanned images which required many billions of math operations. So for that I used C++ and spun out 3 worker threads to speed things up.

Matlab has a "home" version that is only for non-commercial use. It's full featured and only $149 for the base package and another $45 for the image workshop.  I also have a commercial version but it's about 5 years old which is largely compatible.

One thing I’m misunderstanding is how the 7^3 grid can have the same spacing as the 8^3 grid? Also, how does one centre the smaller 7^3 grid in between the cubes created by the 8^3 grid nodes?

Think of a ruled page with an 8x8" grid spaced 1" apart. Now put dots in the center of each square. The dots form a 7x7 grid but the spacing between dots is the same as the 8x8 grid. Extrapolate to 3D.

What possessed you to come up with such an intriguing concept for this patch layout?

I was thinking about the stacked marble problem. If you put marbles in a box, each directly on top of the ones below you get fewer marbles in the box than if you put each marble in the detents of the layer below. I then realized that every other layer of marbles was exactly centered on the middle of the below/above layers. And thus by averaging the Lab values of the 8 colors around each marble, I could easily determine how linear the printer engine was at each point. Then I ran a statistical test with Matlab to see how much such an arrangement improved the average distance and maximum distance of randomly picked RGB values. It reduces the average minimum distance by 25% and the maximum distance by 35%. While the patch size is increased by the second, centered set, the increase is not as much as the required increase for a single set that provides similar distances.

Have you tried it with any other profile creation software? A friend of mine has access to BasICColor Print RGB and in the past I’ve sent him measurement files which he then processes through Print RGB with various settings (that app has an incredible array of settings and options which one can alter/play around with to produce profiles which alter a printer’s output in ways simply not possible to reproduce with i1Profiler). One great thing about that app is that it accepts any and all patch sets equally, afaik. I’ve not tested this but it essentially states this in the user manual and one need not use a specific or even a BasICColor produced app to create the charts/patch sets which are used to creating profiles with Print RGB.

I tried the set of 8 points and 7 points which totals 855 patches and it produced better results than the iSis default 957 patch set. I think I could have trimmed it down to about 600 patches for a specific printer and still gotten better results than the default but I haven't tried that yet.

One thing you haven’t mentioned is how do the profiles you’ve produced using this patch layout do with real world images and how is the smoothness of the transitions in said profiles? Accuracy is great, but for those of us in the photography and fine art markets, accuracy isn’t of great importance if transitions, especially how smooth they are, specifically in images which have (for example) blue skies, large areas of skin tones and other areas of this nature, in which smoothness of the transitions are of utmost importance.

Yes, that's certainly an issue. One that I've done much work on for this is the 9800. There are areas, particularly near both ends of the luminance curve, where it exhibits fairly rapid shifts in a* and b* or L*. The 957 default patch set makes good profiles with the 9800 but I wound up creating a 3828 patch set specifically for printing on the 9800. It does a pretty good job on the smoothness but some of it is beyond what the I1Profiler can follow. Even at max quality (37x37x37 grid points)

Anyway, I’m certainly most interested in following the progress you have with this intriguing and innovative patch set layout procedure.


Sent from my iPhone using Tapatalk

[GWGill]

1. Generate an evenly spaced RGB grid. For instance a 8 point, 3D grid with 8^3 RGB values.
2. Then generate a 7 point, 3D grid with the same spacing as the 8 point grid. This is then offset so each point in this grid is centered in each "cube" of the 8 point grid.

Yep - that's called "body centered cubic". See the ArgyllCMS targen -i option.

[Doug Gray]

Yep - that's called "body centered cubic". See the ArgyllCMS targen -i option.

I'm not surprised Graeme. Should have checked there when I thought of that packing but it was trivial to do in Matlab and I wanted to get some stats on it which Matlab makes easy.

https://argyllcms.com/doc/targen.html#b

The -b parameter sets the outer edge size of the multidimensional body centered grid of test values. The total number of patches of this type will be the -b parameter value to the power of the number of colorants plus the (number-1) to the power of the number of colorants. The grid steps are evenly spaced in device space by default, but if the -p parameter is provided, then, then the steps will be distributed according to the power value. A body centered grid is a regular grid (see -m) with another smaller regular grid within it, at the centers of the outer grid. This grid arrangement is more space efficient than a regular grid (ie. for a given number of test points, it fills the space better.)

Have you by any chance looked at deltaE2000 distribution of the points centered around the surrounding cubes against the average of the 8 surrounds? Does that make a good way to reduce patches that are providing little benefit?

[Alan Goldhammer]

Most RGB printers are pretty well behaved in terms of linearity.  I routinely do a small patch pre-condition profile and then refine it with a larger patch set with increased neutrals for my Epson 3880.  This gives good results in terms of smoothness of the profile.  I only have an i1Pro and don't want to have a several thousand patch set that I have to read manually.  I've not tried any of the other 'targen' switches that are referred to in the above post.

[howardm]

So this begs the question......

How does one 'measure' or put a value on the smoothness or inversely, the lumpiness of a profile/printer/ink combo?

Do papers suffer/experience much in the way of non-linear response (lets assume the printer is linear for this)?

[Mark D Segal]

So this begs the question......

How does one 'measure' or put a value on the smoothness or inversely, the lumpiness of a profile/printer/ink combo?

Do papers suffer/experience much in the way of non-linear response (lets assume the printer is linear for this)?

I'm glad you raised that Howard, because it also triggers a question about a comment in a related thread from a poster who said that profiling accuracy is of less concern to him than the smoothness of tonal gradations. In a way, it's all related. Let's say your gradient file had values all the way up the scale that were unidirectional and just separated enough from each other to preserve apparent smoothness of the gradient. Unless the printer/paper/profile combo could render those file values accurately enough, one could observe discontinuities in the gradient if the extent of the inaccuracies were large enough and contiguous enough to be visible. So the first thing I would look at for quantifying smoothness is how well the system performs at rendering it accurately from a smooth gradient file, which is a doable proposition.

[aaron125]

So this begs the question......

How does one 'measure' or put a value on the smoothness or inversely, the lumpiness of a profile/printer/ink combo?

Do papers suffer/experience much in the way of non-linear response (lets assume the printer is linear for this)?

I don’t think there is a metric of which one can make an objective measurement to give smoothness a ‘score’ like one would when measuring the difference between some colours, for example, to obtain a dE value.

The smoothness of transitions afaik is a purely subjective thing where one must determine for themselves if a particular printer/substrate/ink/profile combination provides the smoothness they desire. Typically something like a Granger rainbow might be used as a kind of (still subjective, but at least it’s a known image and therefore can be used to visually compare and make a choice regarding the way a profile handles various transitions) benchmark to show how transitions in areas of the colour spectrum are handled by a profile.

I hope that’s not too confusing.


Sent from my iPhone using Tapatalk

[digitaldog]

Most RGB printers are pretty well behaved in terms of linearity. 

Expect just about every Epson printer (which is feed RGB data but isn't an RGB printer) using the native driver. Better than it was a decade ago but far, far from linear.

[Mark D Segal]

Expect just about every Epson printer (which is feed RGB data but isn't an RGB printer) using the native driver. Better than it was a decade ago but far, far from linear.

Andrew, not clear what you mean quantitatively by "far, far from linear", but my experience with the Epson SC-P5000 and various papers, printing a 100 patch grayscale ramp that increments one L* value at a time from L*1 to L*100 indicates that for the L channel, variance from reference values is usually well below dE 1 (for all the tones within gamut) and variance from zero of the a* and b* values also well less than dE 1 (both dE 76 - for dE 2000 the variances would be generally less). So to me, this seems not exactly linear, but pretty good.

[Doug Gray]

I'm glad you raised that Howard, because it also triggers a question about a comment in a related thread from a poster who said that profiling accuracy is of less concern to him than the smoothness of tonal gradations. In a way, it's all related. Let's say your gradient file had values all the way up the scale that were unidirectional and just separated enough from each other to preserve apparent smoothness of the gradient. Unless the printer/paper/profile combo could render those file values accurately enough, one could observe discontinuities in the gradient if the extent of the inaccuracies were large enough and contiguous enough to be visible. So the first thing I would look at for quantifying smoothness is how well the system performs at rendering it accurately from a smooth gradient file, which is a doable proposition.

I agree that a printed, continuous gradient is a good way to view smoothness and that's where eyeballs are much better than spectros. The latter is excellent for determining accuracy but not so good at smoothness unless you scan tens of thousands of patches. And one can enhance the latter by scanning or photographing the gradient and increasing contrast. How far contrast can be increased and still retain visual smoothness (noise aside), would be a good metric. At least for that particular gradient. Analyzing the smoothness of the entire gamut is a very hard problem.

[Mark D Segal]

I agree that a printed, continuous gradient is a good way to view smoothness and that's where eyeballs are much better than spectros. The latter is excellent for determining accuracy but not so good at smoothness unless you scan tens of thousands of patches. And one can enhance the latter by scanning or photographing the gradient and increasing contrast. How far contrast can be increased and still retain visual smoothness (noise aside), would be a good metric. At least for that particular gradient. Analyzing the smoothness of the entire gamut is a very hard problem.

To determine the appropriate number of test values needed for evaluating the apparent smoothness of a gradient with a spectro I think it's important to start with what constitutes a JND (just noticeable difference) in human visual perception. This vastly reduces the necessary number of patches. I agree that analyzing the smoothness of the entire gamut is a very hard problem - and that may be an understatement. One can sample it I suppose and come off with what is a practically useful appreciation.

[digitaldog]

Andrew, not clear what you mean quantitatively by "far, far from linear", but my experience with the Epson SC-P5000 and various papers, printing a 100 patch grayscale ramp that increments one L* value at a time from L*1 to L*100 indicates that for the L channel, variance from reference values is usually well below dE 1 (for all the tones within gamut) and variance from zero of the a* and b* values also well less than dE 1 (both dE 76 - for dE 2000 the variances would be generally less). So to me, this seems not exactly linear, but pretty good.

Take a gradient in any working space and print it without a profile, just as you would a target. Something like this:

[Mark D Segal]

I don't understand - What does that tell me about linearity?

[dehnhaide]

I don't understand - What does that tell me about linearity?

It could tell more about how well behaved the printer could be by default, prior to profiling. If that kind of pedigree is of any importance...

[digitaldog]

I don't understand - What does that tell me about linearity?

PRINT it. As discussed, probably with ACPU. Output with the native driver appear linear?

[digitaldog]

It could tell more about how well behaved the printer could be by default, prior to profiling. If that kind of pedigree is of any importance...

Exactly! Profiles do not make linearity when non linearity exists.

[Mark D Segal]

I don't see how I would learn much about the true extent of linearity just by printing that target and looking at it, because I would need to assess linearity with data. A visual check is indicative and helps detect major issues, which can be useful, but nothing much more refined.

[digitaldog]

I don't see how I would learn much about the true extent of linearity just by printing that target and looking at it, because I would need to assess linearity with data. A visual check is indicative and helps detect major issues, which can be useful, but nothing much more refined.

Of course it tells you a lot about the linearity or lack thereof based on the steps that block up (or don't) due to the behavior of the driver and media settings OF WHICH you'll profile from. Lots to learn once you actually make a few prints:
Try printing it without any color management in say ImagePrint (still have a copy?) vs. the native Epson driver. You'll see a major difference. But not until you make a print because unless you've experienced it, you've only imagined it.... 
And IF you have the hardware, you can measure each step IF you want actual numbers.