I1Profiler Optimization Pass Disappoints

[Doug Gray]

Decided to run a comparison of using I1Profiler's second, optimization pass and compare it to just using larger patch count with about the same total patches.

Since I1Profiler uses the first pass to select additional color patches and hence should know where the printer needs the most additional patches, I expected that the result would outperform just using a larger set of patches to begin with.

It didn't.

Patches  AveCol  AveNeu       0-15   15-25   25-35   35-45   45-55   55-65   65-75   75-85  85-100
  729       0.57    0.67          1.00     0.55     0.72     0.71     0.54     0.60     0.53     0.50     0.86       P1000_729_I1P.icm
 1369       0.49    0.49         0.79     0.44     0.45     0.56     0.41     0.39     0.45     0.36     0.52       P1000_729_Opt_I1P.icm
 1375       0.47    0.45         0.77     0.34     0.40     0.47     0.38     0.37     0.36     0.37     0.52       P1000_729_512_N134_I1P.icm

The first line is the profile performance of a standard, linear grid with 9x9x9, evenly spaced RGB values. This was used to create an additional 640 "optimized" patches. Just over half of the additional, optimized patches were near neutrals. The rest, scattered colors.

The second line is the performance of this optimized profile. As expected it's better by .08 dE00 in the color patches and .18 dE00 in the near neutrals.

The third line is from one of my generic patch sets using packed RGBs but with additional near neutrals. Specifically the set labeled with the suffix M9PN in my crowdsourcing thread where it can be downloaded. Since it is a generic set, and not tuned to the printer I expected it to be slightly worse than the optimized profile. But it wasn't. More surprising is that the near neutral performance was materially better with the generic profile which had less than half the additional near neutrals (134) compared to the "optimized" one.

This suggests it might be better to skip the optimizer pass on the I1Profiler and just run a larger patch set. At least if one just uses the add smart patches option.

While this test was with the Pro1000/glossy, I suspect similar results would happen with other printers.

[nirpat89]

Hi, Doug:

Tying to understand how the profile calculations are done.  In the first case, does it take the set of 729 input values and compare against the measured values and do some sort of best-fit regression to cover the whole color space.  Then does it keep the original data at this point or tosses them away?  If later, then when the next set of 640 patches are added, the subsequent analysis is done with predicted values from the first icc profile (with its inherent error) combined with the new measurements.  Is that how it is done?  If it is, perhaps this would be the source of a kind of statistical "hysteresis" over the third profile where the all patches were analyzed in one shot. 

I don't know if it makes sense.  Always enjoy your posts...learning something new every time.


:Niranjan.

[digitaldog]

I always offer and run Optimization because:
1. It either does nothing or...
2. It helps a bit.
And of course, the construction of the optimization target plays a role.

[Doug Gray]

Hi, Doug:

Tying to understand how the profile calculations are done.  In the first case, does it take the set of 729 input values and compare against the measured values and do some sort of best-fit regression to cover the whole color space.

Argyll's colprof and XRite's I1Profiler use slightly different approaches. With the defaults, I1Profiler tends to do a  close fit when the target RGB patches are widely spaced while Colprof does not. I1Profiler profiles tend to be a bit more accurate for small patch counts but a bit less accurate when the patch count gets above about 2000. As to how it generates the "smart patches" I can't say other than I expected them to produce better results than just a larger patch set. A lot of the added patches are near neutrals but the larger patch set I compared the optimized set with produced slightly better color accuracy and materially better near neutral accuracy. Since I1Profiler's optimize pass produces over 300 near neutrals while my generic, large profile only had 134 near neutrals I'm rather shocked that the results weren't better rather than worse.

Then does it keep the original data at this point or tosses them away?  If later, then when the next set of 640 patches are added, the subsequent analysis is done with predicted values from the first icc profile (with its inherent error) combined with the new measurements.  Is that how it is done?  If it is, perhaps this would be the source of a kind of statistical "hysteresis" over the third profile where the all patches were analyzed in one shot.

I1Profiler's optimize pass adds the number of smart patches you enter to the ones from the original set. This test used 729 original patches and in the optimize pass I added 640 additional patches which were about half color and half near neutrals. These are combined and the new profile is created from them.

Basically, what I did was create a profile with the 729 base patch set and a second profile by combining them with the 640 patches. The third profile was made using the generic (not tuned to the printer) patch set which was the 729 set with an additional 512 patches centered in the device rgb cubes as well as 134 near neutral to improve B&W performance. There is a description of this process here. These produce slightly better profiles relative to their patch size but it isn't huge. Just a bit better.

All the patches for the three targets were combined, duplicates removed, and additional, independent patches added to create a set of 3, 957 count target pages. The accuracy results were done using only the independent patches since comparing accuracy using the same patches that profiles are created from is very misleading. Profiles that match the values from which they were created are not only unreliable, but they can actually invert things. A solid way to measure accuracy is to print a different set of patches using absolute intent, measure them and compare the measured results.

My approach here is a bit different and provides a way to measure profile accuracy for both colors and near neutrals for a large number of patch sets with the same, independent set of RGB values. That same measured set can then be used doing a reverse lookup. That is finding the Lab values that produce the exact RGB device values for each profile. This would be the same as if one had those Lab values in an image and printed them using Abs. Intent and measured them then produced a dE 2000 difference.

[Doug Gray]

I always offer and run Optimization because:
1. It either does nothing or...
2. It helps a bit.
And of course, the construction of the optimization target plays a role.

I've never seen the optimization pass do nothing and it certainly helped a lot in my test. It may or may not be visible though for larger patch sets. Especially compared to just using a larger patch set at the go.

I was rather surprised that the improvement was not as good as just using the larger generic patch set that had only 134 near neutrals. I did not expect that.

To be fair though, the differences are sub .10 dE00. To get statistically significant dE00 averages to .01 resolution requires reading 500 or more, independent  color patches.

The averages for color patches are significant to .01, for the total neutrals, .02, and for each individual L* segment of the near neutrals, .05

[Alan Goldhammer]

Doug, have you investigated Argyll's preconditioning approach?  I do a two page random patch set to get an preliminary profile and then use that to generate the larger patch set with the new neutral tool that Graeme put into the software.  That's the patch set I use for the final profile.  I don't have the type of tools you have to test for the quality of the profile and pretty much rely on standard test images.  The dEs from the reading are always quite good.

[Doug Gray]

Doug, have you investigated Argyll's preconditioning approach?  I do a two page random patch set to get an preliminary profile and then use that to generate the larger patch set with the new neutral tool that Graeme put into the software.  That's the patch set I use for the final profile.  I don't have the type of tools you have to test for the quality of the profile and pretty much rely on standard test images.  The dEs from the reading are always quite good.

I have not. This was actually my first effort to do a decent quantitative comparison. I chose I1Profiler since it's commonly used and the optimize pass had been mentioned from time to time with mixed results. I did expect it to be better than a generic profile of the same size.

I do mean to take a look at Graeme's near neutral additions. I think it to be a major, and desirable change. This can make a real difference as most near neutral dE1976 deltas are magnified when looked at with deltaE2000. Exception being L* at the low and highs which are lower for deltaE, but a* and b* can bump up differences quite a bit across the L* board.


Not that deltaE 2000 doesn't have it's own issues but I find it better than deltaE76 or deltaE94. To my eyes at least.

Funny thing is that for testing large numbers of profile configurations, Argyll is great because I can make 100+ different profiles using batch overnight then compare them in a few minutes.

Added:
I'm in the process of testing Argyll's patch set with added known neutrals and optimized to a reference profile. Wouldn't surprise me if it's much better than the I1Profiler's 2 pass system. Especially on the neutrals. My added near neutrals/packed RGB, which beat I1Profiler's optimize pass, were not based tuned from a prior profile. I should know in a few hours.

[Doug Gray]

The Argyll profile using an existing profile to spread the colors, produces results for color patches identical (.47) to the generic, packed profile with 134 extra fine grid neutrals and slightly better than a similarly sized, second pass from I1Profiler.

Patches       AveCol  AveNeu       0-15   15-25   25-35   35-45   45-55   55-65   65-75   75-85  85-100
 1369         0.47     0.54             0.91     0.53     0.49     0.55     0.47     0.48     0.43     0.38     0.49  P1000 ArgOpt_a_I1P.icm

However, the near neutrals are somewhat worse.

Examining the near neutrals added by the Argyll targen using "targen -d2 -n 140 -f 1362  -c Printer-P1000_MALL3.icm" they are along the relative colorimetric axis and do not include a fine grid. The generic profile uses 134 extra near device neutrals (these are not colorimetric). The generic profile gives a bit better results against the random set of 192 near device neutral patches. The additional relative colorimetric are quite similar to those made by my tracking program using the relative colorimetric option. I would expect testing with strictly colorimetric neutrals to exhibit similar improvements but the lack of a fine grid probably accounts for the somewhat higher dE values in the neutrals since they are all near device neutrals and not strictly on colorimetric neutrals. See my posts on a program to create patch sets with tracking neutrals.

Colorimetric neutrals, like the I1Profiler's 2 pass optimization and Argyll's patch generation both require an initial pass to create an profile.

OTOH, the generic profile with packed RGB colors and the fine grid near device neutrals gives excellent results for colors and near neutrals.

[Doug Gray]

I did a comparison of these 3 profiles performance along the actual neutral axis. L* from 2 to 100 using Rel. Col as well as L* from 2 to 93 using Abs. Col.  Rel. Col. L*100 is the same as paper white while Abs. Col. can no longer track past L*=93 as it is limited to the paper's reduced reflectance from a perfect white.

The three profiles are:
The Argyll patch set based on a reference profile and adding tracking neutrals.
The I1Profiler's 2 pass, optimization process with a reference profile.
A patch set not tuned to a reference profile. Generic, packed RGB with 134 added device near neutrals.

As anticipated, the Argyll patch set outperformed the other two with the largest difference being Rel. Col. accuracy which was .32 compared to .42 and .44 for the I1Profiler optimization patch set and the generic (no reference profile) patch set.

It also outperformed using Abs. Col. neutrals but not as much (.36 v .41).

Note that Rel. Col. accuracy carries over into Perceptual Intent and this and Rel. Col are by far the main intents used in printing photos.


Note that all of these actual neutrals are better than the near neutral device RGB color accuracies shown earlier in the thread. This is because those represent highly desaturated colors, typically within about a*/b* within +/- 15 of actual neutrals. Those would apply when printing B&W with light tints for artistic purposes.

[GWGill]

The Argyll profile using an existing profile to spread the colors, produces results for color patches identical (.47) to the generic, packed profile with 134 extra fine grid neutrals and slightly better than a similarly sized, second pass from I1Profiler.

The (rough) Argyll equivalent to a fine near neutral grid would be to crank up the targen -N parameter when using a prior profile. There may be some interaction between -n and -N though, since specified test points tend to "repel" automatically generated ones.

[Doug Gray]

The (rough) Argyll equivalent to a fine near neutral grid would be to crank up the targen -N parameter when using a prior profile. There may be some interaction between -n and -N though, since specified test points tend to "repel" automatically generated ones.

Makes sense. I haven't tried that so far. I did something similar with the "tracking profile" program that created actual neutrals, selectable by Rel. or Abs. Col, but not the full fine grid. Just the stagger approach where R, G, and B were offset sequentially by varying amounts. IIRC, it seemed to improve other near neutrals but at a slight loss of accuracy on the Rel. Col and Abs. Col neutrals. Subsequent testing indicates a more complete fine grid worked better at color patches that were near the neutral axis but I haven't tested that combined with the tracking RGB values. It may, like the staggered RGBs, come with a cost to the neutrals.

BTW, might be nice to have a Abs. Col. tracking set option too. OTOH, Rel. Col. and Abs. Col. tracking are quite good with M2 and most realistic use of Abs. Col. is with OBA free paper. In that case the two are already quite close. So it's probably of little additional value. Use cases are minimal.