Pages: [1]   Go Down

Author Topic: In Depth Profile Evaluation/Comparison (using PatchTool)  (Read 2595 times)

darlingm

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 361
    • Westland Printworks
In Depth Profile Evaluation/Comparison (using PatchTool)
« on: October 14, 2012, 08:14:23 pm »

Background - I've been trying to dive quite deep into color management.  Below are my thoughts on comparing several printer profiles for Epson Enhanced Matte on my Epson Stylus Pro 9900.  (Hey, Enhanced Matte is cheap!)  I reproduce a lot of artwork, but certainly print for photographers as well.  I am extremely picky about artwork reproduction, and every step forward in color management is a gigantic timesaver.

Profiles Compared - I compare Epson's stock profile, ImagePrint 9's Profile Valet's (stock) profile, and custom profiles I made using my i1 Photo Pro 2 through both i1Profiler and basICColor print.  For my custom profiles, I used 5837 patches, then 3536 patches in the 2nd iteration.  Definitely overkill, but again, Enhanced Matte is cheap, and I'm trying to learn as much as I can on it to make other medias go smoother.

Comparison Method - I created a set of 442 RGB colors, under a nice distribution, with slightly higher weighting in shadow, highlight, and grayscale areas.  I then assigned these RGB colors to each profile and printed them.  (That is, each print had a patch for 35R/35G/35B, however that color would of course translate to different LAB colors.)

I then used BabelColor's PatchTool to compare the LAB values of what was intended to be printed on the 442 patch vs the measurements of what was actually printed.

Conclusions

I am severely disappointed to see no statistical advantage over using a custom target by either i1Profiler or basICColor print over the Epson stock profile.  (Granted, this is using my randomly distributed 442 patch set, this is limited to Epson Enhanced Matte only and may not be indicative of trends on other medias.)  Granted basICColor print's max error is lower than Epson's stock profile, but not by much, and both custom profiles have a much higher average, best 90%, and worst 10% deltaE's.

ImagePrint's profile is interesting.  There are more accurate colors, but it's variance is ridiculous in my opinion.  The maximum error jumps from around 11 for the other profiles to 16.4.  So, ImagePrint's profile's advantage will depend on the image.  For most colors, it will be much more accurate.  But, when it's off, it's really off.  As a side note, most of the colors it is really off with are in the red-orange area of the spectrum, which I've noticed is quite off on other medias when printing gamut tests, like the Granger Chart.)

Questions

* I know profiles help with color translation but won't be completely accurate - however I'm surprised to see such a high average DeltaE.  I never would have imagined it to be in the neighborhood of 4.  Is this common?

* It it very unusual that the i1Photo Pro 2 & i1Profiler seem to have created an inferior printer target vs Epson's stock profile?  (Granted, this is on the 442 random patch set I generated, so the true averge DeltaE of all colors could show a different story.)

* If I want to print a spot color (that I know the LAB value of, and it's in gamut), this means there's no way to get it substantially closer to correct other than cycling through hard proofs, no matter what hardware and software is obtained?


Data from BabelColor's PatchTool
(All numbers are DeltaE - CIEDE2000)

ProfileAvg
(All)
Best
90%
Worst
10%
Sigma
(All)
Sigma
(Best 90%)
Sigma
(Worst 10%)
Max Error
(10th Pct)
Max Error
(Median)
Max Error
(90th Pct)
Max Error
(95th Pct)
Max Error
(All)
Epson 9900 Stock Profile3.683.247.71.751.031.751.963.395.66.9311.56
i1Profiler (5837 patches, then 3536 2nd iteration)4.033.558.391.80.981.662.423.656.287.7811.33
basICColor print (from above i1Profiles measurements)4.393.898.931.891.121.262.63.946.918.9810.83
ImagePrint ("epx9mk_epenhmatte_pph_rday.icm")2.141.428.664.213.315.660.563.147.349.3016.43

Logged
Mike • Westland Printworks
Fine Art Printing • Amazing Artwork Reproduction • Photography
http://www.westlandprintworks.com • (734) 255-9761

darlingm

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 361
    • Westland Printworks
Re: In Depth Profile Evaluation/Comparison (using PatchTool)
« Reply #1 on: October 14, 2012, 08:40:34 pm »

Figured I may as well post this too...

ColorThink has a nice video explaining one way to compare profiles against each other at: http://www.youtube.com/watch?v=sTxaHhMYSYE

They created a list of 4448 RGB values, assigned them to ProPhoto, and transferred that into each profile, and graphed it.

If it's warped, segments are missing, or points bunch up, that indicates issues.  It's better for everything to be uniform and smooth.

I'm shocked with how different these look, that the first three profiles are roughly the same quality under the statistical calculations.
Logged
Mike • Westland Printworks
Fine Art Printing • Amazing Artwork Reproduction • Photography
http://www.westlandprintworks.com • (734) 255-9761

Scott Martin

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1315
    • Onsight
Re: In Depth Profile Evaluation/Comparison (using PatchTool)
« Reply #2 on: October 15, 2012, 12:03:57 pm »

There's lots of variables to discuss when it comes to making your own custom profile. Did you compare the results using various measurement modes (M0, M1 and M2) for example? Have you done some tests to discover the "sweet spot" target (IE: a target with not too many and not too few target patches and one that uses lots of gray patches).

Also, I'd really encourage you to include some visual evaluation elements to your process. I think its important to include these in conjunction with numerical analysis. Sometimes the differences are surprising. And the numerical results alone can often lead to false conclusions.

And most importantly, the real magic (and differences) will be found using the Perceptual intent!
Logged
Scott Martin
www.on-sight.com

digitaldog

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 20614
  • Andrew Rodney
    • http://www.digitaldog.net/
Re: In Depth Profile Evaluation/Comparison (using PatchTool)
« Reply #3 on: October 15, 2012, 12:52:19 pm »

I agree with Scott that it can be a really deep rabbit hole to dig through using some of the color analysis of color patches without first making some actual prints. The proof is in the print and since ICC profiles don't know squat about images and treat everything as a big pile of solid colors (not color in context), best to send a lot of images through the profile. I love using the Roman 16 test images to evaluate visually the profile but the are not inexpensive!

One can build a group of colors (400 or more) and look at the dE reports and depending on how you select those colors, you can get a high(er) or lower dE report. Does that tell you about how the profile will preform with an image? It can tell you something is way off (a soft proof might do that too). It is useful analysis but I'd be testing output of images first, color dE values 2nd.

When I tested using a custom target generated out of i1P versus the same number of patches from ProfileMaker Pro, I saw a slight visual advantage using the i1P target.
Logged
http://www.digitaldog.net/
Author "Color Management for Photographers".

darlingm

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 361
    • Westland Printworks
Re: In Depth Profile Evaluation/Comparison (using PatchTool)
« Reply #4 on: October 18, 2012, 03:35:48 am »

I agree there's a lot more to it than numerical analysis, but I'm trying to add that to my visual assessments.  I started comparing with M0, and got hung on it being frustrated with such high deltaE values for colors that were supposed to be in gamut.  However, I think I've found an error in my testing, which I created a new post on since that's really a separate issue, and I'm hoping that's causing a false reading of these high deltaE values.
Logged
Mike • Westland Printworks
Fine Art Printing • Amazing Artwork Reproduction • Photography
http://www.westlandprintworks.com • (734) 255-9761
Pages: [1]   Go Up