Z3200 - Better ICC Color Profiles

[Mark Lindquist]

Shoot me a PM with an email address. I have a CGATs file, formatted for ProfileMaker Pro but can format differently.

Thanks Andrew, PM sent.

Mark

[Brad P]

High resolution: http://digitaldog.net/files/Not_All_Profiles_are_created_equally.mp4
Low resolution (YouTube): https://www.youtube.com/watch?v=TNdR_tIFMME&feature=youtu.be[/i]

One more thank you Andrew - I hadn't ever tested my profiles with Bill's Balls, linked in your video, and I used that to test it later today.  That is a super test that revealed a problem that other tests haven't.  Backstory - when printing out my 1728 profile a few months ago (which is a lot of ink and paper on the HPZ3200), I noticed a couple white spots on two patches where some flecks of something prevented the green ink from hitting the paper.  Slyly I thought, I filled them in with a green pen trying to match.  Till now, I thought I had gotten away with it.  But nope, big time thick banding in the green balls.  Anyway, I'll redo the patch and won't be doing that again.

Just a real world note of thanks and hopefully a bit of levity.

[Mark Lindquist]

Mark, i just made some profiles for a paper with the 'HP advanced profiling solution' (APS) that i bought with the z3100
I have a 1728 reference so it can do that too.

i made a 220, a 918 and a 1728 profile with the APS and a 440 patch profile that came with the printers software.

Result ... and for me a trending result:
1728 and 918 has some banding in places with gradients in areas with strong colors were the 440 and 220 have not.
the 440 profile was the better one in this case. ( the 220 profile was not accurate enough )
On the other hand the 918 and 1728 had more extreme colour.

so my basic idea is :  more patches- more accurate colour, but also a greater chance for banding issues

What are your experiences?
I know this is a difficult subject ( can of worms) for a straightforward answer.

Kers,

Another thought about this.  Consider that the APS you are using is now about a 15-20 year old technology, meaning that the HP Puck, the profile target,etc, is made from a Z3100 machine technology and Gretag MacBeth era technology.  Fast forward and today’s Spectrophotometer is i1, with presumably many innovations and considerable evolution involved.  It was well known information that the Z3200ps with its standard small patch target outperformed the outdated APS profiling system, and that the APS would have required a significant upgrade to work with the new spectrophotometers and all that entailed, so HP dropped it.  The ability to make 1728 patch targets and generate standard cgats files is in part a hold-over from that technology previously called APS, yet as we know, we are required to use third party software such as Argyll or RGB Drop, (et al), etc., in order to generate the actual ICC profiles.  As it is, however, the machine is entirely adept at producing standard cgats in the form of .csv files that can be changed to standard .TXT files as well.  So in essence, the APS has been rendered obsolete, which may explain your issues with banding, yet I suspect there are other issues related to printheads/settings/and controls that are potentially contributing to that effect.  This is my opinion, of course, your mileage may vary.

Best,

Mark

[MHMG]

That is very cool Mark,

Thanks for sharing, as always.

Do you know if I can drop this file into my X-Rite i1 Pro 2 software to create the icc profile?

John

Hi John, I recently went through that whole exercise with Mark L.'s help and the results were very interesting. First some background information for new owners of Z3200PS printers. HP marketing literature still claims one is getting the APS "advanced profiling" solution with the PS version of the Z3200, but in fact, the display calibrator hardware became obsolete a number of years ago, so HP just quit including the APS hardware/software kit with the machine. However, some of the target generation capability still lies within the current color utilities, and it allows the user to print the 1728 target, measure it, and save a CGATs configured .CSV file. I am able to successfully read that file with BasicColor RGB profiling software, and Argyll CMS (using Mark's iCCGen GUI app to give it a nice and simple GUI for profile generation), but Xrite's i1Profiler software rejected it. I suspect that with a little more exploration of the header file information there may be a simple "cut and paste" workaround with Microsoft Excel or a text editor to trick i1Profiler into reading the HP generated CGATs file, but I haven't had time to do that experiment. Anyway, I have attached a png screenshot of some curves I quickly plotted in Excel to show the tone reproduction response of the ICC profiles I built using the Z's 1728 auto printed and measured output. The graphs could use a little more cleaned up text in the titles. One graph is the tone plot from the Argyll/ICCGen produced profile (title in the graph begins with "iGen". It was made with Relcol/BPC rendering.  The second graph is from the profile built with BasicColor RGB drop software (reads "BcC" in the title) using Relcol/BPC as well. Third graph has a very subtle tonal difference because it was made with the BcC profiling engine using perceptual rendering. The midtones are ever so slightly higher in L value to coax a little better shadow detail linearity out of the tone curve with the perceptual rendering intent. The roll off in the curve as Lmin (Dmax) is approached is typical tone reproduction output on fine art matte papers (this one was Moab Entrada Natural).

Both the Argyll and BasicColor profiles built from the Z3200's measured 1728 patch output are quite good, IMHO. Since HP's 1728 patch layout is the standard 12x12x12 uniform grid arrangement, I don't see anything to gain from Bill Atkinson's 1728 patch target other than if you want to use less paper and print smaller patch sizes, but then you will need another spectrophotometer to measure the final printed target as well.

In the meantime, it's fair to conclude that the Z3200 printed and measured the 1728 patches in perfectly satisfactory manner. That leaves new Z3200 printer owners just having to find some profiling app to crunch the numbers since HP's APS software that used to handle the task is no longer being delivered with the new machines. Argyll using Mark L's iCCGen helper applet is an essentially straightforward and "free" approach to crunch the data (but please consider a donation to the Graeme Gill's Argyll website http://www.argyllcms.com/index.html). BasicColor RGB drop is as sweet as as it gets with respect to ease and quality of profile generation (but you pay for that ease of use)!

cheers,
Mark
http://www.aardenburg-imaging.com

[Doug Gray]

Mark,

I'm willing to take a crack at converting the spectral file to a CGATs that I1Profiler recognizes. I regularly munge data and write out CGATS files I1Profiler reads. If successful, which is highly likely, I will then post instructions or possibly open a github repo where people can download code.

[Mark Lindquist]

Mark,

I'm willing to take a crack at converting the spectral file to a CGATs that I1Profiler recognizes. I regularly munge data and write out CGATS files I1Profiler reads. If successful, which is highly likely, I will then post instructions or possibly open a github repo where people can download code.

Doug,

If you like, I can publish your code on z3200.com if you like (unless you’re planning to sell it).  If you’ll be giving it away for free, please consider it, because it would be a good place to keep this info in the same place.  I have to tune up the installation instructions for ICC Gen as it has a bug in the instructions.  As for the x-rite/ z series issue, MHMG is thinking that it’s a simple cut and paste of the x-rite header to replace the Z3200 header, so hopefully it should be easy.  Please consider doing a simple GUI for folks who don’t use terminal.  That’s all ICC Gen is, a shell that uses terminal to utilize Argyll, yet enables users to bypass using terminal themselves.

I’d be happy to publish any work around you come-up with and make a graphic for it and give full credit to you and anyone else involved in making it if you’re up for it.

I’m interested in providing free work arounds for the .csv and .TXT files generated through the Z Series utility so everyone so inclined can have a total solution for using the Z Series automated ICC profile capabilities. 

Mark McCormick is super busy with Aardenburg, but I’m confident he will collaborate, as he and I have already started to investigate this.  Any involvement you could contribute would be more than welcome.

Thanks-

Mark L.

[MHMG]

Mark,

I'm willing to take a crack at converting the spectral file to a CGATs that I1Profiler recognizes. I regularly munge data and write out CGATS files I1Profiler reads. If successful, which is highly likely, I will then post instructions or possibly open a github repo where people can download code.

Hi Doug, see attached file. Have at it As I noted earlier the BasicColor and Argyll profiling apps had no trouble reading this HPZ3200 spectrophotometer-generated CGATS file and building a proper ICC profile using the data. However, when I tried to load the file into i1Profiler, it says it's an "invalid" CGATS file. Probably something simple I'm overlooking about how to load a measurement file into i1Profiler or perhaps a case of needing to tweak some of header row formatting to get i1Profiler to accept it. Good luck.


Mark
http://www.aardenburg-imaging.com

[digitaldog]

Hi Doug, see attached file. Have at it As I noted earlier the BasicColor and Argyll profiling apps had no trouble reading this HPZ3200 spectrophotometer-generated CGATS file and building a proper ICC profile using the data. However, when I tried to load the file into i1Profiler, it says it's an "invalid" CGATS file. Probably something simple I'm overlooking about how to load a measurement file into i1Profiler or perhaps a case of needing to tweak some of header row formatting to get i1Profiler to accept it. Good luck.


Mark
http://www.aardenburg-imaging.com

Will it load in ProfileMaker Pro? Re-save, i1P should (should but may not) open it.

[Doug Gray]

Will it load in ProfileMaker Pro? Re-save, i1P should (should but may not) open it.

Good solution Andrew!

And, yes, you can make profiles with PM5 if you have a license. But if you don't and want to use I1Profiler do this:

1. Load the file in PM5's Measure Tool, then save as a txt file.
2. Load the new, CGATS file in I1Profiler in the "Measurements" section skipping "Patch Set" and "Test Chart." It will then ask for data type. The latest X-rite is XRGA but it's probably better to select one of the older standards like the GMB. I1Profiler will next request the uV mode (M0, M1, or M2). Select M2. Spectral data is stripped so this can only be used for standard D50 profiles.


A few notes:

The file is a uV cut file using a white LED source (no uV from those). Spectral data from 380nm through 400nm is zero as is everything above 710nm

You don't need a license to load and save the file with PM5's Measurement Tool. While it will state it is in "demo" mode, the necessary functions are enabled to load the HP file and save an I1Profiler compatible CGATS file.

[digitaldog]

I was thinking of using PMP (actually MeasureTool*) just to attempt a CGATs conversion that i1P might like.
* no dongle needed with MeasureTool to do this as you point out.
And when, WHEN will X-rite give us all that functionality we don't have any more from MeasureTool or ColorPort in their high end expensive package?

[MHMG]

I was thinking of using PMP (actually MeasureTool*) just to attempt a CGATs conversion that i1P might like.
* no dongle needed with MeasureTool to do this as you point out.
And when, WHEN will X-rite give us all that functionality we don't have any more from MeasureTool or ColorPort in their high end expensive package?

Just followed Andrew's suggestion to try PM5's measure tool app to resave the HP.csv file as a text file, and then Doug's tips on where/how to insert into the i1Profiler workflow. I had earlier just tried to trick i1Profiler by replacing the .csv extension with a .txt extension but had not been successful with that simple method. However, the Measuretool-resaved text file did come into i1Profiler, and I successfully build a profile. Gamut is identical to the BasicColor RGB result. Looking at a softproof of a corlorful test image, the perceptual tag is a little more aggressive on tone curve than the one built by BcC RGB Drop, but relcol with BPC looks like a dead ringer for both profiling Apps.

One problem with this approach is not only do you have to download Measuretool, but one has to have a Mac machine running OS10.6 or older (I do indeed have all this old legacy hardware/software), because Measuretool needs Rosetta code to run on a Mac. Nonetheless, this workflow did succeed.  I still think that once these different text files are inspected more carefully in Excel, one will be able to make the changes directly in Excel and not need to resort to Measuretool, but more eperimenting needed!

Although this effort may seem somewhat arcane to folks who don't own a Z3200, it does show that HP's built in Spectro along with its printer driver utility can indeed generate reasonably CGATs compatible files that can further be used in conjunction with numerous other ICC profiling packages. With a 44 inch roll loaded into my Z3200, the Z's printer utility automatically laid out and printed the 1728 patch target set using just 14 inches of paper from the leading edge of the roll. Letting the Z then measure unattended was a real treat compared to printing an i1Pro2 compatible target with a "no color adjust" utility and then hand measuring.  For those lucky folks with more automated spectros like i1iSIS, this HP spectro workflow might not appeal, but for those of us having to resort to more manually generated ICC profiles, the HP Z3200's Spectro adds a welcome relief to the fatigue of measuring high patch counts by hand

All the best,
Mark
http://www.aardenburg-imaging.com

[digitaldog]

Just followed Andrew's suggestion to try PM5's measure tool app to resave the HP.csv file as a text file, and then Doug's tips on where/how to insert into the i1Profiler workflow. I had earlier just tried to trick i1Profiler by replacing the .csv extension with a .txt extension but had not been successful with that simple method. However, the Measuretool-resaved text file did come into i1Profiler, and I successfully build a profile. Gamut is identical to the BasicColor RGB result.

FWIW, while I'm happy that worked, my suggestion was to help the OP get a patch file I sent (CGATs) into whatever he's using to make a new 1728 target.

[Mark Lindquist]

FWIW, while I'm happy that worked, my suggestion was to help the OP get a patch file I sent (CGATs) into whatever he's using to make a new 1728 target.

Andrew, Thanks for your comments and for sending the CGATS file.  Frankly I don't know how to import or convert the data you sent me into a file that the Z3200 can read.  I'm not the OP, but am interested in importing other targets that the Z3200ps can read that are greater than 1728 - specifically, a 2371 Patch Target.  The Z accomplishes this by printing a target for later use, then it reads it and creates a .CSV file (Mac) or a .TXT file (Windows).  I'm not sure how to import a target patch larger than 1728 that is in the same format as what the i1 profiler in the Z3200 spectrophotometer uses in its final form as illustrated below:



The Z3200 is finicky about reading these targets - any idea how to convert large patch targets into this specific target formation?

BTW - you were partially correct, that the Z is somewhat crippled because it doesn't have a program anymore to be able to create an ICC profile from a .csv or .txt file.  However, as demonstrated and with one of your solutions, it's entirely possible to use third party software.  As I stated earlier, this use of the ESP and free third party software is a long ways away from a $3K plus machine that is automated.  As it applies to specifically Z3200 owners, it's of vital importance for anyone trying to get the most out of the spectrophotometer keeping relatively sophisticated profiles in-house.  The CGATS files generated by the Z3200 are equivalent to what is coming out of other systems.  Agree or disagree, it's what we have to use.  And thanks again for your work around.  As MHMG just showed, it works.  Getting a simple header change to make it work directly would be significant for us.

Best,

Mark

[digitaldog]

Mark, I was hoping a similar technique the other Mark tried would work for you.
You've got an exported PMP CGATs 'patch file' for all of Bill's 1728 colors. I don't know anything about the software you use with the Z. The idea is, if (big if) it supports custom targets, it should import that text file and allow you to now define the Spectrophotometer, size of patches and number of pages etc. It would create a target you could print, measure and that original CGATs would be used as the 'reference' for creation of the profile. But again, I have no idea how this works with your product.

[Mark Lindquist]

Thanks Andrew - I'm looking into it.  Appreciate your help and input.  Will see if I can get it working.  Others here may jump in as well.

Best -

Mark

[deanwork]

Thanks Mark and Andrew,

It's interesting that I can generate an icc fron the hp expanded patch file on the Z3200 and install it .

For the time being I'll just use my X-Rite 2033 patch target like I do with my Canon and my Z3100 - that's a whole lot easier and faster, since I'm familiar with that, but it is good to have a backup.

John

Hi Doug, see attached file. Have at it As I noted earlier the BasicColor and Argyll profiling apps had no trouble reading this HPZ3200 spectrophotometer-generated CGATS file and building a proper ICC profile using the data. However, when I tried to load the file into i1Profiler, it says it's an "invalid" CGATS file. Probably something simple I'm overlooking about how to load a measurement file into i1Profiler or perhaps a case of needing to tweak some of header row formatting to get i1Profiler to accept it. Good luck.


Mark
http://www.aardenburg-imaging.com

[Mark Lindquist]

Mark, I was hoping a similar technique the other Mark tried would work for you.
You've got an exported PMP CGATs 'patch file' for all of Bill's 1728 colors. I don't know anything about the software you use with the Z. The idea is, if (big if) it supports custom targets, it should import that text file and allow you to now define the Spectrophotometer, size of patches and number of pages etc. It would create a target you could print, measure and that original CGATs would be used as the 'reference' for creation of the profile. But again, I have no idea how this works with your product.

Hot Dang.  I got it.  It worked.  Very cool.  Thanks Andrew.  Cha-ching.

Mark

[Doug Gray]

Another way to fix it without running Measure Tool:

Just change the .csv file to .txt (it isn't really a csv file).
Then open the file with a text editor and change this line in the header:

NUMBER_OF_FIELDS   49

to

NUMBER_OF_FIELDS   44

This also preserves the spectral data which means you have the ability to create custom profiles that can be made so that a print will look great under even the worst lighting such as some office fluorescents.

The problem is that the HP saves fields from 380nm to 780nm. I1Profiler expects only from 380nm to 730nm. Making this change causes the extra data, which is zero anyway, to be ignored.

Then load it in to I1Profiler as a CGATS file, select M2, and GMB as before.

[MHMG]

Another way to fix it without running Measure Tool:

Just change the .csv file to .txt (it isn't really a csv file).
Then open the file with a text editor and change this line in the header:

NUMBER_OF_FIELDS   49

to

NUMBER_OF_FIELDS   44

This also preserves the spectral data which means you have the ability to create custom profiles that can be made so that a print will look great under even the worst lighting such as some office fluorescents.

The problem is that the HP saves fields from 380nm to 780nm. I1Profiler expects only from 380nm to 730nm. Making this change causes the extra data, which is zero anyway, to be ignored.

Then load it in to I1Profiler as a CGATS file, select M2, and GMB as before.

Thanks Doug, for that!  A quick fix with Text editor on my Mac, and i1Profiler loaded the HPZ3200-measured CGATS file with no problem. I didn't even need to drag the file into Excel!

kind regards,
Mark
http://www.aardenburg-imaging.com

[Mark Lindquist]

Thanks Doug, for that!  A quick fix with Text editor on my Mac, and i1Profiler loaded the HPZ3200-measured CGATS file with no problem. I didn't even need to drag the file into Excel!

kind regards,
Mark
http://www.aardenburg-imaging.com

Thanks for your changes Doug - very interesting.  Excellent.

Mark