Prints Suddenly Anemic

[GWGill]

Graeme, I extracted the CGATs (See attached) and also noticed the white points were quite different.

Thanks.

Ignoring the white point for the moment, running profcheck against the data files and the profiles I get good agreement between the old data and old profile, and poor agreement between the new data and new profile (i.e. data vs. absolute A2B). The most usual reason for this with handheld instruments is user error - i.e. reading the wrong strip. I'm not sure what the explanation would be when using an isis. Is it possible to get randomized chart layouts mixed up when using an isis ? Is it possible for the isis to mis-read patches ?

Running profcheck -w and looking at the visualization shows systematic differences between patch data and the profile though. Most of the discrepancies seem to be at the surface of the gamut, and hint at clipping behavior - i.e. that the RGB has been clipped, and so a lot of a values near the gamut surface are being mapped to the gamut surface values. Naturally this would distort the smoothed characterization.

(Interestingly an Argyll made profile has smaller but non-normal level errors when made from the new data set.)

There are huge discrepancies when the new profile is checked against the old data set - i.e.
max. = 46.732429, avg. = 11.529630, RMS = 14.164539. (There is similarly a huge difference with an Argyll made profile.)

So to me it looks like something is seriously wrong with the new measurement data, if the old data is taken to be nearer the truth.

It's interesting that the new data file doesn't match the old one in the number of patches, and in fact appears to be a different chart.

[Doug Gray]

Thanks.

Ignoring the white point for the moment, running profcheck against the data files and the profiles I get good agreement between the old data and old profile, and poor agreement between the new data and new profile (i.e. data vs. absolute A2B). The most usual reason for this with handheld instruments is user error - i.e. reading the wrong strip. I'm not sure what the explanation would be when using an isis. Is it possible to get randomized chart layouts mixed up when using an isis ? Is it possible for the isis to mis-read patches ?

It's absolutely possible!

I've found several sources of error:

1. Registration errors that occur due to back hitching to measure with the uV LED in addition to the white LED that's used for M2 measurements. The uV pass then provides data used to calculate M1/0 spectra. I saw a 10 dE error between reading M2 only and M0/1/2 which requires a back hitch on a large A3+ target. This turned out to be because the paper feed was hanging slightly off my table near one end. When that happened friction increases and is asymmetrical when the stepper motor advances forward v reverse. This went away when I rearranged things so that the paper was always flat on the table never bending over the table edge.


2. Errors due to small amounts of dust inside the iSis sensor assy. This seems to occur with matte paper more than glossy. The worst thing about this is that it can occur suddenly and abruptly and there is usually no obvious warning signs.  Symptoms were readings that were way off and, oddly, sensitive to the scan direction. Typically every other row is scanned in opposing directions. These sometimes caused dE errors in the 10-20 range over multiple patches. Worse. They repeated when reading the same chart again so sometimes don't show up using a duplicate scan verification check. I also discovered an easy way to check this. Adding a registration bar on the bottom in addition to the one on the top allows a chart to be read normally and upside down. Then I just test the two scan CGATs for consistency. This is all automated now for my work. Every now and then it will detect the problem which goes away once I peel the cover back and blow forced air under the sensor assy.

I've also explored registration errors of letter size targets using an iSis all black target. By adding thin, white strips around each of the black squares even small amounts of contamination from light reflecting off the white strips becomes easily measureable. This has made me quite comfortable using 6x6mm patches on a 957 patch 8.5x1l US letter page.

https://forum.luminous-landscape.com/index.php?topic=119448.msg1003106#msg1003106
https://forum.luminous-landscape.com/index.php?topic=121256.msg1008190#msg1008190

Running profcheck -w and looking at the visualization shows systematic differences between patch data and the profile though. Most of the discrepancies seem to be at the surface of the gamut, and hint at clipping behavior - i.e. that the RGB has been clipped, and so a lot of a values near the gamut surface are being mapped to the gamut surface values. Naturally this would distort the smoothed characterization.

(Interestingly an Argyll made profile has smaller but non-normal level errors when made from the old data set.)

There are huge discrepancies when the new profile is checked against the old data set - i.e.
max. = 46.732429, avg. = 11.529630, RMS = 14.164539. (There is similarly a huge difference with an Argyll made profile.)

So to me it looks like something is seriously wrong with the new measurement data, if the old data is taken to be nearer the truth.

It's interesting that the new data file doesn't match the old one in the number of patches, and in fact appears to be a different chart.

Great observation Graeme!  Since the chart layouts are also in the created profile I'm going to create the associated tiff files and do some experiments. Stay tuned

[Doug Gray]

I reconstructed the charts since the profiles contain the chart parameters. They are not randomized but follow from entering the patch count w/o randomizing. Charts are all 8.5" wide and of differing paper lengths.

Looking at the data from the two profiles the differences cannot be explained by any of the issues that I posted in my previous post. The differences are extreme and not associated with any given page, nor top/bottom, left/right registration issues.

I also agree with what Graeme noted that it appears colors in one are clipped well before the gamut boundary. This suggests a possible avenue to explore which I am now doing.


Suggestion for Mick. Until this is ironed out, Simplify, simplify.

1. Create profiles using the same iSiS, the same printer driver setting (ie:dpi differs), and the same paper. Use 8 bit everywhere and follow the same process to print charts and make profiles.
2. Don't bother letting them dry more than an hour as it makes almost no difference after 30 minutes with pigment printers.
3. Use the single patch set, the 957 patch count, Default iSis that fits on a single 8.5x11 sheet. Higher counts can improve profile dEs but much less than 1 dE average. It's a waste of time, paper, and ink until this is resolved.

[Mick Sang]

I'm not sure whether my thoughts are logical, but if the older driver/OS/ColorSync/... became "corrupted" by some OS update, the now newer printer driver could work - but with the old! (or a newly made) profile. The "new" one was somehow corrupted by this quirk, as Doug Gray showed us (if I got him right).

At first it seemed like this post from Michael had hit the nail on the head. Yesterday we made a new profile using 8 bit charts printed via the new driver (10.35) on a MAC running OS 10.14.6 Mojave. We let the charts dry for 24 hours. I know you don’t need to let matte papers dry for that long. But we wanted to eliminate as many variables as possible. A print through the new profile was good. All reds were spot on. But, how could it be that the driver alone could make this difference? It didn’t.

There are huge discrepancies when the new profile is checked against the old data set - i.e.
max. = 46.732429, avg. = 11.529630, RMS = 14.164539. (There is similarly a huge difference with an Argyll made profile.)

So to me it looks like something is seriously wrong with the new measurement data, if the old data is taken to be nearer the truth.

It's interesting that the new data file doesn't match the old one in the number of patches, and in fact appears to be a different chart.

This analysis and new information makes much more sense to me. While I don't yet know how to do such in-depth analysis, I had also seen malformations in the form of the newer profiles as graphed in ColorThink Pro using points joined with lines which visually exaggerates such deformities. But, I wasn’t sure where those anomalies had come from. On the other hand, the form of the newest profile which was made late yesterday looks very smooth and uniform. I did no further tests at that point as it was more important to get a print through the new profile.

Based upon this new information from Mr. Gill and Mr. Gray, I wondered what could be so different between the creation of the old Jan 2017 profile and the recent February 2020 profile both of which were analyzed by Mr. Gray and Mr. Gill versus the new profile made yesterday evening which yielded a good print. Aside from the fact that the charts for the 2017 profile were read by an Isis-1 and the charts for the Feb 2020 and the newest profile were read by an Isis-2, the most obvious difference is that the charts for the old 2017 profile  and the newest one from last night were not scrambled. The charts for the Feb 2020 profile were scrambled. Either the scrambled charts were made incorrectly by i1Profiler or they must have shifted between the saved workflow and the saved Tiffs. If the latter is the case, this was not at all obvious.

Another difference between the old 2017 profile and all the newer profiles in question including the one made last night is that the 2017 profile was generated using i1Profiler v. 1.6.6.19866 and the new profiles were all made using version 3.2.1.

So, if this is all correct now, then you gentlemen have helped me to solve this issue for which we are extremely grateful. Now we will have a great deal of additional work on our hands to remake many profiles which were made recently using those same scrambled charts for our new 9570. That said, I still wonder why the prints were ok (not perfect - but not anemic) using the same defective profiles with different, newer printers. And of course I'd like to understand why scrambled charts from our i1Profiler caused this failure and how to overcome it.

Mr. Gray, Mr. Gill, Mr. Simpson, Mr. Rodney, Michael (fineartelier) and anyone else who tried to help us with this, I thank you very, very much for your time, your suggestions, your experience and your expertise. This foray has yielded several excellent lessons. I do not believe that we would have been able to figure this thing out anytime soon without your help. I know that I wouldn’t have been. We have learned a lot in the process as well. My goal now is to eventually learn how to make such analyses as those done by Mr. Gill and Mr. Gray.

Mick

[Doug Gray]

Mick,

Since you are using the I1Profiler Patch generator you should be aware of a problem. It generates 16 bit (RGB fractional values from 0:255). You can see this when you click on a patch after generating a set. These have been shown to produce strange results sometimes with I1Profiler. Usually very subtle but sometimes really large, unexplained results. In particular they don't correspond to the tif files printed nor do they correspond to the values saved in the profile. These are 8 bit. One of them rounds, the other truncates. Also, small fractiona value changes can produce big differences in the profile. It's almost as if I1Profiler uses different math processes for fractional RGB values. Ethan and I discussed this some years ago.

Easy way to avoid this is, when you generate a patch set the very first thing to do is save it. Saving it eliminates the fractional RGB parts. Then reload the saved patch chart immediately and continue. In the future always load that patch chart when you prepare to make or print targets.

See this for a more complete discussion:

https://forum.luminous-landscape.com/index.php?topic=128070.msg1083715#msg1083715

[Mick Sang]

Since you are using the I1Profiler Patch generator you should be aware of a problem. It generates 16 bit (RGB fractional values from 0:255). You can see this when you click on a patch after generating a set. These have been shown to produce strange results sometimes with I1Profiler. Usually very subtle but sometimes really large, unexplained results. In particular they don't correspond to the tif files printed nor do they correspond to the values saved in the profile. These are 8 bit. One of them rounds, the other truncates. Also, small fractiona value changes can produce big differences in the profile. It's almost as if I1Profiler uses different math processes for fractional RGB values. Ethan and I discussed this some years ago.

Easy way to avoid this is, when you generate a patch set the very first thing to do is save it. Saving it eliminates the fractional RGB parts. Then, always load that patch chart when you prepare to make or print targets.

See this for a more complete discussion:

https://forum.luminous-landscape.com/index.php?topic=128070.msg1083715#msg1083715

SON OF A........ !!  "Sometimes really large," no kidding. Well, what next?  So, all this time we've been carrying on like morons completely unaware of this. I don't know if the math you're referring to caused our problem or it has come to light as a result of the fact that we scrambled the charts. Prior to our decision to start doing that everything seemed to be working well.  I'm reminded: "Don't fix it if it ain't broke." All we want to do is to make good, accurate, reliable ICC profiles for all of our printers.

So, before having the knowledge which you have thankfully provided here, we nevertheless did save the charts immediately after saving the workflow which was done immediately after creating them. So, hopefully that has provided accurate charts and reference files. That seems to be the case at least for our unscrambled chart workflows.  If not, what else can we do?

Thank you, again, for your valuable help and advice Mr. Gray. I'll check out the link you've kindly provided.

Mick.

[Doug Gray]

Mick,

Both profiles you provided above were made with unscrambled patches. I don't think scrambling or not makes any difference in the problem but it might. I haven't investigate it. Most of my work is with scrambled patches. I sometimes duplicate all the patches and then scramble them. This provides some statistical data on how optimal things like vacuum level and head spacing is.

[Mick Sang]

Both profiles you provided above were made with unscrambled patches. I don't think scrambling or not makes any difference in the problem but it might. I haven't investigate it. Most of my work is with scrambled patches. I sometimes duplicate all the patches and then scramble them. This provides some statistical data on how optimal things like vacuum level and head spacing is.

Well, if that's the case then I apologise and once again I'm dazed and confused. There were actually 2 or 3 different profiles made for that printer and paper combo in the period of January - April 2020. We would see the problem then put it aside and use another printer. But we have been scrambling patches lately. So, I thought that that April profile had been made from scrambled charts. Since it wasn't, what could have caused this? None of those profiles through that period work. Only the old one from Jan. 2017 and the newest one from last night work.

There was also definitely something going on with that older driver (9.65) in combination with OS10.14.6 (Mohave). One of the tests which we made while trying to resolve the issue involved our printing the image in question using various other profiles fo other papers which had worked well in the past on that machine. But, we got the same anemic result. The driver at the time was 9.65. Now, however, the result is no longer anemic using those profiles either. The only thing that is changed is that driver.

In case you are interested, here is a link to the newest profile which was made with unscrambled patches.: https://www.dropbox.com/sh/156dgw79mwtaowe/AACOnY0u4l-TuPyVc-o-cBBpa?dl=0

It was also made with large patch charts. We are listening to you and will try to follow your guidelines as well as others that have come along during this. But, I didn't want to introduce anything new until this is settled.

Thank you, sir.

Mick

[Doug Gray]

Using a set of 10,000 random RGB values, these two profiles had an average dE 1976 of 2.3

'CSI 4900 1440 RGB Epson UltraPremPres Matte.icc'
'CSI 4900 2880 RGB Epson UltraPremPresMATTE_M2.icc'

This is about right and fairly typical of the differences one gets from a DPI change like 1440<->2880. Both of these profiles behave reasonably.

---
This is the profile that Graeme commented on with clipped colors near the gamut boundary:
CSI 4900 2880 RGB Espon UPP Matte_M2.icc

All 3 RGB channels in the extracted data exhibit the clipping effect. I've never seen anything like it.

[Mick Sang]

Thank you again, sir.

This is the profile that Graeme commented on with clipped colors near the gamut boundary:
CSI 4900 2880 RGB Espon UPP Matte_M2.icc

All 3 RGB channels in the extracted data exhibit the clipping effect. I've never seen anything like it.

Nor have I.  It is also the profile which I described in my previous post as having deformities as viewed in the grapher of ColorThink Pro. But, I'm only referring to the effect in print.

Using a set of 10,000 random RGB values, these two profiles had an average dE 1976 of 2.3

Can you or would you direct me as to where I can find instruction on how to do the test which you describe here?

Mick

[Mick Sang]

Using a set of 10,000 random RGB values, these two profiles had an average dE 1976 of 2.3

Do you have a preference of dE 1976 over dE 2000? If so, may I ask why?

Mick

[Doug Gray]

Can you or would you direct me as to where I can find instruction on how to do the test which you describe here?

Mick

I wish I could but it's just a quick set of code I wrote to compare your profiles for consistency. It's only took a few minutes but uses a lot of functions in Matlab I've written for other purposes over the years for my main focus. That was precision target creation and analysis for a camera surveillance company I was an investor/director in.

I have no idea where one would find similar stuff. Graeme's Argyll s/w is superb and would probably be what I'd be adapting if I were not using Matlab. There may be folks out there that have done similar things using his great stuff. Another possibility is Marti's Little CMS. It's pretty easy to code with C/C++.

A side effect was becoming obsessed with getting the most out of the inkjet tech.

I generally prefer dE2000 when two very close colors are being compared but for larger distances I use dE1976. The latter is much simpler math wise but the former is better when looking at close colors in a neutral surround. But dE2000 is not as good when looking at color differences where the surround is more saturated.

Vision is just very non-linear.

[Mick Sang]

I wish I could but it's just a quick set of code I wrote to compare your profiles for consistency. It's only took a few minutes but uses a lot of functions in Matlab I've written for other purposes over the years for my main focus. That was precision target creation and analysis for a camera surveillance company I was an investor/director in.

Must be nice. I envy you. So, I'll stick with ColorThink Pro and learn as much as I can.

Mr. Gill and his Argyll s/w are indeed extremely impressive to me, based upon his posts here and other information I have encountered and read over the years. His s/w appears to be code intensive, though, which is beyond me at this stage. But, I have yet to encounter any negative speak about it. To the contrary, it is highly respected. Results from it are apparently at least a match and are often superior to the best from i1Profiler in terms of profile quality and accuracy. Otherwise, my only direct experience with Mr. Gill's products is with the ArgyllPRO ColorMeter which I have on my cell. It is a fantastically helpful tool which makes the i1 Pro even more useful.

Thank you again, sir, for all your help.

Mick

[Doug Gray]

Must be nice. I envy you. So, I'll stick with ColorThink Pro and learn as much as I can.

Don't. It was not a good investment. However I did learn a lot and had a great deal of fun in that episode of my life. I'm really a techie that's done a bit of this and that. I like to deep dive and understand stuff. And I enjoy helping people like those that helped me. If I encounter people along the way where my experience is useful, I, like most others, try to help. You just happen to have an interesting problem. You want to learn and you put a lot of effort into it. That makes it fun for me.

I'm sure if you look around you will notice you do the same for others from time to time. We all have to make money but we also should be helping each other because we all win doing so.

Mr. Gill and his Argyll s/w are indeed extremely impressive to me, based upon his posts here and other information I have encountered and read over the years. His s/w appears to be code intensive, though, which is beyond me at this stage. But, I have yet to encounter any negative speak about it. To the contrary, it is highly respected. Results from it are apparently at least a match and are often superior to the best from i1Profiler in terms of profile quality and accuracy. Otherwise, my only direct experience with Mr. Gill's products is with the ArgyllPRO ColorMeter which I have on my cell. It is a fantastically helpful tool which makes the i1 Pro even more useful.

Graeme's work is truly outstanding. He's one of the people I've donated  to.

[Mick Sang]

You just happen to have an interesting problem. You want to learn and you put a lot of effort into it. That makes it fun for me.

Truth be told, I, too, enjoy this stuff immensely. I have a lot to learn and really enjoy the learning of it. The science fascinates me. Every time I watch a print head flying back & forth with an image forming beneath with every pass, I am totally in awe of the technology. I hope someday to be able to help people in need of it as much as you and the others have helped me.

Graeme's work is truly outstanding. He's one of the people I've donated  to.

I agree. I have as well.

Mick