Post by: Doug Gray on February 25, 2019, 05:41:36 pm
1. Generate an evenly spaced RGB grid. For instance a 8 point, 3D grid with 8^3 RGB values.
2. Then generate a 7 point, 3D grid with the same spacing as the 8 point grid. This is then offset so each point in this grid is centered in each "cube" of the 8 point grid.
This makes a kind of packed structure that significantly improves both the average minimum distance and maximum distance of random RGB points which should produce better profiles compared to similar patch counts generated by I1Profiler. And, initial testing indicates it does! So far I've tested this on a single page chart with this dual grid set (512+343) plus adding additional neutrals to fill it out to the iSis 957 single page set.
And there is another cool effect. One can easily identify areas of the printer's RGB device space where non-linearities are largest. One can compare each of the centered RGB points to an average of the 8 corners it is surrounded with on the larger grid. If perfectly linear, the deltaE between the averaged 8 surrounds and the centered point would be 0. And this works. The distribution of these deltaE values turns out to be a great way to compare printer "lumpiness." I've noted that my old 9800 is quite a bit more lumpy than the 9500 and especially the Pro1000 and I now have a way to create a mechanism to characterize this lumpiness. For the 9800 the average dE for the above case is .95 while the Pro1000 came in at .47. So this is a good explanation why the 9800 needs a lot more patches to reach the same level of color accuracy as the Pro1000.
This suggests further optimization can be done.
Make a patch set 12^3 grid points and add in a centered 11^3 grid. Print the patches and measure. Now take the original set of 12^3 (1728), sort the patches against the dE (which shows which patches are making a difference) , and add the worst dE sets back into the original 1728 until the total reaches a 2 page iSis set of 1914 patches.
I'm pretty sure this would make a better profile patch set for each of my printers. And I'd only have to go through the process twice for each printer to cover the two main paper types. After that I'd be able to make great profiles for normal paper/ink and aging changes without printing and scanning bunches of pages each time.
Post by: aaron125 on February 25, 2019, 06:17:35 pm
One thing I’m misunderstanding is how the 7^3 grid can have the same spacing as the 8^3 grid? Also, how does one centre the smaller 7^3 grid in between the cubes created by the 8^3 grid nodes?
What possessed you to come up with such an intriguing concept for this patch layout? Have you tried it with any other profile creation software? A friend of mine has access to BasICColor Print RGB and in the past I’ve sent him measurement files which he then processes through Print RGB with various settings (that app has an incredible array of settings and options which one can alter/play around with to produce profiles which alter a printer’s output in ways simply not possible to reproduce with i1Profiler). One great thing about that app is that it accepts any and all patch sets equally, afaik. I’ve not tested this but it essentially states this in the user manual and one need not use a specific or even a BasICColor produced app to create the charts/patch sets which are used to creating profiles with Print RGB.
One thing you haven’t mentioned is how do the profiles you’ve produced using this patch layout do with real world images and how is the smoothness of the transitions in said profiles? Accuracy is great, but for those of us in the photography and fine art markets, accuracy isn’t of great importance if transitions, especially how smooth they are, specifically in images which have (for example) blue skies, large areas of skin tones and other areas of this nature, in which smoothness of the transitions are of utmost importance.
Anyway, I’m certainly most interested in following the progress you have with this intriguing and innovative patch set layout procedure.
Sent from my iPhone using Tapatalk
Post by: Doug Gray on February 25, 2019, 07:09:31 pm
Very interesting indeed! Do you have a script or something which you could share so others may give this a try?All my works uses scripts/functions I wrote in Matlab. They need the standard Matlab package with the image workshop which has a lot of useful color management and image related routines. I've made a number of tools that add on to this and do things like profile conversions, CGATs reading and writing. Matlab is extremely powerful and efficient and can do pretty much anything I've been able to imagine. The only exception so far was dealing with the issue of scanner wide area crosstalk. Even there I used Matlab for analysis and investigation but it was too slow to use for correcting scanned images which required many billions of math operations. So for that I used C++ and spun out 3 worker threads to speed things up.
Matlab has a "home" version that is only for non-commercial use. It's full featured and only $149 for the base package and another $45 for the image workshop. I also have a commercial version but it's about 5 years old which is largely compatible.
Quote
Think of a ruled page with an 8x8" grid spaced 1" apart. Now put dots in the center of each square. The dots form a 7x7 grid but the spacing between dots is the same as the 8x8 grid. Extrapolate to 3D.
One thing I’m misunderstanding is how the 7^3 grid can have the same spacing as the 8^3 grid? Also, how does one centre the smaller 7^3 grid in between the cubes created by the 8^3 grid nodes?
Quote
I was thinking about the stacked marble problem. If you put marbles in a box, each directly on top of the ones below you get fewer marbles in the box than if you put each marble in the detents of the layer below. I then realized that every other layer of marbles was exactly centered on the middle of the below/above layers. And thus by averaging the Lab values of the 8 colors around each marble, I could easily determine how linear the printer engine was at each point. Then I ran a statistical test with Matlab to see how much such an arrangement improved the average distance and maximum distance of randomly picked RGB values. It reduces the average minimum distance by 25% and the maximum distance by 35%. While the patch size is increased by the second, centered set, the increase is not as much as the required increase for a single set that provides similar distances.
What possessed you to come up with such an intriguing concept for this patch layout?
Quote
I tried the set of 8 points and 7 points which totals 855 patches and it produced better results than the iSis default 957 patch set. I think I could have trimmed it down to about 600 patches for a specific printer and still gotten better results than the default but I haven't tried that yet.
Have you tried it with any other profile creation software? A friend of mine has access to BasICColor Print RGB and in the past I’ve sent him measurement files which he then processes through Print RGB with various settings (that app has an incredible array of settings and options which one can alter/play around with to produce profiles which alter a printer’s output in ways simply not possible to reproduce with i1Profiler). One great thing about that app is that it accepts any and all patch sets equally, afaik. I’ve not tested this but it essentially states this in the user manual and one need not use a specific or even a BasICColor produced app to create the charts/patch sets which are used to creating profiles with Print RGB.
Quote
Yes, that's certainly an issue. One that I've done much work on for this is the 9800. There are areas, particularly near both ends of the luminance curve, where it exhibits fairly rapid shifts in a* and b* or L*. The 957 default patch set makes good profiles with the 9800 but I wound up creating a 3828 patch set specifically for printing on the 9800. It does a pretty good job on the smoothness but some of it is beyond what the I1Profiler can follow. Even at max quality (37x37x37 grid points)
One thing you haven’t mentioned is how do the profiles you’ve produced using this patch layout do with real world images and how is the smoothness of the transitions in said profiles? Accuracy is great, but for those of us in the photography and fine art markets, accuracy isn’t of great importance if transitions, especially how smooth they are, specifically in images which have (for example) blue skies, large areas of skin tones and other areas of this nature, in which smoothness of the transitions are of utmost importance.
Quote
Anyway, I’m certainly most interested in following the progress you have with this intriguing and innovative patch set layout procedure.
Sent from my iPhone using Tapatalk
Post by: GWGill on February 25, 2019, 07:43:24 pm
1. Generate an evenly spaced RGB grid. For instance a 8 point, 3D grid with 8^3 RGB values.Yep - that's called "body centered cubic". See the ArgyllCMS targen -i (http://www.argyllcms.com/doc/targen.html#i) option.
2. Then generate a 7 point, 3D grid with the same spacing as the 8 point grid. This is then offset so each point in this grid is centered in each "cube" of the 8 point grid.
Post by: Doug Gray on February 25, 2019, 08:01:37 pm
Yep - that's called "body centered cubic". See the ArgyllCMS targen -i (http://www.argyllcms.com/doc/targen.html#i) option.
I'm not surprised Graeme. Should have checked there when I thought of that packing but it was trivial to do in Matlab and I wanted to get some stats on it which Matlab makes easy.
https://argyllcms.com/doc/targen.html#b
The -b parameter sets the outer edge size of the multidimensional body centered grid of test values. The total number of patches of this type will be the -b parameter value to the power of the number of colorants plus the (number-1) to the power of the number of colorants. The grid steps are evenly spaced in device space by default, but if the -p parameter is provided, then, then the steps will be distributed according to the power value. A body centered grid is a regular grid (see -m) with another smaller regular grid within it, at the centers of the outer grid. This grid arrangement is more space efficient than a regular grid (ie. for a given number of test points, it fills the space better.)
Have you by any chance looked at deltaE2000 distribution of the points centered around the surrounding cubes against the average of the 8 surrounds? Does that make a good way to reduce patches that are providing little benefit?
Post by: Alan Goldhammer on February 26, 2019, 08:32:23 am
Post by: howardm on February 26, 2019, 10:08:59 am
How does one 'measure' or put a value on the smoothness or inversely, the lumpiness of a profile/printer/ink combo?
Do papers suffer/experience much in the way of non-linear response (lets assume the printer is linear for this)?
Post by: Mark D Segal on February 26, 2019, 10:33:52 am
So this begs the question......
How does one 'measure' or put a value on the smoothness or inversely, the lumpiness of a profile/printer/ink combo?
Do papers suffer/experience much in the way of non-linear response (lets assume the printer is linear for this)?
I'm glad you raised that Howard, because it also triggers a question about a comment in a related thread from a poster who said that profiling accuracy is of less concern to him than the smoothness of tonal gradations. In a way, it's all related. Let's say your gradient file had values all the way up the scale that were unidirectional and just separated enough from each other to preserve apparent smoothness of the gradient. Unless the printer/paper/profile combo could render those file values accurately enough, one could observe discontinuities in the gradient if the extent of the inaccuracies were large enough and contiguous enough to be visible. So the first thing I would look at for quantifying smoothness is how well the system performs at rendering it accurately from a smooth gradient file, which is a doable proposition.
Post by: aaron125 on February 26, 2019, 10:42:09 am
So this begs the question......I don’t think there is a metric of which one can make an objective measurement to give smoothness a ‘score’ like one would when measuring the difference between some colours, for example, to obtain a dE value.
How does one 'measure' or put a value on the smoothness or inversely, the lumpiness of a profile/printer/ink combo?
Do papers suffer/experience much in the way of non-linear response (lets assume the printer is linear for this)?
The smoothness of transitions afaik is a purely subjective thing where one must determine for themselves if a particular printer/substrate/ink/profile combination provides the smoothness they desire. Typically something like a Granger rainbow might be used as a kind of (still subjective, but at least it’s a known image and therefore can be used to visually compare and make a choice regarding the way a profile handles various transitions) benchmark to show how transitions in areas of the colour spectrum are handled by a profile.
I hope that’s not too confusing.
Sent from my iPhone using Tapatalk
Post by: digitaldog on February 26, 2019, 10:52:30 am
Most RGB printers are pretty well behaved in terms of linearity.Expect just about every Epson printer (which is feed RGB data but isn't an RGB printer) using the native driver. Better than it was a decade ago but far, far from linear.
Post by: Mark D Segal on February 26, 2019, 11:17:58 am
Expect just about every Epson printer (which is feed RGB data but isn't an RGB printer) using the native driver. Better than it was a decade ago but far, far from linear.
Andrew, not clear what you mean quantitatively by "far, far from linear", but my experience with the Epson SC-P5000 and various papers, printing a 100 patch grayscale ramp that increments one L* value at a time from L*1 to L*100 indicates that for the L channel, variance from reference values is usually well below dE 1 (for all the tones within gamut) and variance from zero of the a* and b* values also well less than dE 1 (both dE 76 - for dE 2000 the variances would be generally less). So to me, this seems not exactly linear, but pretty good.
Post by: Doug Gray on February 26, 2019, 11:31:07 am
I'm glad you raised that Howard, because it also triggers a question about a comment in a related thread from a poster who said that profiling accuracy is of less concern to him than the smoothness of tonal gradations. In a way, it's all related. Let's say your gradient file had values all the way up the scale that were unidirectional and just separated enough from each other to preserve apparent smoothness of the gradient. Unless the printer/paper/profile combo could render those file values accurately enough, one could observe discontinuities in the gradient if the extent of the inaccuracies were large enough and contiguous enough to be visible. So the first thing I would look at for quantifying smoothness is how well the system performs at rendering it accurately from a smooth gradient file, which is a doable proposition.
I agree that a printed, continuous gradient is a good way to view smoothness and that's where eyeballs are much better than spectros. The latter is excellent for determining accuracy but not so good at smoothness unless you scan tens of thousands of patches. And one can enhance the latter by scanning or photographing the gradient and increasing contrast. How far contrast can be increased and still retain visual smoothness (noise aside), would be a good metric. At least for that particular gradient. Analyzing the smoothness of the entire gamut is a very hard problem.
Post by: Mark D Segal on February 26, 2019, 11:40:57 am
I agree that a printed, continuous gradient is a good way to view smoothness and that's where eyeballs are much better than spectros. The latter is excellent for determining accuracy but not so good at smoothness unless you scan tens of thousands of patches. And one can enhance the latter by scanning or photographing the gradient and increasing contrast. How far contrast can be increased and still retain visual smoothness (noise aside), would be a good metric. At least for that particular gradient. Analyzing the smoothness of the entire gamut is a very hard problem.
To determine the appropriate number of test values needed for evaluating the apparent smoothness of a gradient with a spectro I think it's important to start with what constitutes a JND (just noticeable difference) in human visual perception. This vastly reduces the necessary number of patches. I agree that analyzing the smoothness of the entire gamut is a very hard problem - and that may be an understatement. One can sample it I suppose and come off with what is a practically useful appreciation.
Post by: digitaldog on February 26, 2019, 12:10:48 pm
Andrew, not clear what you mean quantitatively by "far, far from linear", but my experience with the Epson SC-P5000 and various papers, printing a 100 patch grayscale ramp that increments one L* value at a time from L*1 to L*100 indicates that for the L channel, variance from reference values is usually well below dE 1 (for all the tones within gamut) and variance from zero of the a* and b* values also well less than dE 1 (both dE 76 - for dE 2000 the variances would be generally less). So to me, this seems not exactly linear, but pretty good.Take a gradient in any working space and print it without a profile, just as you would a target. Something like this:
Post by: Mark D Segal on February 26, 2019, 12:24:26 pm
Post by: dehnhaide on February 26, 2019, 12:42:38 pm
I don't understand - What does that tell me about linearity?It could tell more about how well behaved the printer could be by default, prior to profiling. If that kind of pedigree is of any importance...
Post by: digitaldog on February 26, 2019, 12:45:51 pm
I don't understand - What does that tell me about linearity?PRINT it. As discussed, probably with ACPU. Output with the native driver appear linear?
Post by: digitaldog on February 26, 2019, 12:46:38 pm
It could tell more about how well behaved the printer could be by default, prior to profiling. If that kind of pedigree is of any importance...Exactly! Profiles do not make linearity when non linearity exists.
Post by: Mark D Segal on February 26, 2019, 03:44:38 pm
Post by: digitaldog on February 26, 2019, 03:47:48 pm
I don't see how I would learn much about the true extent of linearity just by printing that target and looking at it, because I would need to assess linearity with data. A visual check is indicative and helps detect major issues, which can be useful, but nothing much more refined.Of course it tells you a lot about the linearity or lack thereof based on the steps that block up (or don't) due to the behavior of the driver and media settings OF WHICH you'll profile from. Lots to learn once you actually make a few prints:
Try printing it without any color management in say ImagePrint (still have a copy?) vs. the native Epson driver. You'll see a major difference. But not until you make a print because unless you've experienced it, you've only imagined it.... ;)
And IF you have the hardware, you can measure each step IF you want actual numbers.
Post by: Mark D Segal on February 26, 2019, 03:55:19 pm
Post by: digitaldog on February 26, 2019, 03:58:03 pm
Make a print, you'll see, at least on every Epson I've ever tried, this isn't at all the case.
Post by: GWGill on February 26, 2019, 07:26:49 pm
Have you by any chance looked at deltaE2000 distribution of the points centered around the surrounding cubes against the average of the 8 surrounds? Does that make a good way to reduce patches that are providing little benefit?Sorry, it's a long time ago that I played with BCC sampling, and I moved on to other sampling approaches. Certainly I've used similar ideas to estimate curvature of the device response, and the default targen "Optimised Farthest Point Sampling" uses the difference between linear device value and measured value interpolation of each Voronoi region as a curvature measure.
A lot of the original inspiration came from 3 papers written by Don Bone, and in particular the paper "Adaptive color-printer modeling using regularized linear splines" which was a CSIRO technical report TR-HJ-92-19 and also published by the SPIE (http://spie.org/Publications/Proceedings/Paper/10.1117/12.149033) summarizes a lot of it, but also includes some interesting work on adaptive sampling. At the time (1993) strip and table reading instruments were rare, and Don was using a point by point measurement, so he came up with an interactive adaptive sampling approach. After measuring a smaller uniform grid, and used a couple of techniques to decide on which points to print and measure next. (Since he was working with a copier, it was relatively easy to print each sample as needed.) He was effectively used a curvature criteria, in that areas which were poorly predicted by linear interpolation were sampled in more detail.
One of the reasons I moved away from regular device grids is that they tended to interact with spline like models badly, since they "resonated" easily. Some sort of damping or regularization is always needed with spline or polynomial models, but it's less critical if the sampling distribution is more stochastic than regular.
Post by: aaron125 on February 26, 2019, 07:45:45 pm
Sent from my iPhone using Tapatalk
Post by: Mark D Segal on February 26, 2019, 08:27:06 pm
Actually I see little if any reason to measure the target. More important is to visually see what the profile experience is up against in it's 'raw' unprofiled behavior and the target does just that. But more importantly, it dismisses the generalization that: Most RGB printers are pretty well behaved in terms of linearity.
Make a print, you'll see, at least on every Epson I've ever tried, this isn't at all the case.
OK - I see where you are coming from: use the target to look at how the printer behaves in its native condition uninfluenced by profiles, then, if I want, see the difference it makes when the same target is printed with the printer profiled. That's interesting and I'll do it. I'lll revert to this discussion after I make the two prints and have a look at them. Meanwhile, if you could send me the reference values for the target it would save me some time, because I still believe that visual impressions should be complemented with data.
Post by: digitaldog on February 26, 2019, 08:30:14 pm
OK - I see where you are coming from: use the target to look at how the printer behaves in its native condition uninfluenced by profiles, then, if I want, see the difference it makes when the same target is printed with the printer profiled.Yes. But the profile isn't going to do anything to make the output more linear than it really is.
Post by: Mark D Segal on February 26, 2019, 08:42:35 pm
When I open the target in Photoshop it tells me there is no embedded colour space and provides the usual options for embedding one: Don't Color Manage, Use Working Space, use another working space. Which option do you recommend for this experiment?
I would also observe this is a very small target at 72 PPI. Would it not be better to do this with a larger version?
Post by: digitaldog on February 26, 2019, 08:44:50 pm
You can resize it but you don't need anything large for output and it should be large enough to measure, if that's the idea, with an i1Pro.
Post by: Mark D Segal on February 26, 2019, 10:37:42 pm
You want to treat it like a target because that's what it is.
You can resize it but you don't need anything large for output and it should be large enough to measure, if that's the idea, with an i1Pro.
Targets can be treated in various ways depending on context, that's why I asked. So you haven't answered my question in regard to printing it from Photoshop as a hard-proofing exercise.
However, I did print it from ACPU as if a profiling target and in that context and application of course no such options occur, so it's not an issue for that purpose. Now I have this print of your target sitting here in front of me. What am I supposed to infer about linearity from looking at this? It makes no obvious sense to me. Is it simply telling me that if it diverges from how it looks on the monitor my printer is non-linear?
Post by: digitaldog on February 26, 2019, 10:47:54 pm
Targets can be treated in various ways depending on context, that's why I asked. So you haven't answered my question in regard to printing it from Photoshop as a hard-proofing exercise.Does each step look linear from another, do you see the same separation of each color equally? No.
However, I did print it from ACPU as if a profiling target and in that context and application of course no such options occur, so it's not an issue for that purpose. Now I have this print of your target sitting here in front of me. What am I supposed to infer about linearity from looking at this? It makes no obvious sense to me. Is it simply telling me that if it diverges from how it looks on the monitor my printer is non-linear?
The monitor has absolutely nothing to do with anything in this respect.
Post by: Mark D Segal on February 26, 2019, 11:12:41 pm
Post by: Stephen Ray on February 26, 2019, 11:59:36 pm
Post by: Doug Gray on February 27, 2019, 02:25:52 am
Here's an example of how the Canon Pro1000 is smoother, but also deviates more in a* and b* over the device space neutral tone curve (RGB 0 to 255).
Of note is that on the Epson 9800 the a* and b* are "lumpy" and these lumps are rapid enough that a significant part cannot be compensated for by a profile. Even partially compensating for these lumps requires requires a lot of extra, near neutral patches. The Canon Pro 1000 shows much larger excursions but these are easily compensated with the exception of the large changes as low L*. Additional, near neutral patches in this area works well because the curve is smooth and without the rapid changes in slope that occurs over most of the 9800 curves.
Post by: Mark D Segal on February 27, 2019, 08:00:01 am
Does each step look linear from another, do you see the same separation of each color equally? No.
The monitor has absolutely nothing to do with anything in this respect.
Again, what does "look linear" mean? Maybe I don't know what linear looks like even though it may be staring me in the face.
What the monitor has to do with is that without file reference data, what I see on the monitor is the only other available comparator. Anyhow, I'm moving on from this - no more time for it and less important to me than the overall performance of the PPP system as a whole (printer/paper/profile).
Post by: digitaldog on February 27, 2019, 09:23:59 am
If a picture is worth a number of words, here is a scan of what emerged from the printer. To me, depending on what "linear" looks like, it would seem the top and bottom several rows appear non-linear.Yup!
Post by: Stephen Ray on February 27, 2019, 12:50:28 pm
Exactly! Profiles do not make linearity when non linearity exists.If a printer were non-linear, say heavy in the cyan channel, would a profile not correct the colors to be where they should?
Post by: digitaldog on February 27, 2019, 01:13:40 pm
If a printer were non-linear, say heavy in the cyan channel, would a profile not correct the colors to be where they should?Printer profiles do as the name implies: it profiles (fingerprints) device behavior. Such profiles know nothing about "correct" colors, certainly in context. They take the data provided (in this case RGB) and convert to another RGB color space through a profile connection space, usually Lab and as such, by the time the profile receives this Lab data, this output profile has no idea about the 'original' RGB color space.
Post by: Stephen Ray on February 27, 2019, 01:34:35 pm
Printer profiles do as the name implies: it profiles (fingerprints) device behavior. Such profiles know nothing about "correct" colors, certainly in context. They take the data provided (in this case RGB) and convert to another RGB color space through a profile connection space, usually Lab and as such, by the time the profile receives this Lab data, this output profile has no idea about the 'original' RGB color space.In this context of InkDensity.tif, your answer is true because the file is without an ICC tag to begin with?
Post by: aaron125 on February 27, 2019, 01:35:07 pm
If a printer were non-linear, say heavy in the cyan channel, would a profile not correct the colors to be where they should?I’ve always thought that the easiest way to remember what a profile is does is to think of it in terms of: calibration/characterisation. If one thinks of a monitor and before it was common for the computer to adjust the internal monitor LUTs, one used to have to rely on altering brightness, RGB gains, etc oneself. So, first one calibrates the monitor, putting it into a particular state such that the brightness is equal to x, whitepoint is equal to y, etc. and then it is profiled, characterised in said state. So the character of the monitor is determined from flashing up a bunch of different colours and this character is akin to a map. If I want colour w to be displayed, what signals do I need to send to the monitor such that it will display w as close as it physically can? In that regard, whether it be a monitor or a printer (rather than flashing a bunch of colours, the printer prints out said colours in terms of being fed a bunch of colours from a patch target; the calibration part for most RGB printers these days is commonly selecting a particular driver setting for the media, so choosing Epson Luster configures the printer to put down a specific amount of ink from a specific head height, etc.) or pretty much and device, I don’t think a profile is able to correct much at all. It’s simply a map of the device capabilities, which is it’s character/peculiarities. To that end, no, I don’t believe a profile is able to correct some colours to put them where they should be, because how is the profile supposed to know where those colours should be placed(?).
Sent from my iPhone using Tapatalk
Post by: digitaldog on February 27, 2019, 01:39:14 pm
In this context of InkDensity.tif, your answer is true because the file is without an ICC tag to begin with?No, as it has to be assumed to be in some color space for any conversion. It is untagged because it's to be treated as a target. But you can't run it through any output profile without some assumption to get to Lab in the CMM. Depending on what software is being used for the conversion, that assumption is either sRGB or whatever RGB working space is set in Photoshop.
The idea is to print this out without color management. To test differing media settings prior to making a profile. To visually see what media setting, based on the output without color management might be the best choice. The idea is to dismiss the generalization as I noted, that RGB printers are well behaved. Some are, some are not. Much has to do with the driver handling the data. As you can see from Mark's output, or if you print this yourself, certainly on an Epson, the native driver isn't close to producing linear output. So we're kind of going around in circles, the target was created for a specific goal and I brought it up simply for others to use to see how their drivers behave with a fixed media setting and without color management.
Post by: Stephen Ray on February 27, 2019, 02:04:10 pm
No, as it has to be assumed to be in some color space for any conversion. It is untagged because it's to be treated as a target. But you can't run it through any output profile without some assumption to get to Lab in the CMM. Depending on what software is being used for the conversion, that assumption is either sRGB or whatever RGB working space is set in Photoshop.Gotcha.
The idea is to print this out without color management. To test differing media settings prior to making a profile. To visually see what media setting, based on the output without color management might be the best choice. The idea is to dismiss the generalization as I noted, that RGB printers are well behaved. Some are, some are not. Much has to do with the driver handling the data. As you can see from Mark's output, or if you print this yourself, certainly on an Epson, the native driver isn't close to producing linear output. So we're kind of going around in circles, the target was created for a specific goal and I brought it up simply for others to use to see how their drivers behave with a fixed media setting and without color management.
Post by: Stephen Ray on February 27, 2019, 03:23:46 pm
To that end, no, I don’t believe a profile is able to correct some colours to put them where they should be, because how is the profile supposed to know where those colours should be placed(?).So, you're saying a CMM is the actual component responsible for the corrections, yes?
Sent from my iPhone using Tapatalk
Post by: digitaldog on February 27, 2019, 03:26:26 pm
So, you're saying a CMM is the actual component responsible for the corrections, yes?Has nothing to do with corrections, CMM or otherwise. The CMM is simply the profile connection space component that converts between color spaces. Profiles define. They don't correct. Calibration can 'correct' (produce a desired behavior of a device, like a display). The profile reflects this condition for the CMS.
Humans correct. Based on viewing pixels in context and making subjective decisions (this is too dark, the WB isn't pleasing etc).
Post by: Stephen Ray on February 27, 2019, 03:30:05 pm
Has nothing to do with corrections, CMM or otherwise. The CMM is simply the profile connection space component that converts between color spaces. Profiles define. They don't correct. Calibration can 'correct' (produce a desired behavior of a device, like a display). The profile reflects this condition for the CMS.Again, gotcha.
Humans correct. Based on viewing pixels in context and making subjective decisions (this is too dark, the WB isn't pleasing etc).
Post by: aaron125 on February 27, 2019, 03:32:00 pm
So, you're saying a CMM is the actual component responsible for the corrections, yes?No, a CMM can’t make any corrections because it has no idea of what is ‘correct’ and what is ‘wrong’. I think what you may be referring to as far as corrections are concerned is actually the calibration process. You initially mentioned correcting the cyan channel. As in RGB devices there is no cyan channel, I’m inferring you’re talking about CMYK devices, regardless of whether it be an inkjet printer or a printing press. In this case, it would be the calibration process which would make the corrections, as in when using a RIP and prior to creating a profile, one sets ink limits, etc. and this would be where the corrections you speak of are made.
Sent from my iPhone using Tapatalk
Post by: Stephen Ray on February 27, 2019, 04:00:03 pm
No, a CMM can’t make any corrections because it has no idea of what is ‘correct’ and what is ‘wrong’. I think what you may be referring to as far as corrections are concerned is actually the calibration process. You initially mentioned correcting the cyan channel. As in RGB devices there is no cyan channel, I’m inferring you’re talking about CMYK devices, regardless of whether it be an inkjet printer or a printing press. In this case, it would be the calibration process which would make the corrections, as in when using a RIP and prior to creating a profile, one sets ink limits, etc. and this would be where the corrections you speak of are made.After the initial RIP setup steps of calibration, etc., one proceeds to generate the ICC output profile as the last step. How would one describe, or what keyword would one use, the process of optimizing the accuracy of colors? I've been calling it "correction" (as in correcting deviant values) but is "re-mapping" or "converting" a better term?
Sent from my iPhone using Tapatalk
Post by: aaron125 on February 27, 2019, 04:09:07 pm
After the initial RIP setup steps of calibration, etc., one proceeds to generate the ICC output profile as the last step. How would one describe, or what keyword would one use, the process of optimizing the accuracy of colors? I've been calling it "correction" (as in correcting deviant values) but is "re-mapping" or "converting" a better term?No, I think you’re spot on and correction is fine imho. I’ve only had a bit of experience with RIPs, mostly using EFI Colorproof XF but I’m pretty sure they all use a reasonably similar process. Afaik, it’s not the setting up of the RIP where one calibrates but it should be done for every new media to be used with the RIP.
Sent from my iPhone using Tapatalk
Post by: Stephen Ray on February 27, 2019, 04:18:05 pm
No, I think you’re spot on and correction is fine imho. I’ve only had a bit of experience with RIPs, mostly using EFI Colorproof XF but I’m pretty sure they all use a reasonably similar process. Afaik, it’s not the setting up of the RIP where one calibrates but it should be done for every new media to be used with the RIP.Thanks for the words of confidence!
Sent from my iPhone using Tapatalk
Post by: Doug Gray on February 27, 2019, 07:26:16 pm
ICC.1:2010 specifies the profile format defined by the International Color Consortium ® (ICC). The
intent of this format is to provide a cross-platform profile format for the creation and
interpretation of colour data. Such profiles can be used to translate between different colour
encodings, and to transform colour data created using one device into another device’s
native colour encoding. The acceptance of this format by application and operating system
vendors allows end users to transparently move profiles, and images with embedded profiles, between
different systems. For example, this allows a printer manufacturer to create a single profile for
multiple applications and operating systems.
A CMM is mostly a module that does math. It's coded to run on a CPU, most commonly a iOS, Windows, or Linux system. It performs the math for applying a profile. A CMM, properly constructed*, will produce highly consistent results for printing because the ICC defines their requirements. OTOH, a Profile is a binary object that is independent of OS's. The same profile can be used on any system. But a printer profile is specific to each printer and paper combination even though it can be used in any OS. A profile is a binary recipe that provides consistency more than anything else by mapping the printer's actual RGB (or CYMK) response to/from what colors it will actually print.
But not all CMMs are created equal. There have been different interpretations of what a CMM should do in certain circumstances. The most serious differences are mapping RGB colorspaces that have different white points. Microsoft's CMM, designed 20 years ago or so, will produce a vastly different image converting Adobe RGB to ProPhoto RGB using Abs. Col. with a large blue shift while Adobe's CMM will not. Adobe's follows a "clarification" the ICC made around 2002 while Microsoft's retains an older interpretation.
Microsoft's CMM also makes larger, though still small, errors converting back and forth between colorspaces.
Post by: Stephen Ray on February 27, 2019, 09:23:42 pm
The smoothness of transitions afaik is a purely subjective thing where one must determine for themselves if a particular printer/substrate/ink/profile combination provides the smoothness they desire. Typically something like a Granger rainbow might be used as a kind of (still subjective, but at least it’s a known image and therefore can be used to visually compare and make a choice regarding the way a profile handles various transitions) benchmark to show how transitions in areas of the colour spectrum are handled by a profile.Is Bill Atkinson's file "Twenty-eight balls" regarded as a good test for smoothness?
Sent from my iPhone using Tapatalk
Post by: Doug Gray on February 27, 2019, 09:56:19 pm
Is Bill Atkinson's file "Twenty-eight balls" regarded as a good test for smoothness?
It is. It was actually designed for that purpose according to an email Andrew posted at dpreview. To do so print it untagged like you would print a profile target chart using ACPU or anything else that disables color management.
It doesn't have to (and typically won't) print the same on different printers. One looks for abrupt banding. A good printer should have no areas where there is abrupt banding. Do not compare it with color managed prints of the same. This shows a different kind of banding more related to how the profiles being used were created. The former is direct to the printer and no profile is involved. The latter can be used to select profiles. For instance the canned profiles in many Epson printers will print the blue ball as black. A different, custom profile may print it somewhat close to how it looks on a display. One can chance the profile's Perceptual table by a few different sliders in I1PRofiler to affect how the different balls print.
Post by: Doug Gray on February 28, 2019, 09:24:18 pm
1. Variation just due to printing two successive pages of the same image. I typically get a variation of .15 to .20 dE00 with glossy type papers and .1 to .15 dE00 with matte.
2. Variation when patches are randomized. Two pages with the same set of patches but randomized differently show a dE00 of about .25-.30 with glossy paper and .15 to .20 with matte. So location changes increases the variation in patches.
3. Subtle effects. I see smaller changes based on how humid it is when the print is made, .05 to .10 differences. And drying time of course effects things as well with about a .10 shift from 2hrs to 24hrs. To control this I try to keep the same drying time between the print and the scan. Within a few hours if the drying time is 24 hrs, within 5 minutes for a 2 hr drying time. Almost all the changes occur while the surface is "wet" which can be easily checked with an IR imager since evaporation, even at low levels when it's 95% dry, can create a 2C down to .05C temp gradient from unprinted regions. This time varies between about 15 minutes on a real dry day to 45 minutes at 70% humidity.
Measuring small changes like the above can be done by making many copies of the same color. For instance duplicating each of 100 patches 9 times and scanning the 9000 patch set. This duplication reduces the variation (standard deviation of the mean) by about a factor of 3. Increasing the duplication to 50 patches reduces the uncertainty by a factor of about 5. This is doable, and I've spend too much time trying to quantify some of these smaller effects.
So one can print a color managed CC with each of the 24 patches duplicated 9 times and that gives a good measure off accuracy. But only for those 24 CC colors. Alternately, one can print say, 318, in gamut, patches duplicated 3x, with known LAB values and get a wider spread of color accurate data. But that still only gets some info or profile quality.
I realized the way I'm approaching comparing profiles to each other needs to be done a different way to get reasonable results in a short time without wasting tons of paper. Here's what I came up with.
Given two profiles we will call profile A and Profile B do the following:
1. Generate 100000 random RGB triplets in device space.
2. Find the expected Lab values using AtoB from each profile.
3. To compare the profiles we concentrate on the differences. So...
4. Calculate the dE00 on the 100000 Lab values from each profile.
5. Sort them for the largest dE00s.
6. Print the the RGB triples that produce the largest dE00 as a target.
7. Measure the actual Lab values of those triplets.
8. Compared the Lab values as measured against the predicted Lab values from each profile.
9. The profile that produces the closest dE00's is a better profile.
Easy, Peasy. Only have to print and measure two, single page targets!
Here's the results of from comparing the default iSis 957 patch against a printer/paper optimized target.
The Optimized patch set was created by using a 9x9x9 grid (729 patches). together with 512 patches on the centers (512 additional). Then the fraction of the patches that produced the largest dE compared to the 8 surrounding patches was sorted and the ones with the largest dE00 ( > .8 ) were added to the 729 to make 957 total patches which were then compared to the 957 iSis default patch set.
Post by: Mark D Segal on February 28, 2019, 10:16:59 pm
Post by: digitaldog on February 28, 2019, 10:36:31 pm
Post by: Doug Gray on February 28, 2019, 10:53:41 pm
Technically interesting and very methodical, but would any one be able to perceive any difference in prints made with different profiles generated from the one approach or the other?
Not with this particular printer/paper. I've generally found the 957 iSis Patch set works very well and haven't found any improvement visually using more patches like the Atkinson set outside of the 9800. I have found some improvement using larger patch sets with the 9800. Especially for B&W where the 9800 has some abrupt shifts the 9500 or Pro1000 does not. I have not yet investigated using this technique with the 9800 but I suspect it would improve it also. Right now my best 9800 profile set uses 3828 patches. I'm hoping this process lets me reduce the patch set size while stull producing excellent profiles.
Ideally, I'd like to make a set of patches that is small but still produces great results for each of my printer/paper combos.
But yeah, I'm a little OCD and I like to quantize stuff.
Post by: Doug Gray on February 28, 2019, 10:57:34 pm
There is yet another variation; noise, slightly different measurements of the same target read two times in a row, even a minute apart. You'd need to do this multiple times, average the dE and subtract that. Fortunately everything is below a visual difference.There is indeed. And it's different between glossy and matte types as well. Typical differences for matte are on the order of .02 to .03 ave. with glossy slightly higher. I chalk it up partially to noise and partially to the glossy having more surface variations and subsequent passes having slightly different registration against the end diamonds.
BTE, To address, and minimize these effects, when I print targets to read with the iSis 2, I add a registration bar at the bottom so I can flip the paper around and read it backwards. Then the CGATs file can be flipped end over end and the results compared and averaged. It turned out to be differ about 1/4 of what reading a second randomized target produces. You can see the second registration bar on the ColorChecker like iSis targets I've posted on other threads.
Post by: MHMG on February 28, 2019, 11:09:18 pm
Technically interesting and very methodical, but would any one be able to perceive any difference in prints made with different profiles generated from the one approach or the other?
The answer lies in what colors are generating the dE2000 errors greater than 1 JND (just noticeable difference = 1 dE2000 unit) and how they present themselves in the printed image, i.e. how they alter visual contrast relationships within the image and also how colorful the colors with highest dE2000 errors are. Viewers are much more aware of neutral and near neutral dE2000 errors than the same dE2000 errors occurring in vivid colors when the colors are presented in the context of a complex image rather than as two isolated colors side-by-side on a neutral surround at a 2 degree or 10 degree angle of view. So, the answer for the results Doug has presented is yes they might be observable and thus meaningful, or no, they may not be observable depending on the nature of the dE2000 errors (hue, chroma or lightness errors?) and how they are geometrically presented within the printed image. Image banding, for example is typically a contrast error that can develop with very small L* errors in neighboring color values. It can also manifest as hue and chroma variations, but more often it grows more objectionable with L* errors. To summarize, evaluating color for color's sake with color difference models like dE2000 is not the same thing as evaluating both color and tonal relationships in the context of complex images containing viewer identifiable information content. Another metric beside dE or dE2000 is required to quantify the results... the reason the I* metric was invented. Other than that, one can only resort to printing an appropriate number of challenging images and then subjectively assess the situation. Easier said than done because choice of image also plays a big role.
cheers,
Mark
http://www.aardenburg-imaging.com
Post by: Mark D Segal on February 28, 2019, 11:21:38 pm
Post by: Mark D Segal on February 28, 2019, 11:22:57 pm
But yeah, I'm a little OCD and I like to quantize stuff.
And I like to quantify stuff.....
-:)
Post by: Doug Gray on February 28, 2019, 11:35:40 pm
The answer lies in what colors are generating the dE2000 errors greater than 1 JND (just noticeable difference = 1 dE2000 unit) and how they present themselves in the printed image, i.e. how they alter visual contrast relationships within the image and also how colorful the colors with highest dE2000 errors are. Viewers are much more aware of neutral and near neutral dE2000 errors than the same dE2000 errors occurring in vivid colors when the colors are presented in the context of a complex image rather than as two isolated colors side-by-side on a neutral surround at a 2 degree or 10 degree angle of view. So, the answer for the results Doug has presented is yes they might be observable and thus meaningful, or no, they may not be observable depending on the nature of the dE2000 errors (hue, chroma or lightness errors?) and how they are geometrically presented within the printed image. Image banding, for example is typically a contrast error that can develop with very small L* errors in neighboring color values. It can also manifest as hue and chroma variations, but more often it grows more objectionable with L* errors. To summarize, evaluating color for color's sake with color difference models like dE2000 is not the same thing as evaluating both color and tonal relationships in the context of complex images containing viewer identifiable information content. Another metric beside dE or dE2000 is required to quantify the results... the reason the I* metric was invented. Other than that, one can only resort to printing an appropriate number of challenging images and then subjectively assess the situation. Easier said than done because choice of image also plays a big role.
cheers,
Mark
http://www.aardenburg-imaging.com
Very true, and by a factor of 5 or more compared to dE76 in may cases. It's one of the reasons I have focused so much on the neutrals for my 9800. I am very pleased by the smoothness of the Pro1000 in the neutrals. Profiles can accommodate smoothness much easier than the rapid changes in a* and b* the 9800 has.
And another point that coincides with your observations. I worry about overfitting. With too many patches the profile is at increased risk of overcompensating resulting in potentially visible changes if one happens to run across an overcompensated region. My sense is that the smallest number of patches needed for a given printer/paper combo will generally yield better results than just using a very large number of patches. Especially w/o duplicate patch averaging but that gets tedious fast. Using duplicates of 4k patches starts to get painful. Even with an iSis. Hence the search. Duplicates of a good, working, optimized 957 patch set is doable and not painful.
Also, your points about JND colors is, indeed, context sensitive. dE2000 is pretty good when patches are not adjacent. If adjacent, smaller dE00 differences are often visible as banding. Especially with large, dark regions. dE00 is excellent at JND for two patches, separated by a thin boundary with 2 degree spread in a surround of L*50. But JND is magnified any time the surround is similar to the patches.
Post by: MHMG on March 01, 2019, 09:11:12 am
...And another point that coincides with your observations. I worry about overfitting. With too many patches the profile is at increased risk of overcompensating resulting in potentially visible changes if one happens to run across an overcompensated region. My sense is that the smallest number of patches needed for a given printer/paper combo will generally yield better results than just using a very large number of patches. Especially w/o duplicate patch averaging but that gets tedious fast. Using duplicates of 4k patches starts to get painful. Even with an iSis. Hence the search. Duplicates of a good, working, optimized 957 patch set is doable and not painful.
Mark Linquist, John Dean, and I have been working collaboratively on pushing the ICC profiling envelope to extremes with very high patch count targets. A few months ago I started building "nested grid" charts that put the RGB spacing frequency in these charts on steroids 8). The biggest one we have printed and measured contains 10445 patches, obviously an insane level to try without an automatic scanning spectrophotometer, but our Z3200 printers have them built in. You cue up the target, and go do something else for a few hours, and the Z3200 chugs along happily (well most of the time) in a corner!
I have been curious about overfitting and other possible glitches as well when using these supersized charts, but at least on the Z3200, these large patch counts don't seem to hurt the outcome at all, and indeed seem to keep improving the Z's color and tonal output in very subtle ways. Grayscale neutrality is superb, and the B&W prints one can make in full color mode using these supersized charts is in a league of its own. Whether the 10,445 patch is noticeably better than results generated from a smaller nested grid chart (e.g., I made one with 1887 patches) is what we are trying to quantify next (both objectively with metrics and subjectively by using our eyeballs!), but these nested grid targets do noticeably improve Z3200 output compared to the Atkinson 12>3 1728 patch color chart. So, at least for our Z's, nested grid color charts show clear benefits.
BTW, the Aardenburg nested grid color charts were built just as you described at the beginning of this thread and what Graeme Gill also describes as "body centered cubic" designs. Great minds think alike?? ;) Anyway, I build them with the value offset technique and other value filtering methods using a combination of Excel and Patchtool to create these "color charts on steroids".
Later today, I will try to start a new thread on the Aardenburg/Linquist/Dean Z3200 ICC profiling collaboration and also share more info about the nested grid method. In the meantime, attached is a photo showing Mark L.'s Z3200 after printing a 9460 patch nested grid color chart, and more printed charts hanging on the wall in the background. We have been busy! The three nested grids are clearly revealed in the Z3200's print of the 9460 color chart on the table in the foreground.
all the best,
Mark
http://www.aardenburg-imaging.com
Post by: John Nollendorfs on March 01, 2019, 11:36:55 am
Post by: howardm on March 01, 2019, 01:49:34 pm
Post by: Alan Goldhammer on March 01, 2019, 02:50:12 pm
Actually, I just peeked and it's $1999. Da*n tempting but I really cannot justify (like THAT ever mattered :D )I would not have any where to put it!!!!
Post by: MHMG on March 02, 2019, 03:16:48 pm
Experiments with Interstitially Nested Grid Profiing Charts and the Z3200. (https://forum.luminous-landscape.com/index.php?topic=129356.msg1098313#msg1098313)
cheers,
Mark
http://www.aardenburg-imaging.com
Post by: Doug Gray on March 04, 2019, 02:40:11 pm
Description of attached chart:
The top row compares packed in single grid profiles over the full gamut. The bottom row compares only neutrals L* from the black point to 100 in Rel. Col. The bottom histograms are lumpy because only a small number of patches participate in creating the profile's response over neutrals. Also, the dE00 numbers are larger because dE00 actually amplifies differences in and near neutrals while strongly attenuating differences in most other areas of the gamut. dE00 is more representative of human visual sensitivity which is higher in the neutrals. The bottom row truncates the histogram at dE00 > 1 while the top row extends to 3.0.
Observations:
Packed grid profiles produce excellent neutrals compared to standard, single grid profiles. The 91 patch profile varied from the 1914 patch profile by an average dE00 of only .52! over the full gamut. And the neutrals, which are much more sensitive to dE00, were .64. The single grid patch set with similar performance required 343 patches.
Generally, Packed, N, patch sets produce better overall results compared to similarly sized , single N+1, patch sets with one grid. But packed patches significantly improve neutral accuracy.
Post by: Doug Gray on March 05, 2019, 01:19:24 am
https://forum.luminous-landscape.com/index.php?topic=129380.msg1098608#msg1098608
Post by: rasworth on March 05, 2019, 03:01:20 pm
I started with Doug's 8/7 set, 890 patches, because it is close to my i1Profiler auto generated 905 patch profile. No problems creating profile, using an i1Pro UV cut (M2) instrument, sticking with Version 2 icc. ColorThink Pro shows a close match on the profile internals. Made a test print with my standard (PrinterEvaluationImage_V002_ProPhoto.tif), no discernable difference to one made with my 905 patch profile. I paid close attention to the neutral strips, again can't tell any difference, both look neutral to my aging eyes. Also the DG87 soft proof was equivalent to my 905 profile, that is to say very good.
Secondly used Doug's 6/5 set, 376 patches, on a single letter size sheet, cranked out under same conditions. To my surprise, no discernable difference on the test print, again paying close attention to the neutrals. The profile internals matched, with a slight difference in gamut volume. And the soft proof was equally good.
I realize this data is non-quantitative and only two sample points, but so far so good. I'm particularly impressed with the results from the 6/5 376 patch set, either there is magic in Doug's algorithm or I have a very "smooth" printer, maybe both. Seems to me I've been wasting paper and ink!
Richard Southworth
Post by: Mark D Segal on March 05, 2019, 03:32:01 pm
Post by: Doug Gray on March 09, 2019, 01:10:27 am
I have created a patch set that incorporates all of these from N=4 to N=9 which fits on 4 iSis Letter size targets. It's interesting to see the improvements as N increases on the same paper with my printers.
A further improvement can be made by adding a finer mesh to the near neutrals as MHMG has discussed. I have done this in the past with some success to improve my 9800's performance with near neutrals for good B&W using standard color profiles. That's going to be my next focus as the larger performance over the color gamut is quite good.
I've added a print out of the stats for the center dE76 and dE00 packed grids running Costco Glossy on my 3 printers and MP101 matte on my Canon Pro1000. Note that the matte performance is significantly better than the glossy. I've found this is also true for general profile accuracy. Matte using Canson's Photo Rag has shown similar improvements and it has a white point around L=98. Also no OBAs unlike Canon's MP101.
The columns on the left are the full gamut. The ones on the right are only the neutrals. The neutrals are more irregular as there are much fewer centered patches in each set only (7 to 15). So they are not monotonically improving but this is simply a statistical variation due to the limited sample set. Also dE00 is much more sensitive in the neutrals as can be seen as sometimes the dE00 is larger than the dE76.
Post by: Doug Gray on March 10, 2019, 12:48:31 am
So I then used a Pro1000 profile I'd made earlier to generate a set of 102 actual (not device) neutral RGB values evenly spaced and added it to the fill out the set. Printed and made a profile from it.
Theory being that this set would provide closer points on the real neutral axis and result in a better profile. It in fact did. On a separate set of patches from L=3 to 95 (the paper's gamut limits) the average dE00 over the neutrals was improved from .56 to .35 which is excellent for a 957 patch set.
So now my patch strategy is:
1. Create a packed set of patches for each printer/paper type(glossy or matte). Looks like the N8 set which fits on a single page is pretty good. N10 needs 2 sheets, N13 needs 4 sheets but it's not clear how much of an improvement these would bring.
2. use a previously generated profile for the printer/paper type from the iSis default set to create device RGB values along the actual neutral axis and add these to fill out unused space.
This creates new profile patch sets tailored to optimize neutrals while giving good performance in the rest of the gamut. These sets need to be created for each paper type/printer combo but the same one can be used for similar types on the same printer. For instance glossy and luster or MP102 and Canson Photo Rag.
Post by: dehnhaide on March 10, 2019, 03:43:54 am
I made a packed patch set with an N=8 outer grid, N=7 inner grid which left 102 unused patches to fill out the 957 patches in a single letter page iSis set.
So I then used a Pro1000 profile I'd made earlier to generate a set of 102 actual (not device) neutral RGB values evenly spaced and added it to the fill out the set. Printed and made a profile from it.
Theory being that this set would provide closer points on the real neutral axis and result in a better profile. It in fact did. On a separate set of patches from L=3 to 95 (the paper's gamut limits) the average dE00 over the neutrals was improved from .56 to .35 which is excellent for a 957 patch set.
So now my patch strategy is:
1. Create a packed set of patches for each printer/paper type(glossy or matte). Looks like the N8 set which fits on a single page is pretty good. N10 needs 2 sheets, N13 needs 4 sheets but it's not clear how much of an improvement these would bring.
2. use a previously generated profile for the printer/paper type from the iSis default set to create device RGB values along the actual neutral axis and add these to fill out unused space.
This creates new profile patch sets tailored to optimize neutrals while giving good performance in the rest of the gamut. These sets need to be created for each paper type/printer combo but the same one can be used for similar types on the same printer. For instance glossy and luster or MP102 and Canson Photo Rag.
Hi Doug,
One, maybe OT question: when printing the targets on your Canon what print quality settings are you chosing among "standard / high / highest (where available)" and is this supposed to make a huge difference in the results?
Thanks!
Sent from my iPhone using Tapatalk
Post by: Alan Goldhammer on March 10, 2019, 08:56:01 am
I made a packed patch set with an N=8 outer grid, N=7 inner grid which left 102 unused patches to fill out the 957 patches in a single letter page iSis set.I think this is similar to my approach of using Argyll to generate a precondition profile and then the second profile using added neutrals with the appropriate Argyll command. Of course I only have an i1 Pro so I have to keep the patch numbers manageable. I still haven't had an opportunity to do the neutrals test that you outline earlier in this thread but will get to that as I'm curious what my results look like.
So I then used a Pro1000 profile I'd made earlier to generate a set of 102 actual (not device) neutral RGB values evenly spaced and added it to the fill out the set. Printed and made a profile from it.
Theory being that this set would provide closer points on the real neutral axis and result in a better profile. It in fact did. On a separate set of patches from L=3 to 95 (the paper's gamut limits) the average dE00 over the neutrals was improved from .56 to .35 which is excellent for a 957 patch set.
So now my patch strategy is:
1. Create a packed set of patches for each printer/paper type(glossy or matte). Looks like the N8 set which fits on a single page is pretty good. N10 needs 2 sheets, N13 needs 4 sheets but it's not clear how much of an improvement these would bring.
2. use a previously generated profile for the printer/paper type from the iSis default set to create device RGB values along the actual neutral axis and add these to fill out unused space.
This creates new profile patch sets tailored to optimize neutrals while giving good performance in the rest of the gamut. These sets need to be created for each paper type/printer combo but the same one can be used for similar types on the same printer. For instance glossy and luster or MP102 and Canson Photo Rag.
Post by: vikcious on March 10, 2019, 01:38:57 pm
Hi Doug,
One, maybe OT question: when printing the targets on your Canon what print quality settings are you chosing among "standard / high / highest (where available)" and is this supposed to make a huge difference in the results?
Thanks!
Sent from my iPhone using Tapatalk
+1
Post by: Doug Gray on March 10, 2019, 05:40:54 pm
I have not investigated changing settings in the Pro1000 but it appears "highest" is not available for the media type I'm using which is Canon Plat. Pro. and MP102 matte.
I intend to test those after I've beat this horse sufficiently and established the best protocol.
Post by: Mark D Segal on March 10, 2019, 10:29:41 pm
This post describes the comparative results for the two profiles (DGN10 vs XRite 2371), both for dE 76 and dE2000 and these four results for the 24 and 48 patch sets, making for a total of 8 comparisons, showing for each Average dE and its Standard Deviation (St. Dev). Hence, total 16 observations. To recall: the lower the dE and the lower the St. Dev, the more desirable the outcome. I also provide insight into Black and White tonality and neutrality.
Firstly, looking at the profile itself, the gamut volume is outstanding, at 997,526. This is the highest gamut volume I have seen with any printer/paper profile I’ve measured or created. I’ve come close with the XRite 2371 target at about 987,000 for the same printer/paper, so pretty close, but Doug’s N10 beats it by a tad. The profile shape is also very satisfactory (Figure 0, 2D representation).
Secondly, the comparative results (Figure 1):No complaints for any of it. For the most part, within each of the dE metrics, differences are on the whole quite trivial. Between dE76 and dE2000, all the dE 2000 results are lower than their comparative dE 76 results. This is normal and expected. I'll let the numbers speak for themselves.
Figures 2 and 3 show the results for the 14 neutral patches of the 48 patch set. I know the file values of these patches are completely neutral because I constructed them that way. Likewise I know the Luminance values in the target file are accurate as intended. The two questions answered in these graphs are (1) How close are the printed Luminance (L*) values to the file Luminance values, and (2) how neutral are they (or do they print with any hue bias, be it Lab a* or b* different from zero)? In the upper portion of the graph, the closer the red line (printed values) adheres to the black line (file values), the closer the Luminance values read from the print cohere with their file reference values. As you can see, the lines are almost convergent for both profiles. In the lower portion of the graphs, the closer the bars to zero (hugging the X axis), the less the hue bias. As you can see, all of them are well below 1 Lab value above or below zero, so much so, that taken together no one pair of a*,b* outcomes will likely be perceptible as non-neutral (this graph is not a dE calculation but shows the ingredients thereof - it is meant to indicate basic arithmetic departures from neutral).
Post by: MHMG on March 10, 2019, 11:34:04 pm
Imagine you have two neighboring tonal elements in a digital image, both with same LAB a* and b* values (as in a perfectly neutral B&W digital file) but one tonal element is 2.0 L* units higher in lightness value (ie., 2 dE different from the neighboring element). So, there's a contrast difference denoted by 2 dE between the two tonal values, nothing more nothing less, but this is definitely a noticeable contrast gradient in the image as seen by the typical human observer. Now consider that a print with these two tone values gets made where the profile causes the darker tone to lighten by 0.5 dE and the lighter neighboring tone to darken by 0.5 dE. You just lost 50% of the visual contrast between the two areas of interest in the printed image, but the color difference measurements (ie. dE or dE2000) indicate that both areas are below a just noticeable difference of 1dE. According to conventional dE wisdom, you shouldn't notice a problem when one patch or another is altered by less then 1 dE unit, but you will in this instance, because you are not comparing it to its former self, you are comparing it to the near neighboring value in the image. The initial L* value differences between the two scene elements denote a real visual contrast difference between the neighboring scene elements. They just got cut in half in the printed output when the darker value rises by .5dE and the lighter one gets darker by .5 dE. The human observer does indeed notice this, but dE measurements reported on a patch by patch level fail to reveal the problem.
Just sayin... A metric which tracks visual contrast between near neighbor elements in an image (dE and its various flavors do not) is required to put this print quality narrative into perspective. I developed a metric that does this and more. I call it the I* metric (https://www.aardenburg-imaging.com/i-metric/). I encourage others to try!
cheers,
mark
http://www.aardenburg-imaging.com
Post by: Mark D Segal on March 11, 2019, 08:58:12 am
I appreciate your arguments about the dE metric, which I have read in your articles quite some years ago, and I agree with the idea of examining differences between neighbouring values in relation to each other for understanding the impact of such differences on human visual perception. Your example above explains it pretty well.
What concerns me though are matters of relevance and context. That part of the international imaging industry and its technical committees most concerned with process control and print accuracy still, for the most part, rely on dE metrics of one flavour or another. I believe the reason rests on the objective being pursued, which is the consistency and/or accuracy with which a machine lays down a file value on paper. That is all I'm looking at in an exercise such as the one I reported above. When we are sitting in front of our monitors making prints, the front end of our work is to edit the photo to taste under softproof. I believe this is where the most important perceptual considerations need to occur. We're looking at a photo as a unified experience with all the varying contrasts and complements of tones and hues and how they relate to each other well before we press Print. Once we have all that set the way we like it, the file values or the meta data are established accordingly and we are then depending on the print pipeline (what I like to call "PPP" - printer, profile, paper) to lay down as close as possible what the file numbers or metadata say. To my mind, that means, for example, if we have set a particular small area of the photo to L*a*b* 20/0/0, and the nearest neighbour to 22/0/0, I want 20/0/0 and 22/0/0 to be laid on paper and measured as such (or close thereto as possible). This doesn't call forth a whole battery of perceptual arithmetic - in fact dE(76) is probably the most relevant metric bespoke to this purpose (but I provided dE2000 results as well because this is the current flavour many are using). The most important perceptual stuff should happen upstream of the PPP pipeline.
I've thought long and hard about which of these metrics make the most sense for the particular context I am working in, without prejudice to the merits of I* for other purposes. And by the way, while on the subject of I* - you are inviting us to try it - that's fine, but have you issued the tools and detailed instructions to make it more tractable to do so? Neither of the two papers I see on your website do that, however well laid-out the principles and illustrations are. Have I missed this implementation material somewhere?
Post by: dehnhaide on March 11, 2019, 09:11:41 am
I've thought long and hard about which of these metrics make the most sense for the particular context I am working in, without prejudice to the merits of I* for other purposes. And by the way, while on the subject of I* - you are inviting us to try it - that's fine, but have you issued the tools and detailed instructions to make it more tractable to do so? Neither of the two papers I see on your website do that, however well laid-out the principles and illustrations are. Have I missed this implementation material somewhere?
I must admit that reading your "duel" is more than enticing, my bread and butter these days. And I thank all of you yet again for your passionate comments.
I have read Mark's l* article and argumentation and indeed I feel like he's keeping me high up in the sky until I wanna try that story myself: learn by practice and share that gospel and preach it to myself and the others. But the lack of practical guidance kinda puts me off. I might be wrong but even in perceptual assessment practice, still, make perfect.
Sent from my iPhone using Tapatalk
Post by: Doug Gray on March 11, 2019, 12:12:10 pm
I've thought long and hard about which of these metrics make the most sense for the particular context I am working in, without prejudice to the merits of I* for other purposes. And by the way, while on the subject of I* - you are inviting us to try it - that's fine, but have you issued the tools and detailed instructions to make it more tractable to do so? Neither of the two papers I see on your website do that, however well laid-out the principles and illustrations are. Have I missed this implementation material somewhere?
I'll third that. I also have problems with dE00. It understates the effects of small changes in areas of low luminance. Especially in large areas as dE00 is designed for visual sensitivity (JND)of two patches surrounded by L50.
Post by: Doug Gray on March 11, 2019, 12:18:42 pm
Excellent results with the 2.3k patch set. As for the gamut size differences, I consider that irrelevant as it occurs at the outer boundary. The 1% difference should be looked at as a .2% difference in reality since the outer boundary differences would be visually strongly attenuated by the dE00 formula. Also, it's attenuated by an additional factor of 2 due to geometrical aspects (area increases twice as fast as linear dimensions).
The neutral patch results are interesting. One of the thing's I've noticed is that deviations along a* are much more sensitive in dE00 than b* while L* deviations are less sensitive. But L* and a* effects are magnified more than dE00 suggests in areas of low luminance as MHMG discusses from the perspective of I*.
Edit: a* and b* were reversed. a* is about 50% more sensitive than b*.
Post by: Mark D Segal on March 11, 2019, 12:59:50 pm
Mark Segal,
Excellent results with the 2.3k patch set. As for the gamut size differences, I consider that irrelevant as it occurs at the outer boundary. ...........
Yes, sure. Not only that, a high proportion of our photos don't need gamuts of those sizes in the first place, but it depends also on the gamut shape relative to the more saturated colours in the photo.
Post by: Doug Gray on March 11, 2019, 01:11:20 pm
Yes, sure. Not only that, a high proportion of our photos don't need gamuts of those sizes in the first place, but it depends also on the gamut shape relative to the more saturated colours in the photo.Yep, that too. As I've often said, I rarely look at gamut size except for looking at different printer's capabilities and even then it's less significant than it appears.
Post by: Doug Gray on March 13, 2019, 03:26:12 pm
There are two issues:
The most significant is that dE00 is much more sensitive to Lab changes along the neutral axis where it is sometimes magnified than in colors with any significant degree of saturation where it's attenuated. This causes average dE00 on neutrals to exceed average dE00 on color patches. The opposite is observed with delta E 1976 (dE76) where neutrals have less variance than color patches.
The other issue is variations in Lab on when the same color is printed in multiple places. The basic problem is that the variation between the same color patches printed in different locations are nearly as large as how well the profiles. To mitigate the matter effect and bring out variations due to the profile itself I have made a known color patch set that is repeated 5 times and randomized.
The patch set contains 2 collections of Lab values. The larger one, with 149 patches, contains evenly distributed colors that are in gamut for glossy type papers on all three of my printers. The smaller one contains 42 evenly spaced L* neutrals. This makes it easy to compare things using the exact same set of data for any of my printers and glossy/luster paper. And it provides a reasonably broad set of color patches that are independent, which is critical, of the set used to create the profile.
This creates a single, letter size iSis page which is scanned. Each of the 5 duplicates are averaged to reduce the variations described above. This reduces same patch variation by about 60% and makes it easier to see the intrinsic profile performance.
Here we compare two profiles. The first is the default, US letter size 957 patch set profile. The second is also a 957 patch profile. It's created by using the N=8, packed grid with added tracked neutrals. The tracked neutrals were created by filling the remaining 957-(512+343) with RGB values derived from evenly spaced L* values. Unlike the added neutrals in the posted collections, these RGB values differ from each other as they are based on what the default profile calculates is needed to produce neutrals using Rel. Col. This makes a printer/paper tailored patch set that does an excellent job reducing dE00 on neutrals.
Here's the results of the 149 color and 37 neutral, in-gamut, patches for the default profile:
Profile: iSis Default
Mean dE00, Color patches: 0.31 Worst 10%: 0.59
Mean dE00, Neutral patches: 0.54 Worst 10%: 1.69
Note how the dE00 is worse on the neutrals. This is unlike the dE76 which is simple and still commonly used. Here's the dE76 results:
Profile: iSis Default
Mean dE76, Color patches: 0.50 Worst 10%: 0.98
Mean dE76, Neutral patches: 0.51 Worst 10%: 1.44
And here's the results from the packed grid with tracking profile. The color patches are slightly better but the neutrals are much better.
Profile: N8 and Adapted Neutrals
Mean dE00, Color patches: 0.28 Worst 10%: 0.54
Mean dE00, Neutral patches: 0.27 Worst 10%: 0.54
For comparison using delta E 1976
Profile: N8 and Adapted Neutrals
Mean dE76, Color patches: 0.46 Worst 10%: 0.88
Mean dE76, Neutral patches: 0.27 Worst 10%: 0.62
Overall, pretty good indication that the packed grid combined with a paper/printer specific patch set, yields significant improvements. Especially in the neutrals.
Attached chart is the dE00 histogram of the 37 L* neutral patches for the default profile and the packed grid profile /w tracked neutrals. Both on 957 patch iSis letter size charts
Post by: rasworth on March 13, 2019, 04:36:22 pm
I don't understand "The other issue is variations in Lab on when the same color is printed in multiple places."
Is this a variation for horizontal location, or print to print, or what?
Richard Southworth
Post by: dehnhaide on March 13, 2019, 05:43:57 pm
Unlike the added neutrals in the posted collections, these RGB values differ from each other as they are based on what the default profile calculates is needed to produce neutrals using Rel. Col. This makes a printer/paper tailored patch set that does an excellent job reducing dE00 on neutrals.
Dear Doug,
Indeed the measured improvement in dE00 for the neutrals is outstanding but I don't get it how should I mofifuythe neutral values in your publish sets to accomplish the same. Care to explain, or maybe attach a modified say N8 for this purpose?
Post by: Doug Gray on March 13, 2019, 06:40:09 pm
Doug,
I don't understand "The other issue is variations in Lab on when the same color is printed in multiple places."
Is this a variation for horizontal location, or print to print, or what?
Richard Southworth
Both. The relative affect of each varies with printers. And patches can have varied readings just reading the same ones a second time though those are tiny. The bigger one is the horizontal location a patch is printed. It's affected by tracking head height variations which depends on the physical design. There are also warm up changes, and how long it's been since an image was last printed. Also, over a longer term, there are bigger changes as the ink volume decreases and it gradually gets more dense as the as fraction of the solvent slowly evaporates. For testing profiles my protocol is to duplicate and throw away the first print.
Easy, if tedious, way to test it is just duplicate color patches in the same or random locations and measure them.
Post by: Doug Gray on March 13, 2019, 06:46:43 pm
Dear Doug,
Indeed the measured improvement in dE00 for the neutrals is outstanding but I don't get it how should I mofifuythe neutral values in your publish sets to accomplish the same. Care to explain, or maybe attach a modified say N8 for this purpose?
Unfortunately, this is unique to each printer/paper type and somewhat hard. It also requires an existing, initial profile. One approach is to create a set of neutral Lab patches in Photoshop. Then convert to printer device space. Add those RGB values at the end of a CGATs file. Really easy to do in Matlab. I wouldn't be surprised if Argyll has a tool to convert a Lab CGATs file to RGB in device space.
Post by: rasworth on March 13, 2019, 07:29:50 pm
Not intending to divert your thread, I believe this is relevant.
I dissected an i1Profiler 915 auto generated patch target input file. In addition to the 9 X 9 X 9 RGB color triplets there were 20 R=G=B neutrals, and 60 near neutrals. The 60 RGB near neutrals were in sets of three, each entry with one of the three channels slightly different than the other two. I attached an excerpt from the patch generation file for clarity.
I assume the intent is to gain accuracy for the profile neutrals. What is your opinion as to the value of this technique?
Richard Southworth
Post by: GWGill on March 13, 2019, 08:21:14 pm
I wouldn't be surprised if Argyll has a tool to convert a Lab CGATs file to RGB in device space.Currently there is no automatic way, although it is possible using icclu and some manual editing. I have added a new option to targen for the next release of ArgyllCMS that allows generation of neutral axis wedge steps using the pre-conditioning profile, which will make this approach much more convenient. (Of course the -N parameter already allows control of the neutral axis patch density for the full spread patches generation, but generating patches right on the neutral axis may be a more efficient supplement.)
Post by: Doug Gray on March 13, 2019, 09:19:23 pm
Currently there is no automatic way, although it is possible using icclu and some manual editing. I have added a new option to targen for the next release of ArgyllCMS that allows generation of neutral axis wedge steps using the pre-conditioning profile, which will make this approach much more convenient. (Of course the -N parameter already allows control of the neutral axis patch density for the full spread patches generation, but generating patches right on the neutral axis may be a more efficient supplement.)
Graeme,
I've also tried patch sets with narrow spreads around the neutral axis. It takes more patches and the results weren't better so I reverted to just a simple set of neutral axis values as determined by the BtoA1 tables. It worked quite well so I halted some experiments along these lines.
There is one area where the application may be determinative. Rel. Col. v Abs. Col. Selecting either the Rel. Col neutral axis which tracks the paper white, or selecting the D50 neutral axis which forces a*=b*=0 might be application specific. With M2 profiles these are close enough that I don't think there is any practical difference but M0/1 with high OBAs might be another matter.
In terms of what folks can do w/o Matlab, I believe PatchTool has the ability to run a CGATs file through the BtoA1 profile LUTs and create scannable I1Pro charts. It also formats and creates printable charts from fractional (16 bit) CGATs files. It's not what I use so I'm unable to provide a recipe but it looks like it would work.
Post by: Doug Gray on March 14, 2019, 01:12:48 am
Doug,
Not intending to divert your thread, I believe this is relevant.
I dissected an i1Profiler 915 auto generated patch target input file. In addition to the 9 X 9 X 9 RGB color triplets there were 20 R=G=B neutrals, and 60 near neutrals. The 60 RGB near neutrals were in sets of three, each entry with one of the three channels slightly different than the other two. I attached an excerpt from the patch generation file for clarity.
I assume the intent is to gain accuracy for the profile neutrals. What is your opinion as to the value of this technique?
Richard Southworth
It's interesting. The neutrals help with the dE00 sensitivity. At least as long as the printer doesn't deviate too far from the actual neutral in device space. I used a smaller space between neutrals because the 9800 is very lumpy, the Pro1000 has larger deviations but is less lumpy. Using more neutrals helped for both of these. However, using RGB "tracking neutrals" from the BtoA1 profile tables produces patches that are close to actual neutral and hence interpolation errors are less.
As for the 60. It has an interesting pattern of increasing the spacing but maintaining a fairly constant ratio of about 20% for the larger value as it walks up. Interesting concept. At the larger values it's not going to be very effective for getting close dE00 tracking along the neutrals. But it might work well at low L*.
Post by: Alan Goldhammer on March 14, 2019, 07:47:51 am
Currently there is no automatic way, although it is possible using icclu and some manual editing. I have added a new option to targen for the next release of ArgyllCMS that allows generation of neutral axis wedge steps using the pre-conditioning profile, which will make this approach much more convenient. (Of course the -N parameter already allows control of the neutral axis patch density for the full spread patches generation, but generating patches right on the neutral axis may be a more efficient supplement.)I look forward to trying out the new release! You also describe my workflow. I use a 924 patch set for the original profile (two letter size sheets) and then use that as the pre-condition for the second one with the -N parameter along with an extra set of B/W patches as well. The second patch set is 1848 (four letter size sheets). I have not done the exhaustive type of study of the profiles (Epson 3880) that Doug has for his Canon printer.
Post by: Alan Goldhammer on March 18, 2019, 12:20:56 pm
I agree that a printed, continuous gradient is a good way to view smoothness and that's where eyeballs are much better than spectros. The latter is excellent for determining accuracy but not so good at smoothness unless you scan tens of thousands of patches. And one can enhance the latter by scanning or photographing the gradient and increasing contrast. How far contrast can be increased and still retain visual smoothness (noise aside), would be a good metric. At least for that particular gradient. Analyzing the smoothness of the entire gamut is a very hard problem.When you print out your neutral patch set for profile analysis, what settings do you use? I've finally got my i1Pro set up with Win10 and want to look at the Moab Entrada profile that I created using a two step process with Argyll. I have a 100 step B/W generated patch set that is randomized. I don't want to go beyond this right now.
Post by: Doug Gray on March 18, 2019, 03:18:16 pm
When you print out your neutral patch set for profile analysis, what settings do you use? I've finally got my i1Pro set up with Win10 and want to look at the Moab Entrada profile that I created using a two step process with Argyll. I have a 100 step B/W generated patch set that is randomized. I don't want to go beyond this right now.
My current profile validation reference patch set consists of 149 color patches that are in gamut and spread evenly across all printer glossy gamuts in Lab space as well as a 42 patch set of neutrals evenly spread from L*=0 to 100. OOG neutral patches, which occur near L=0 and 100, are not counted in the statistics. Each of these 191 patch sets is duplicated 5 times and randomized to fit on a single iSis page. The duplication provides more precise info and cuts the error related to printing in different locations by about 60% while not changing error related to the profile accuracy. It also provides independent info on the error associated with printing the same color in different locations.
I have a bunch of automated scripts that suss out the info of interest to compare printers/paper. Generally the 9500 II is less accurate and varies more than the 9800 and Pro1000. The 9800 produces the most repeatable colors printed in different locations but profile accuracy isn't as good with as gradients are lumpier over short color gradients. The Pro1000 produces the best overall color accuracy and slightly smoother neutrals. However, both the 9800 and Pro1000 are fairly close to each other overall. The principal traits of the Pro1000 are larger gamut and lower L* mins.
Post by: Alan Goldhammer on March 18, 2019, 04:00:23 pm
My current profile validation reference patch set consists of 149 color patches that are in gamut and spread evenly across all printer glossy gamuts in Lab space as well as a 42 patch set of neutrals evenly spread from L*=0 to 100. OOG neutral patches, which occur near L=0 and 100, are not counted in the statistics. Each of these 191 patch sets is duplicated 5 times and randomized to fit on a single iSis page. The duplication provides more precise info and cuts the error related to printing in different locations by about 60% while not changing error related to the profile accuracy. It also provides independent info on the error associated with printing the same color in different locations.Thanks, I don't think I can create a small color patch set from your parameters using Argyll though I think Graeme is coming up with a newer version that might be able to do that. I was more interested in you Photoshop setting for printing regarding the rendering intent or whether that makes a difference at all.
Post by: Doug Gray on March 18, 2019, 04:55:46 pm
Thanks, I don't think I can create a small color patch set from your parameters using Argyll though I think Graeme is coming up with a newer version that might be able to do that. I was more interested in you Photoshop setting for printing regarding the rendering intent or whether that makes a difference at all.
I believe Graeme mentioned a technique that would work with the current Argyll programs but would require adding in the additional patches and editing a CGATs file. Makes sense to me but the Matlab tools I have pretty much automate things so I haven't looked at it.
I print 16 bit tiff verification charts using Abs. Col. intent selecting the profile being tested. I print charts used to create profiles using the so called "null profile trick" where I assign an untagged chart with an arbitrary printer profile. This is then printed using the same profile. The actual printer profile used makes no difference. In this case only, rendering intent also has no effect. Photoshop sees this as the RGB values already being in device space so prints them directly ignoring rendering intent. This technique has always worked in Windows with all Photoshop versions since they discontinued the ability to print w/o color management. Canon's plugin also allows printing w/o color management producing identical results but is limited to current canon printers and doesn't even work on the 9500 II.
Post by: samueljohnchia on March 22, 2019, 05:55:39 pm
I want to make the final profiles in Argyll to take advantage of the luminance-preserving gamut mapping which Graeme coded at my suggestions. I'm digressing a bit more now, but for anyone else interested in creating top-notch CMYK profiles, you'll be a little frustrated with Argyll as the black generation controls, while more sophisticated than i1Profiler in general, don't work as expected. It's a bit buggy in its behaviour and the black ink curve is resistant to being shaped in certain important ways, wanting to always have a convex shoulder at the shadow turning point. Another weird bug is sometimes you see is one gets 100% for K ink in perceptual rendering at dmax but only say ~98% or less in Relative Colorimetric rendering. A more serious issue is with how Argyll computes the CMYK ink combinations. The profile (observed in ColorThink Pro's neutral rendering curves) nearly reaches it's total ink limit at L*10, then reduces the ink limit as we proceed towards L*0 before gradually building up to the total ink limit again. This results in crazy looking ink curves. But actual printed result looks OK for some reason! :o Nonetheless this behaviour is annoying and certainly not beautiful. At normal profile table resolutions, Argyll profiles develop strongly visible Mach banding (as described by Graeme), while i1Profiler and another software I tried is able to render gradients smoothly despite the LUT resolution being smaller. I do know about the suggestion to use device-link profiling for far more precise conversions. But to be brutally honest, the differences in precision are not really visible in conversions of regular photographs for printing, and thus not worth the trouble (same goes for smart mapping workflows in Argyll) apart from Argyll's own buggy profiles. But when some "magic" combination(s) of commands produce(s) a beautifully rendering profile from Argyll, I must say it does work out quite well.
Post by: Doug Gray on March 22, 2019, 10:30:04 pm
Hi Doug, I was just wondering how much of the thinking behind creating optimal patch sets for RGB printer profiling can be applied to CMYK profiling. I have been working with a company who has an HP Indigo printer to make some prototype books of late and will be going to offset to make the full run soon. At the moment I've resorted to i1Profiler's Smart Generator targets rather than customizing my own. I'm wondering how one would figure out an efficient way to sample the printer's CMYK space.I don't see a way to do this. With CYMK. let alone more ink versions, there is an infinite set of CYMK values that create the same Lab color. With "rgb" printers the algos are fixed so a device rgb value corresponds to only one Lab color and vice versa for in gamut Lab values.
Quote
Exactly.
Especially how to sample the neutral spine adequately. Its not clear what CMYK values will result in a neutral-ish output. And certainly many combinations of CMYK will.
Quote
Perceptual mapping is undefined but Rel. Col. is. It should maintain the CIExy values of the media white point down to the darkest color achievable with the same CIExy. Requesting a darker color than that is often mapped to a lower L* but with a shift in CIExy. But this is out of gamut mapping and is also undefined by the ICC.
There is of course more mind-boggling options about black generation and total ink limits etc. I have tried the ArgyllCMS method of using a "pre-conditioning" profile to assist targen in more efficiently exploring the CMYK printer space. Unfortunately it crashes all the time when attempting larger patch sets around 3000 or so, and when I look at the sampling points in 3D graphing software, ironically they look less equidistant in printer space than i1Profiler's auto generated targets. Most unexpected since for RGB profiling, Argyll's target certainly is better than i1Profiler at this. Grame has on several occasions pointed out this advantage of Argyll to me, in fact also mentioned that Argyll's profile building math is more optimised for equidistant sampling exploration of printer space rather than the (i1Profiler) method of equidistant orthogonally in (arbitrary) RGB cube space.
I want to make the final profiles in Argyll to take advantage of the luminance-preserving gamut mapping which Graeme coded at my suggestions. I'm digressing a bit more now, but for anyone else interested in creating top-notch CMYK profiles, you'll be a little frustrated with Argyll as the black generation controls, while more sophisticated than i1Profiler in general, don't work as expected. It's a bit buggy in its behaviour and the black ink curve is resistant to being shaped in certain important ways, wanting to always have a convex shoulder at the shadow turning point. Another weird bug is sometimes you see is one gets 100% for K ink in perceptual rendering at dmax but only say ~98% or less in Relative Colorimetric rendering.
Quote
A more serious issue is with how Argyll computes the CMYK ink combinations. The profile (observed in ColorThink Pro's neutral rendering curves) nearly reaches it's total ink limit at L*10, then reduces the ink limit as we proceed towards L*0 before gradually building up to the total ink limit again. This results in crazy looking ink curves. But actual printed result looks OK for some reason! :o Nonetheless this behaviour is annoying and certainly not beautiful. At normal profile table resolutions, Argyll profiles develop strongly visible Mach banding (as described by Graeme), while i1Profiler and another software I tried is able to render gradients smoothly despite the LUT resolution being smaller. I do know about the suggestion to use device-link profiling for far more precise conversions. But to be brutally honest, the differences in precision are not really visible in conversions of regular photographs for printing, and thus not worth the trouble (same goes for smart mapping workflows in Argyll) apart from Argyll's own buggy profiles. But when some "magic" combination(s) of commands produce(s) a beautifully rendering profile from Argyll, I must say it does work out quite well.
I have zero experience with RIPs and CYMK so I'm just commenting from what's defined by the ICC and the intrinsic math involved.
Post by: samueljohnchia on March 23, 2019, 08:45:20 am
With "rgb" printers the algos are fixed so a device rgb value corresponds to only one Lab color and vice versa for in gamut Lab values.
I would like to point out that profile building software also do not produce random combinations of C,M,Y for the same black generation settings (fixed K). After black generation is decided upon, and total ink limits, the recipe for C,M,Y is consistent for each profiling software's programmed logic. I.e., you are not going to get a different profile each time you re-run the build. In this regard, it is not really different from how "rgb" printer drivers work.
This is giving me an idea of how to get it to work. One needs a preliminary CMYK profile built, with one's final choice of black generation settings. Convert a patch set of neutral and near-neutral RGB samples into CMYK printer space using this preliminary profile. Those CMYK combinations will then be sampling and bracketing the neutral spine of the printer when re-purposed as profiling target patches. Then combine with i1Profiler or Argyll's targen auto-generated target for CMYK.
What do you think about this method?
Quote
But this is out of gamut mapping and is also undefined by the ICC.
My main concern is with out of gamut mapping. It is undefined by the ICC and it is also the greatest source of grief for me. No profiling solution available right now exists which does the right kind of mapping. ArgyllCMS comes closest to a 20 plus year old software which I used as inspiration to advise and suggest to Graeme a much better way of performing gamut mapping. The mapping in Argyll is still not ideal yet, but it is all I have now given that I've got no coding knowledge.
Post by: Doug Gray on March 23, 2019, 10:57:43 am
I would like to point out that profile building software also do not produce random combinations of C,M,Y for the same black generation settings (fixed K). After black generation is decided upon, and total ink limits, the recipe for C,M,Y is consistent for each profiling software's programmed logic. I.e., you are not going to get a different profile each time you re-run the build. In this regard, it is not really different from how "rgb" printer drivers work.Seems like a good approach. Certainly worth a shot.
This is giving me an idea of how to get it to work. One needs a preliminary CMYK profile built, with one's final choice of black generation settings. Convert a patch set of neutral and near-neutral RGB samples into CMYK printer space using this preliminary profile. Those CMYK combinations will then be sampling and bracketing the neutral spine of the printer when re-purposed as profiling target patches. Then combine with i1Profiler or Argyll's targen auto-generated target for CMYK.
What do you think about this method?
Quote
I don't often encounter materially OOG images but the idea of a luminance preserving approach seems good. If I needed to do that I'd approach it from the perspective of preprocessing the image to map each OOG pixel into the closest in-gamut value that preserves L* as well as hue. The problem, of course, is that this will create banding at the gamut boundary. OTOH, letting the hue vary while fixing L* is problematic too. Something has to give. One could create a transitioning region to prevent banding but this would reduce the saturation near gamut edges. In effect one would be creating a Perceptual mapping prior to color managed colorimetric printing. I also think this would be best done in the profiling s/w. Since I have little need in this area I haven't looked at Graeme's code. Graeme certainly would have the best insight on this.
My main concern is with out of gamut mapping. It is undefined by the ICC and it is also the greatest source of grief for me. No profiling solution available right now exists which does the right kind of mapping. ArgyllCMS comes closest to a 20 plus year old software which I used as inspiration to advise and suggest to Graeme a much better way of performing gamut mapping. The mapping in Argyll is still not ideal yet, but it is all I have now given that I've got no coding knowledge.
Post by: Doug Gray on March 23, 2019, 04:06:59 pm
After some experimentation, I've found the best results on the neutrals, which is critical for B&W using color profiles, is just a large number of tracking, actual neutrals, rather than staggered, near neutrals when there is a good, existing profile. When there isn't, such as making the first profile of a new paper, staggered, near neutrals over device space produces the best profiles. So the program will allow for both of these.
Also, I've found that the Pro1000 printer colors shift slightly after printing the first page and subsequent pages are much closer. Thus, my profiling (and testing) process now includes printing duplicates of the first page and discarding the initial print. This warm up effect is, surprisingly, a source of error that becomes quite significant with patch sizes over about 1K where the effect becomes larger than the improvement in accuracy from more patches.
Post by: Doug Gray on March 25, 2019, 01:53:04 am
The second mode uses a profile created from the first set to generate patch sets with tracking neutrals and near neutrals. This set can also use a programmable, larger number of packed grids for more accurate color rendition. I've also added options to duplicate and scramble patches to reduce the intrinsic error associated with printing the same patch in different locations.
I'm going to test this on the 9800 as well since it's rather lumpy on the neutral axis and I have extensive prior work optimizing its neutrals. I'm optimistic based on current P1000 tracking results.
Rather interesting coding this in C++. Debugging is a bit harder but the tools have really progressed. It requires a bit of a different mindset with a compiled program v Matlab which is interpreted and highly interactive.
Post by: Alan Goldhammer on March 25, 2019, 08:36:45 am
Have an operational program. It has two main modes. Initial mode is generating a patch set that has packed grids as well as near neutrals which does a pretty decent job of printing along the neutral axis.How many patches are in the set? Can it run on Argyll for profile target generation and profile construction?
The second mode uses a profile created from the first set to generate patch sets with tracking neutrals and near neutrals. This set can also use a programmable, larger number of packed grids for more accurate color rendition. I've also added options to duplicate and scramble patches to reduce the intrinsic error associated with printing the same patch in different locations.
Post by: vikcious on March 25, 2019, 08:38:06 am
Have an operational program. It has two main modes. Initial mode is generating a patch set that has packed grids as well as near neutrals which does a pretty decent job of printing along the neutral axis.
The second mode uses a profile created from the first set to generate patch sets with tracking neutrals and near neutrals. This set can also use a programmable, larger number of packed grids for more accurate color rendition. I've also added options to duplicate and scramble patches to reduce the intrinsic error associated with printing the same patch in different locations.
I'm going to test this on the 9800 as well since it's rather lumpy on the neutral axis and I have extensive prior work optimizing its neutrals. I'm optimistic based on current P1000 tracking results.
Rather interesting coding this in C++. Debugging is a bit harder but the tools have really progressed. It requires a bit of a different mindset with a compiled program v Matlab which is interpreted and highly interactive.
Good news! Really anxious about this. looking forward for a beta when available! ;)
Post by: Doug Gray on March 26, 2019, 01:33:36 am
Program ICCProfilePatches
There are two modes of operation. For those uncomfortable with commmand line programs it operates interactively. Just start the program in a command prompt window by typing ICCProfilePatches. It asks whether you want an initial, small patch set to characterize the neutral axis. If you already have a good profile you can skip this step. If not, answer (y)es and it will create a small, 232 patch set, with an outer grid of 5x5x5 and inner of 4x4x4 along with 50 added device rgb neutrals. It always prints the same RGB set so you only need to run this once to create a training profile. It's not very good for colorsbut has a pretty large set of near neutrals and will do a decent job printing B&W. A profile from this set is then used to generate near neutrals along the actual neutral curve in the next step. This only needs to be done once for any given paper type and printer. You can use the same profile in the future when making the larger tracking profile so long as the printer and paper type (MK v PK) are the same.
The other option is used to create the larger patch set with printer tracking neutrals. You have an option to enter the grid size plus the number of tracking, near neutrals. The grid size should be no less than 6 which will make under 400 patches with 70 neutrals. Adjust for your instrument. I use 8 or 10 for 1 or 2 iSis letter size pages. You will be asked to enter the profile to track to which can be from the first step or any other good profile you have for that printer and paper type.
That's it. You get CGATs RGB patches that can be loaded into I1Profiler or Argyll to create images and profiles. Feel free to experiment with different grid sizes and neutral counts to see what the totol patch size comes out to.
This is a copy of the two interactive modes I recently ran.
ICCProfilePatches
ICCProfilePatches V0.1
Create small patch set to generate profile for initial tracking? (y/n): y
RGB patches in outer grid:125
RGB patches in grids:189
RGB patches in grid after removal of neutrals:182
Using device neutrals
Added tracking neutrals:50
Total Patches:232
ICCProfilePatches
ICCProfilePatches V0.1
Create small patch set to generate profile for initial tracking? (y/n): n
How many values are on a major edge? (suggest between 6 to 12): 8
How many Tracking neutrals do you wish to insert? (suggest between 30 to 200): 60
Enter name of profile to create Rel. Col. neutral tracking from: SmallTargetPatches_25-03-19.icm
RGB patches in outer grid:512
RGB patches in grids:855
RGB patches in grid after removal of neutrals:842
Added tracking neutrals:60
Total Patches:902
For the brave, here's the options to runt the program directly from a command. The most useful feature not found in the interactive mode is the "-r Dups" which duplicates all the patches "Dups" times. It also randomizes the patches since the location of a patch can and does alter the actual Lab color when it's scanned. I see a standard deviation of about .20 dE00 from printing patches at different locations while just printing the same pattern on separate sheets can have half that variation. When the patches are randomized, an index file is printed out that can be loaded into Excel and used to sort measurement CGATs files to de-random,ize them. The duplicate patches are averaged by I1Profiler and provide slight improvements. As the grid size increases, the improvement from averaging increases. Particularly on good, smooth, printers like the Pro1000. Basically, there is a tradeoff between the variation of same patch color v linearity of the grid which improves as the grid points increase.
See readme file for options.
This was the result on my 9800 which is more "lumpy" than the Pro1000. It was quite good, only slightly worse on color patches than the 3828 patch set I was using on the 9800. However, it was slightly better on the neutrals. Getting the neutrals dE00 significantly below the color dE00s is rare since dE00
Profile: 9800 Two stage, optimized
Mean dE00, Color patches: 0.45 Worst 10%: 0.90
Mean dE00, Neutral patches: 0.38 Worst 10%: 0.63
Post by: Alan Goldhammer on March 26, 2019, 07:17:29 am
Post by: samueljohnchia on March 26, 2019, 10:04:37 am
Quote
Getting the neutrals dE00 significantly below the color dE00s is rare since dE00 exaggerates Lab differences on or near the neutrals while suppressing differences once the colors become even slightly saturated.
Doesn't this correlate better with human perception? Don't we tend to notice colour casts in neutrals more easily than in colours?
Post by: Doug Gray on March 26, 2019, 10:46:28 am
Hi Doug, thank you for sharing so generously. I hope to try this out for myself soon. Quick question first, how are you arriving at the near neutrals tracking sampling from the preliminary profile? This sounds kinda like the approach I mentioned with CMYK earlier. Just wondering.What the program does is create a series of evenly spaced steps in L* from the black point to 100 (in Re. Col.) with a* and b* set to 0. It's then runs these through the supplied profile to generate RGB values that correspond to this. Then the RGB values are adjusted slightly with each channel in turn increased a small amount while the other two are decreased a smaller amount. This produces staggered near neutrals much like what the I1Profiler's patch generator does with near neutrals except that these more closely track the actual neutrals.
Quote
Yes, you are quite correct. I was trying to say dE00 typically magnifies Lab differences and "exaggerates" implies it increases dE00 inappropriately. I'll fix that post.
Doesn't this correlate better with human perception? Don't we tend to notice colour casts in neutrals more easily than in colours?
Post by: rasworth on March 26, 2019, 11:24:15 am
I picked a very small color set and 50 neutrals to emphasize the neutrals treatment. I "trained" Doug's patch generator with an existing profile, generated the patch set, brought it into ColorThink Pro, attached the printer training profile, and graphed it in Lab space. The image clearly shows the new neutrals hugging the L axis in the training profile space, and the three sets of near neutrals now closer to the axis.
Richard Southworth
Added by edit - For clarity's sake, the next step would be to use the new patch set to generate another printer profile, taking advantage of the nearer neutrals.
Post by: rasworth on March 26, 2019, 11:37:08 am
Richard Southworth
Modified to insert more informative image.
Post by: Doug Gray on March 26, 2019, 12:26:06 pm
I just discovered a "feature" of Doug's program - if one gives it an invalid training profile the program reverts to untrained neutrals.
Richard Southworth
Yep. That's intentional and it prints out a comment that it couldn't find the icc profile. I felt it best to default to using device space neutrals rather than just abandoning things. Still makes a pretty good patch set with all the extra neutrals even if they aren't tracking since it couldn't find the reference profile.
Post by: rasworth on March 26, 2019, 12:42:43 pm
I invoked your executable directly, without first establishing a command window, and all worked fine except the program exited with no chance to read the final messages. I realize the proper method is to bring up a command window before any execution, but it is very convenient to double click the executable directly.
Richard Southworth
Post by: dehnhaide on March 26, 2019, 12:46:19 pm
You may want to insert a pause or two, to keep the program from exiting before messages are read.
I invoked your executable directly, without first establishing a command window, and all worked fine except the program exited with no chance to read the final messages. I realize the proper method is to bring up a command window before any execution, but it is very convenient to double click the executable directly.
Richard Southworth
Not exactly what would be expected as norm from command line executable. Otherwise it would mean that the Doug's executable would have to first call on the "cmd" and than execute it's code... That would be lame. 99% percent of Windows binaries that require interaction need be called from a cmd prompt.
Post by: rasworth on March 26, 2019, 12:59:21 pm
And I will wager 9 out of 10 who first try this program will start it the first time with no command window.
Richard Southworth
Post by: rasworth on March 26, 2019, 01:57:14 pm
Doing some boundary value checking - entering "0" for edge value works ok, gives only neutrals. However entering "1" gives an invalid rgb triplet for the first entry.
"2" is sort of a sweet spot, gives 0-0-0 and 255-255-255 plus the 6 fully saturated rgb values. It's the right setting for generating neutrals targets.
Ok, I'll quit trying to break your program, it's a significant contribution to the profile maker's toolset.
Richard Southworth
Post by: Doug Gray on March 26, 2019, 02:33:19 pm
Doug,
Doing some boundary value checking - entering "0" for edge value works ok, gives only neutrals. However entering "1" gives an invalid rgb triplet for the first entry.
"2" is sort of a sweet spot, gives 0-0-0 and 255-255-255 plus the 6 fully saturated rgb values. It's the right setting for generating neutrals targets.
Ok, I'll quit trying to break your program, it's a significant contribution to the profile maker's toolset.
Richard Southworth
Richard, These are useful suggestions and there is definitely is a use case for creating only staggered neutrals so people can add it to a good performing patch set they already have.
Thanks for exploring things and making these suggestions.
Post by: Doug Gray on March 26, 2019, 02:51:00 pm
Post by: rasworth on March 26, 2019, 03:49:44 pm
You're welcome, version 2 runs great! Thanks again for your contribution.
Richard Southworth
Post by: Alan Goldhammer on March 26, 2019, 05:49:28 pm
The program has been modified per Richard's suggestions. Thanks, Richard!Where is Version 2 posted?
Post by: Doug Gray on March 26, 2019, 05:58:25 pm
Where is Version 2 posted?I replaced the original zip file in reply #105.
Post by: JRSmit on March 27, 2019, 06:00:54 am
If i load the cgats file in a worksheet, and then do graph it shows the colors, but also transformed into a sRGB space.
This cannot be correct.
What is it i do wrong?
Post by: samueljohnchia on March 27, 2019, 08:20:20 am
Quote
What is it i do wrong?
When graphing the RGB patch values in CTP, you need to assign printer space to it (the printer profile) rather than srgb. Then they will be graphed as you expect.
Post by: samueljohnchia on March 27, 2019, 08:27:10 am
What the program does is create a series of evenly spaced steps in L* from the black point to 100 (in Re. Col.) with a* and b* set to 0. It's then runs these through the supplied profile to generate RGB values that correspond to this. Then the RGB values are adjusted slightly with each channel in turn increased a small amount while the other two are decreased a smaller amount. This produces staggered near neutrals much like what the I1Profiler's patch generator does with near neutrals except that these more closely track the actual neutrals.Yes, you are quite correct. I was trying to say dE00 typically magnifies Lab differences and "exaggerates" implies it increases dE00 inappropriately. I'll fix that post.
Thanks Doug for the insight into how it works and also confirming that we're thinking the same about dE00!
Post by: JRSmit on March 27, 2019, 10:13:47 am
When graphing the RGB patch values in CTP, you need to assign printer space to it (the printer profile) rather than srgb. Then they will be graphed as you expect.Hi samuel, the data set has no profile assigned whatsoever. Apparently it then defaults to sRGB. I want to see the matrix as Dough created it. This without any colorspace.
Post by: samueljohnchia on March 27, 2019, 11:53:19 am
Hi samuel, the data set has no profile assigned whatsoever. Apparently it then defaults to sRGB. I want to see the matrix as Dough created it. This without any colorspace.
Hi Jan, what you are asking for makes no sense. Without defining the colour space, RGB values (numbers alone) cannot be graphed in CTP. It is no different from say telling me how big a sheet of paper is in numbers but with no units.
What I think you want is to view the sampling positions of Doug's patch set in relation to printer space. So you need to load the RGB patch data in the Worksheet window, and then assign the preliminary printer profile to the list and graph it.
Post by: JRSmit on March 27, 2019, 12:34:21 pm
Hi Jan, what you are asking for makes no sense. Without defining the colour space, RGB values (numbers alone) cannot be graphed in CTP. It is no different from say telling me how big a sheet of paper is in numbers but with no units.Indeed, when in worksheet, i was able to plot in Lab. showed the matrix very nice.
What I think you want is to view the sampling positions of Doug's patch set in relation to printer space. So you need to load the RGB patch data in the Worksheet window, and then assign the preliminary printer profile to the list and graph it.
Post by: Doug Gray on March 27, 2019, 07:27:40 pm
It's also looking like setting the stagger value to 0 may be optimum for the final profile patch set.
Post by: JRSmit on March 28, 2019, 04:05:48 am
Post by: Doug Gray on March 28, 2019, 10:39:59 am
Dough, what do you mean with a tracking profile?
It's my term for a profile made with near neutrals patches that have been more closely aligned with the printer's neutral axis. A more precise description would be a profile made with neutrals that are tracked to the printer/paper. Profiles are at their most accurate when printing colors that are closest to the colors of the patches the profile was created with. Creating a profile that includes patches previously determined to be closer to the printer's actual colors (neutral colors in this case) improves accuracy. Because human vision is most sensitive for colors that are near the neutral axis, there is a benefit to focusing specifically on the neutral axis.
It can also produce the best results for B&W printing when using a standard RGB, profiled, workflow.
Post by: rasworth on March 28, 2019, 11:29:57 am
1. Ran the program with the outer grid set to 2 and number of neutrals at 256, no profile selected, creating a cgats file for i1Profiler target generation. The result fits on one letter size sheet, patches sized for an i1Pro. The first attachment is the target image.
2. Scanned the target into i1Profiler, generated the first pass profile. My i1Pro is a uv cut model, so the profile is categorized as M2.
3. Ran Doug's program, again with outer grid set to 2 and number of neutrals at 256, this time with the first pass profile selected, creating the second pass cgats file for a second i1Profiler target. Same dimensions, one letter size sheet, this time with the tracking neutrals. The second attachment contains two plots, the left hand size are the neutrals before tracking and the right hand side after tracking, both in the printer profile space.
4. Printed the second pass target, scanned into i1Profiler to create the second and final profile. Interestingly the six color patches were enough to create a "viable" full color profile, no test prints made but the soft proof in Photoshop looked reasonable, although the gamut was significantly reduced.
5. To test created 101 Lab neutrals, L = 0 to 100, a = b = 0, did an absolute rendering intent conversion thru the second pass profile, and printed in i1Profiler target form. After drying, scanned into i1Profiler, converted from spectrum data to a Lab list via the i1Profiler measurement output capability.
6. Finally compared the 101 scanned neutrals to the original list, third attachment shows the dE2000 results. The paper has a minimum L of about 5 and a maximum of 94, so I discarded the first six and last six measurements. The remaining 89 dE's average to .48, with 4 exceeding 1.00.
My conclusion is the combination of Doug's program, ColorThink Pro, and i1Profiler provides a quick means to generate a high quality B&W (R=G=B image, not grayscale) profile. My data is probably somewhat "incestuous" since I used the same instrument for both generating the profiles and testing, but I'm pleased with the results.
Richard Southworth
Added by edit - I believe my last statement is misleading, Doug's program and i1Profiler are all that is required to generate the profiles, ColorThink Pro comes into play for testing.
Post by: rasworth on March 28, 2019, 01:32:22 pm
"-p PageSize Set the max page patch count size, use with negative -n" has me stumped, please explain.
Richard Southworth
Post by: Doug Gray on March 28, 2019, 01:47:22 pm
Doug,
"-p PageSize Set the max page patch count size, use with negative -n" has me stumped, please explain.
Richard Southworth
It's an option to specify the max number of patches on a target page. The default is 957 since that's the max letter size count for an i1iSis. It's an option so that it can be changed to accord with what instrument/paper size one has.
Combining it with -n -1, to generate whatever number of neutrals is needed to fill out the page(s). However, it's a bit buggy and I will be shortly revising my program. Most of the changes are to allow scrambling and repeating the neutrals w/o repeating the color patches. There are also some minor other bugs I'm fixing in the tracking neutral generation.
After testing printers and different settings I should have a more optimized program later today. Nothing like printing independent colors/neutrals to improve things.
Nice graphs, BTW.
Post by: rasworth on March 28, 2019, 02:01:25 pm
Richard Southworth
Post by: Doug Gray on March 29, 2019, 02:47:49 am
Hopefully later Friday.
Post by: dehnhaide on March 29, 2019, 03:20:41 am
5. To test created 101 Lab neutrals, L = 0 to 100, a = b = 0, did an absolute rendering intent conversion thru the second pass profile, and printed in i1Profiler target form. After drying, scanned into i1Profiler, converted from spectrum data to a Lab list via the i1Profiler measurement output capability.
Hi Richard,
Thanks for sharing the workflow. I don't seem to understand how you did the "absolute rendering intent conversion thru the second pass profile". Can you clue me how this done, please! I suspect it's done in CTPbut not sure how!
Thanks
Post by: rasworth on March 29, 2019, 10:03:12 am
You're right, used CTP. I first generated 101 neutrals in Excel, dumped it out as a txt file and pasted a simple CGATS wrapper around it. Brought it into a CTP worksheet and did an absolute conversion thru the final BW printer profile (see first attachment).
I dumped out the RGB cgats file, extracted the numeric values back into Excel and rounded them to whole integer values, and pasted back into the original file, now labeled rounded. It's probably not necessary, but I'm not sure what i1Profiler does with decimal valued RGBs. I then brought the txt file into i1Profiler as a target patch file and printed it with the i1Profiler command with no color management (second attachment).
The process is quicker to do than describe, and allows one to use an instrument in scan mode, in my case an i1Pro, to read 101 test patches in a minute or so.
Richard Southworth
Post by: dehnhaide on March 29, 2019, 10:17:36 am
dehnaide,
You're right, used CTP. I first generated 101 neutrals in Excel, dumped it out as a txt file and pasted a simple CGATS wrapper around it. Brought it into a CTP worksheet and did an absolute conversion thru the final BW printer profile (see first attachment).
I dumped out the RGB cgats file, extracted the numeric values back into Excel and rounded them to whole integer values, and pasted back into the original file, now labeled rounded. It's probably not necessary, but I'm not sure what i1Profiler does with decimal valued RGBs. I then brought the txt file into i1Profiler as a target patch file and printed it with the i1Profiler command with no color management (second attachment).
The process is quicker to do than describe, and allows one to use an instrument in scan mode, in my case an i1Pro, to read 101 test patches in a minute or so.
Richard Southworth
Thanks Richard for the explanations. Much appreciated!
Post by: Doug Gray on March 29, 2019, 02:00:03 pm
You might find this of some value. It's a CGATs file in ProPhoto RGB of Lab values from 0 to 100 and can be loaded into I1Profiler. Then a target chart can be saved as a tif file and assigned the ProPhotoRGB profile. It can then be printed from Photoshop using standard color management in Abs. Col. I've included rounded as well as fractional CGATs. Do not use the fractional with I1Profiler since it can only make 8 bit tifs. The fractional file can be used with PatchTool which does create 16 bit tif files.
I've also included lab.txt and labr.txt for reference. The Labr.txt file contains the fractional Lab values that the rounded, 8 bit rgb file generates. These can differ by just under .4 dE though the average dE between the rounded and unrounded is .1. The printed output from I1Profiler should be compared to the Labr.rxt values.
Post by: dehnhaide on March 29, 2019, 02:18:33 pm
Richard et al,Thanks for sharing this Doug. Is there another way to print in 16bits TIFF aside from PatchTool? This is yet another paid tool and i1Profiler and CTP are enough of an investment for me.
You might find this of some value. It's a CGATs file in ProPhoto RGB of Lab values from 0 to 100 and can be loaded into I1Profiler. Then a target chart can be saved as a tif file and assigned the ProPhotoRGB profile. It can then be printed from Photoshop using standard color management in Abs. Col. I've included rounded as well as fractional CGATs. Do not use the fractional with I1Profiler since it can only make 8 bit tifs. The fractional file can be used with PatchTool which does create 16 bit tif files.
I've also included lab.txt and labr.txt for reference. The Labr.txt file contains the fractional Lab values that the rounded, 8 bit rgb file generates. These can differ by just under .4 dE though the average dE between the rounded and unrounded is .1. The printed output from I1Profiler should be compared to the Labr.rxt values.
Sent from my iPhone using Tapatalk
Post by: Doug Gray on March 29, 2019, 03:20:40 pm
Thanks for sharing this Doug. Is there another way to print in 16bits TIFF aside from PatchTool? This is yet another paid tool and i1Profiler and CTP are enough of an investment for me.
Not that I know of. I use Matlab (also not free) to create 16 bit tiffs for fractional precision and Photoshop to print them. Some here have Patchtool so I thought I'd include the fractional CGATs for them. The 8 bit ones do quite well. The 8 bit Lab file, which shows the actual Labs of the 8 bit ProPhoto is the closest workaround for 8 bit workflows.
Post by: Alan Goldhammer on March 29, 2019, 03:50:29 pm
Thanks for sharing this Doug. Is there another way to print in 16bits TIFF aside from PatchTool? This is yet another paid tool and i1Profiler and CTP are enough of an investment for me.ArgyllCMS which is free permits one to prepare 16 bit TIFF targets. There is switch in the printtarg utility that will do this for you.
Sent from my iPhone using Tapatalk
Post by: dehnhaide on March 29, 2019, 03:57:49 pm
ArgyllCMS which is free permits one to prepare 16 bit TIFF targets. There is switch in the printtarg utility that will do this for you.Yeah, I know that but wanted to still be able to print i1Profiler compatible targets to avoid going back and forth between ArgyllCMS and i1Profiler.
Post by: rasworth on March 30, 2019, 09:43:58 am
I suspect you've speculated along these lines, would be interested in knowing your thoughts.
The concept of an initial relatively simple printer profile to "train" neutrals to line up more closely with the L axis, seems to work very well in terms of the neutrals accuracy of the final profile. How can we apply this same approach to the color entries? It's a tougher problem, have to work in three dimensions with the L, a, and b axis's, but it would seem that we could "nudge" the rgb values nearer to the points in printer space where they line up with more usable L*a*b* values.
Richard Southworth
Post by: Doug Gray on March 30, 2019, 02:08:05 pm
I was investigating the linearity of low luminance (in an RGB sense, not L*) packed RGB patches and discovered that, at low values of RGBs, there is a greater need for more closely spaced grids.
This is even more critical for good rendering of large, dark areas because dE00 understates visual sensitivity of differences when partially adaptating to the darker image. Recall that dE00 best comports with visual sensitivity to differences between two colors surrounded by neutral gray. When the surrounding area is darker, visual differences magnify.
This is of great interest to me because of a goal of creating a special display environment where the print is illuminated with 2 to 4 times more light in order to create displayed prints that appear superluminous. That is, they appear to have portions that are brighter than is physically possible for a reflective print which can only reflect less than 100% of the light hitting it. Doing this involves tricks like creating a "white border" that is actually gray but appears white. This can be done by suspending the print away from anything that provides visual clues about the illumination.
Doing a good job of this illusion is a principal reason to work on dark color accuracy beyond normal dE metrics.
So I'm adding an option to halve the spacing on RGB values below some threshold. (60 seems to work well for the Pro 1000). I'm making the program in Matlab and will port the changes to C++ and post the new program.
Tracking neutrals works quite well so that option will be retained but the specific implementation may be affected when the smaller spaced, low RGB, grid is added. But I don't expect major differences.
Post by: rasworth on March 30, 2019, 02:23:32 pm
Good stuff, keep it up, and thanks for your efforts that benefit all of us.
Richard Southworth
Post by: Doug Gray on April 02, 2019, 12:32:49 am
I've added options to generate high density subgrids in low and high RGB values. This is mostly useful for generating more accurate patch sets for images with larger, darker areas.
I've attached a sample, agnostic patch set with a 10x10x10 main grid and 6x6x6, low RGB value high density grid. The high density grid is repeated because the elements are close and this reduces variation in patches.
These are the options for the command line operation but it also works in interactive mode..
Usage: ICCProfilePatches [-f] [-t] [-r repeats] [-p pagesize] [-m MGrid]\n "
[-s StaggeredNeutralDistance] [-n neutralCnt] [-i iccProfile]\n"
[-C ColorimetricMode] [-c creator] [-o outfile]\n";
-t Test run. Don't create output. Use to get statistics\n"
-f Emit fractional RGB values, ints are default\n"
-r Remove neutrals in grids (use when adding neutrals separately)\n"
-v Scramble Patches\n"
-m MajorGridDim Major Grid dimension\n"
-M repeatcount Major Grid Repeat Count\n"
-l LowGridDim Low Grid Dimension (scales and places half spacing packed grid at low RGB values)\n"
-L repeatcount Low Grid Repeat Count\n"
-h HighGridDim High Grid Dimension (scales and places half spacing packed grid at high RGB values)\n"
-H repeatcount High Grid Repeat Count\n"
-s staggerVal Stagger alternate RGB neutrals by this magnitude\n"
-n Counts Counts neutrals (device or colorimetric) are added\n"
-C Color mode 1 (Colorimetric) or 3 (Absolute Colorimetric)\n"
-o CGATsFilename The name of the CGATs file\n"
-i ICC profile Profile used to adapt neutrals to actuals\n"
-c creator Who dunnit\n";
Post by: dehnhaide on April 02, 2019, 12:55:39 am
Here's the latest (Ver .4). It's a 32 bit Windows program so works in both x64 and x86 Windows.
Thanks a lot dear Doug for your hard work and for sharing it with us.
Post by: rasworth on April 02, 2019, 09:52:13 am
Thanks. What's the rationale for the sub-grid option at high RGBs?
Richard Southworth
Post by: rasworth on April 02, 2019, 10:08:08 am
Quick run in interactive mode, 8 for major grid, 40 for neutrals, no training profile. Got more patches than I expected, seems to have added a sub-grid group by default?
Richard Southworth
Added by edit - OK, now I remember how the first version worked, n-1 sub-grid in interactive. Sorry to pull the trigger so quickly. However, I believe the program might be a little more useful if the sub-grid were not a default, i.e. would just generate straight matrix. Maybe I can get there by command line entering 0s for the sub-grids.
Post by: Doug Gray on April 02, 2019, 10:18:51 am
Doug,Generally of little value with my 9500II or Pro1000, but helps color accuracy in RGB values where the triplets are above 200,200,200 or so on the Epson 9800. The 9800 is a bit non-linear in this area. Also, for whatever reason, the 9800 maps higher RGB values to darker colors than the other two printers so increasing the grid density in this area reduces the distance between a requested color and the profile patches used to generate the profile interpolation tables. Still, it's barely noticeable on the 9800 and doesn't show up at all on my other printers. I also noticed that some of the default I1Profiler tables have a higher density in this area. It was a simple addition so I threw it in.
Thanks. What's the rationale for the sub-grid option at high RGBs?
Richard Southworth
One other note. The high density grid options are done at twice the main grid density. Duplicates in both the low and high RGB grids are removed from the main grid. Since the main grid is first in the CGATs file, it has gaps but the apparently missing values exist in either the low or high RGB density segments. There just is no point in duplicating values that overlap.
Post by: rasworth on April 02, 2019, 10:47:26 am
Richard Southworth
Post by: Doug Gray on April 02, 2019, 10:58:57 am
Doug,
Quick run in interactive mode, 8 for major grid, 40 for neutrals, no training profile. Got more patches than I expected, seems to have added a sub-grid group by default?
Richard Southworth
Added by edit - OK, now I remember how the first version worked, n-1 sub-grid in interactive. Sorry to pull the trigger so quickly. However, I believe the program might be a little more useful if the sub-grid were not a default, i.e. would just generate straight matrix. Maybe I can get there by command line entering 0s for the sub-grids.
Currently sub-grids are always generated. These prove to produce better profiles generally because of the closer spacing of RGB values to each other. For instance selecting the main group N at 8, with no additional neutrals produces 855 patches which is 8^3+7^3. It also produces 15 (8+7) neutral patches where as without the packing groups, there are only 8 neutrals. Even selecting a grid size of 6, produces 11 neutrals, which is 3 more than the 8 produced with only a main grid and no packing grid. And that makes 512 patches compared to the 341 that setting the grid size to 6 generates. Color accuracy is roughly the same but with less patches (341 v 512) and it has 3 more neutrals.
However, if one elects to add neutrals all these neutrals from the main grids are removed except for black and white patches in favor of the evenly spaced neutrals requested which are then added.
Post by: rasworth on April 02, 2019, 12:38:02 pm
I hate to admit, but I had to graph (attached) a 8 major/4 dark/0 neutrals set to understand. Your explanation is clear enough, I had trouble getting my head around the dark grid being a double frequency representation of both the outer and inner main grids. Neat.
Richard Southworth
Post by: Doug Gray on April 02, 2019, 01:34:17 pm
Not a big deal, but I did notice duplicate 0-0-0 entries, at the beginning of the neutral patches, for my interactive mode 8 major grid run.
Richard Southworth
Yes, I added white and black patches after removal of the neutrals before adding additional neutrals. This is because the staggering, using "-s" is in multiples of 4. I decided not to truncate as there may be reasons why someone wants the exact number of added neutrals. For instance to fill out a MxN chart page. This required the white patch. The black patch guarantees a device RGB of 0,0,0 which may not be the case using tracking profiles since they return RGB values closest to L*=0 and that may not be 0,0,0 - though it often is. I1P Profiling software requires both a black patch and white patch. As for the added neutral patches, there is no option to repeat those. It's a more effective to increase the count, say from 50 to 100 rather than double the same patches. This helps smooth the neutrals slightly as any given RGB value can be higher or lower than it should be based on the surrounding neutrals.
I meant to mention that the program also creates a file with the same name as the output CGATs file with "Index" prepended. This is just has one number per line indicating the original position of each RGB value. It can be used to unscramble, randomized patch sets when choosing the "-v" (for vary location) option. Just load it into Excel along with the measured CGATs file and sort against it.
Post by: Doug Gray on April 02, 2019, 01:46:26 pm
Doug,
I hate to admit, but I had to graph (attached) a 8 major/4 dark/0 neutrals set to understand. Your explanation is clear enough, I had trouble getting my head around the dark grid being a double frequency representation of both the outer and inner main grids. Neat.
Richard Southworth
Cool Richard,
Try 8 major/8 dark/0 which doubles the resolution in for values < 128,128,128. Brings out the density change even more dramatically.
Post by: rasworth on April 02, 2019, 01:46:57 pm
Everything you write makes a lot of sense, obviously well thought out.
I made a run at "-m 8 -l 15 -n 0" and another at "-m 15 -l 0 -n 0", both gave 6119 patches. Your removal strategy is working well.
Richard Southworth
Post by: rasworth on April 02, 2019, 01:59:17 pm
Richard Southworth
Post by: Doug Gray on April 02, 2019, 02:10:47 pm
8 major / 8 dark / 0 neutrals does emphasize the technique (see attached). I believe you stated earlier the benefit primarily occurs at rgb 64 and below?
Richard Southworth
Yes, at least that's what my testing on my 3 printers indicates. The darkest RGB values have about 30% more nonlinearity. The smaller distances is also why I suggest replication "-L 2" using them. And doesn't add that many more values, 64 more patches for "-L 2 -l 4" OTOH, having an i1iSis sort of warps my perspective.
Post by: Doug Gray on April 02, 2019, 02:24:15 pm
But it could! And it isn't even that hard! All one has to do is get the CGATs measurement file from a patch set with lots of neutrals that is printed using ABW. Zero out the a* and b* from the Lab values. Then add in enough fake colors to make I1Profiler happy. Make a regular profile.
Then just print B&W images selecting that profile together with ABW mode. Should work just fine in Photoshop.
I don't do much B&W printing but from my testing it is much smoother and works quite well. It's just that I1PRofiler doesn't directly support it.
If this works like I think it will, I'll add the feature to the program. Basically, one just prints a full B&W tone curve containing all 256 neutrals using ABW. The patches wo9uld be measured and the CGATs file read by the program which would then spit out another CGATs file that I1Profiler could read to generate a standard "color" profile but one that precisely maps image B&W values.
Post by: Alan Goldhammer on April 02, 2019, 02:51:44 pm
Post by: rasworth on April 02, 2019, 02:52:34 pm
I created a 1000 patch set (fills up 13"x19" for i1Pro) using -m 8 -l 4 -n 86 with neutrals training from previous profile. It seems to me your new version causes the near neutrals to hug the L* axis more closely (plot attached). Is this controlled by the -s parameter?
Richard Southworth
Post by: Doug Gray on April 02, 2019, 03:02:24 pm
Doug,
I created a 1000 patch set (fills up 13"x19" for i1Pro) using -m 8 -l 4 -n 86 with neutrals training from previous profile. It seems to me your new version causes the near neutrals to hug the L* axis more closely (plot attached). Is this controlled by the -s parameter?
Richard Southworth
Yes, I changed the default "-s" parameter to 2. You can modify it. If you set it to 0 it will follow the actual neutrals but the value of 2 gives it a small amount of variation which might help is some instances. If you examine the CGATs file, the full set of options used is stored in the description tag like this:
Opts:t0f0r0v0 m8M1l3L1h0H1s0n76C1 iprofilename A bit cryptic but each letter is followed by the value and the bools are either 1, if selected, or 0 if not.
Post by: Doug Gray on April 02, 2019, 03:06:58 pm
Doug, your ABW approach will only work with Windows OS as anyone on a MAC cannot use ABW profiles. This pathway was broken by Apple at least five years ago and I remember Eric Chan complaining about it at the time.
Well, ABW is a lonely orphan. OTOH, my code, while in portable C++, is only compiled for Windows. I could compile it for iOS but I'd have to buy an Apple. Eventually I'll post the source to GitHub and someone with iOS could compile it. That wouldn't solve the ABW of course but would make the patch generation program available to Apple users. :(
Or one could borrow someone's PC. :)
Post by: Doug Gray on April 02, 2019, 07:31:00 pm
On the 9800, I created a patch set for ABW using the idea mentioned above, saved the RGBLAB CGATs file with a full 256 measured device neutrals using ABW and I1Profiler happily made a profile.
What's even better is that, after printing the 256 patch set assigned to ProPhoto using the profile and selecting Abs. Col, all the in-gamut L*s variances were incredibly close to 0. Average dE 1976 was .11 dE00 was .09
Unreal. And even better. the DMax dropped. Min. L* went from 4.0 to 2.9.
I now have a great profile for printing using normal color management with ABW!
Post by: Alan Goldhammer on April 03, 2019, 11:27:32 am
Hot Dog!Excellent work. My 3880 bit the dust last week and I have a new Canon Pro-1000 sitting in the box as I need to wait for my neighbor to help get it up on the printer table. I'm interested toe explore how it handles B/W printing as I still do a fair amount of that and have scanned a lot of old Tri-Ex negatives.
On the 9800, I created a patch set for ABW using the idea mentioned above, saved the RGBLAB CGATs file with a full 256 measured device neutrals using ABW and I1Profiler happily made a profile.
What's even better is that, after printing the 256 patch set assigned to ProPhoto using the profile and selecting Abs. Col, all the in-gamut L*s variances were incredibly close to 0. Average dE 1976 was .11 dE00 was .09
Unreal. And even better. the DMax dropped. Min. L* went from 4.0 to 2.9.
I now have a great profile for printing using normal color management with ABW!
Post by: 1PhotoGuyinNM on April 04, 2019, 11:14:51 am
I really appreciate you sharing your observations and work. Now that you appear to be converging on an optimal approach, would it be possible for you to create a document which summarizes the process and provides recommended values (or possible starting points) for profile creation using the tools and techniques that you developed? I am sure it would be extremely useful to the entire community and would help simplify the process. Thanks!
Post by: Doug Gray on April 04, 2019, 04:34:19 pm
Hi Doug,
I really appreciate you sharing your observations and work. Now that you appear to be converging on an optimal approach, would it be possible for you to create a document which summarizes the process and provides recommended values (or possible starting points) for profile creation using the tools and techniques that you developed? I am sure it would be extremely useful to the entire community and would help simplify the process. Thanks!
I'm currently working on adding ABW to the program. Being able to use softproofing and get consistent B&W prints using ABW is badly needed for ABW users getting by in Photoshop.
So the main value of the program is to generate profiles that produce more accurate neutral and near neutral colors for regular color printing and do so with smaller patch totals. The secondary benefit is providing profiles for use in ABW printing.
But after the program is more finished I do intend to write a pdf description and outline the various usages.
Post by: JRSmit on April 05, 2019, 02:42:32 am
Here's the latest (Ver .4). It's a 32 bit Windows program so works in both x64 and x86 Windows.Hi Doug,
I've added options to generate high density subgrids in low and high RGB values. This is mostly useful for generating more accurate patch sets for images with larger, darker areas.
I've attached a sample, agnostic patch set with a 10x10x10 main grid and 6x6x6, low RGB value high density grid. The high density grid is repeated because the elements are close and this reduces variation in patches.
These are the options for the command line operation but it also works in interactive mode..
Usage: ICCProfilePatches [-f] [-t] [-r repeats] [-p pagesize] [-m MGrid]\n "
[-s StaggeredNeutralDistance] [-n neutralCnt] [-i iccProfile]\n"
[-C ColorimetricMode] [-c creator] [-o outfile]\n";
-t Test run. Don't create output. Use to get statistics\n"
-f Emit fractional RGB values, ints are default\n"
-r Remove neutrals in grids (use when adding neutrals separately)\n"
-v Scramble Patches\n"
-m MajorGridDim Major Grid dimension\n"
-M repeatcount Major Grid Repeat Count\n"
-l LowGridDim Low Grid Dimension (scales and places half spacing packed grid at low RGB values)\n"
-L repeatcount Low Grid Repeat Count\n"
-h HighGridDim High Grid Dimension (scales and places half spacing packed grid at high RGB values)\n"
-H repeatcount High Grid Repeat Count\n"
-s staggerVal Stagger alternate RGB neutrals by this magnitude\n"
-n Counts Counts neutrals (device or colorimetric) are added\n"
-C Color mode 1 (Colorimetric) or 3 (Absolute Colorimetric)\n"
-o CGATsFilename The name of the CGATs file\n"
-i ICC profile Profile used to adapt neutrals to actuals\n"
-c creator Who dunnit\n";
Your downloads are not accessible , has something changed?
Post by: Doug Gray on April 05, 2019, 03:21:32 am
Hi Doug,
Your downloads are not accessible , has something changed?
No idea. Seems to not be on the server anymore. odd.
Post by: Alan Goldhammer on April 05, 2019, 07:18:22 am
No idea. Seems to not be on the server anymore. odd.There are also some threads that disappeared yesterday on some of the other sections. I would ask the senior Moderator, Jeremy Roussak, what has happened (he won't talk to me).
Post by: JRSmit on April 05, 2019, 09:20:43 am
No idea. Seems to not be on the server anymore. odd.Too bad, hopefully will become available soon. Tested the previous version, did give a smoother result in the a and b curves from black to white. Withe exception of near blak and near white, on these areas there is a issue. That is why i am hoping on your latest version as it addresses those RGB value areas.
Post by: Alan Goldhammer on April 05, 2019, 11:58:07 am
Post by: Doug Gray on April 05, 2019, 12:17:11 pm
Doug, did you run the Canon printer calibration for the media setting before printing out profile targets? I just got my Pro-1000 set up this morning and before I start to go wild profiling papers it would be good to know how crucial a step this is.When I first set up the printer I ran the installation instructions which clearly includes some sort of initial calibration. I printed profile patches for reference. I then ran a calibration on Canon Plat. Gloss. It slightly reduced gamut and the patches varied so I cleared it back to the installation default. So all testing since has been with that native installation/calibration.
I think that's the best way to go for people that make their own profiles. I do like the idea of Canon's individual printer calibration where one is importing profiles and it should reduce printer to printer variation. But I don't think the approach is as good as directly profiling a printer.
Post by: Alan Goldhammer on April 05, 2019, 12:33:35 pm
Post by: Doug Gray on April 05, 2019, 02:20:46 pm
Yes, it does a required alignment/calibration when you first set the printer up. The printer documentation is really not clear on the usefulness of the media calibration process and I thought the same as you, that one's profile would address all the color management issues. I've heard that if you do a calibration and prepare a profile, that profile can be used reliably on someone else's calibrated printer. That's not my concern. I just didn't want to print out lots of targets and do profiling and then find out that it was not the optimal way to go.Yep, A really good technical approach to dealing with aging as well as inter printer compatibility with vendor profiles. Poorly documented though. Especially interactions between calibrations and paper types affected. Could be useful but I'd have to do a lot of experiments to see what works and what doesn't. Not worth the time unless I had multiple printers where it might be handy to use the same profile for each.
Post by: Doug Gray on April 06, 2019, 01:40:45 pm
I have recently discovered a peculiarity of I1Profiler when using lots of neutrals/near neutrals. And it's greatly magnified when used with smaller numbers of color patches.
This can produce, in some cases, wildly wrong profiles. I have seen profiles with huge variations along the neutral axis. Not often but sometimes.
I strongly recommend setting the "Smoother"slider at 80 instead of the default, 50, when using profiles with lots of added neutrals/near neutrals.
I'm currently working on finding the best settings by making profiles with these settings but for now, be aware of this potential problem.
Post by: rasworth on April 06, 2019, 03:20:43 pm
This combo (attached) of 256 neutrals/near neutrals and 6 colors went thru ok, with smoothness set to 50. How does "wildly wrong" manifest itself? Do we look for bad neutrals in a test print? Is there a quick check in ColorThink Pro?
Richard Southworth
Post by: Doug Gray on April 06, 2019, 04:10:32 pm
Doug,
This combo (attached) of 256 neutrals/near neutrals and 6 colors went thru ok, with smoothness set to 50. How does "wildly wrong" manifest itself? Do we look for bad neutrals in a test print? Is there a quick check in ColorThink Pro?
Richard Southworth
Richard,
I've seen profile errors just along the neutral axis of over 10 dE. Rare though. When it occurs it sees to be most often when the stagger value is 0 or near 0. Virtually goes away with the smoothness slider set to 80. It's really strange and difficult to pin down because it's so infrequent.
However, it disturbs me to see this sort of thing at all. Suggests some weird stuff going on in I1Profiler. Almost like some matrixes that were being inverted for curve fitting had near singular Eigenvalues. When this happens math errors can go through the roof. Since it takes a fair amount of work to detect this and strip out the near singular results it may be the cause. But this is rank speculation based on my own experience with DSP work. So I'm testing their program by introducing step functions in the neutral response and looking at how the smoothing setting interacts with it. It would be way faster if they had a batch processing capability like ProfileMaker 5.
In the meantime I suggest setting the smoothness slider at 80.
My gold standard is to print, and scan the LAB neutrals in steps of 1 and plot the results. This is time consuming. Right now testing an idea has a turnaround time of about 10 minutes. However, the magnitude of the errors, when they occur, are easily seen in my Matlab tools.
This is sort of fun, I like problems. Especially unexpected ones. Makes me want to dig and find out what's going on.
Post by: rasworth on April 06, 2019, 04:25:29 pm
As an EE (retired) I do remember eigenvalues and singularities, although it's been a while.
Thanks for the alert,
Richard Southworth
Post by: GWGill on April 06, 2019, 10:11:43 pm
Suggests some weird stuff going on in I1Profiler. Almost like some matrixes that were being inverted for curve fitting had near singular Eigenvalues.Any sort of curve fitting seems to have that sort of risk. I found that regularly spaced samples seems to exacerbate such issues - more stochastically spaced ones seemed less prone to such problems.
Post by: Doug Gray on April 07, 2019, 01:02:32 am
Any sort of curve fitting seems to have that sort of risk. I found that regularly spaced samples seems to exacerbate such issues - more stochastically spaced ones seemed less prone to such problems.
I haven't seen this on the packed grids. I'm actually rather astonished at how regular all the patch samples come out looking at just the average of the 8, equidistant LAB values compared to the centers. A simple bilinear interpolation should be sufficient and almost always stable. However, the problem came up when I created RGBLAB sets along the neutral axis for making ABW profiles with I1Profiler. Some of these exhibit wild variations.
Interestingly, and to my surprise. When these large excursions occurred, it was not in the AtoB tables. They retained accuracy to well under .05 dE. It was the BtoA tables that were toasted over 10 dE. And well within gamut and not anywhere near the boundaries.
One obvious conclusion is that the RGB<->LAB interpolation is done independently for the two tables and with no cross checking.
I'm going to try adding noise to the added samples. May help the math.
One nice side effect of independent interpolation of the tables is that it makes it easy to cross check profiles by running a very large RGB dataset through AtoB (save the Lab) then ->BtoA->AtoB (save Lab2) and check the variations.
EtoA: For those curious about what causes this without the tech jargon, finding the 3DLUT values involves having some metric that looks at all the nearby data (patch RGB and Lab values) to find an estimate of the values at another point on the 3DLUT grids. Sometimes, there are points within the collection that, varied together in certain ways, produce virtually no change. When this happens, it's called a singularity and the math explodes. It may seem odd that something that has essentially no impact explodes the math but it's because the variables have almost no effect on the accuracy so the math explodes as it tries to make them work. The solid solution is to find and remove these values when they have negligible effect then the math works well with little error on the remaining variables. This process is done with something called Singular_value_decomposition (https://en.wikipedia.org/wiki/Singular_value_decomposition) and it's very effective but rather CPU intensive.
Post by: Doug Gray on April 07, 2019, 06:05:48 pm
So now to port the MATLAB B&W code to C++ and finish this thing. I will eventually post the code and exe to GitHub where it won't get stripped when the servers here run out of room and delete attachments.
Post by: GWGill on April 08, 2019, 02:27:58 am
I'm retracting my warning. The problem I saw with the really wild errors was a result of poor synthesis of "color" patches used to fool I1Profiler into making a "color" profile that is excellent for ABW printing.Hmm. Were you doing something like creating an RGB profile with patches only along the neutral axis ?
Post by: Doug Gray on April 08, 2019, 10:51:19 am
Hmm. Were you doing something like creating an RGB profile with patches only along the neutral axis ?
No, of course that doesn't work. What I had done was use measured neutrals from ABW along with synthetic "color" patches to fill out the gamut. The problem was that the synthetic patches L* were significantly out compared to the synthetic "color" patches. Errors increased as that divergence increased and eventually exploded. Really an error on my part. I have since aligned the L*'s of the synthetics with the measured ABW L*. And that works very well. This makes excellent ABW profiles. Goal was to make profiles that produce accurate mapping so, for instance, a ABW print on one printer/paper matches that of another printer/paper. Within gamut limits much like a regular color profile.
Post by: GWGill on April 08, 2019, 06:55:51 pm
This makes excellent ABW profiles.So I gather that nothing in the workflow is insisting that ABW profiles be monochrome profiles ?
Would it even cope with monochrome profiles ?
(i.e. is the Epson printer being sent RGB data and converting to monochrome ?)
Post by: Doug Gray on April 08, 2019, 07:17:58 pm
So I gather that nothing in the workflow is insisting that ABW profiles be monochrome profiles ?I1Profiler, or Argyll's colprof, would just see the patch set as another RGB/Lab dataset and make a profile. It is a "color" profile in that the only measured values are the monochrome ones from the ABW mode print. Color patches are synthesized to align with the monochrome L*s. As such, it is not designed to convert a color image to B&W when printing. Conversion to B&W must be done first. Soft proofing accurately shows the tone curve for Perc. Rel. and Abs. Col. but does not show paper/ink tint colors. It always shows as neutrals. They are, after all, not something one can edit in a B&W image but something set with sliders in the driver (where available)
Would it even cope with monochrome profiles ?
(i.e. is the Epson printer being sent RGB data and converting to monochrome ?)
Post by: Doug Gray on April 09, 2019, 01:19:52 am
Neutral patches were made from L*=0 to 100 in steps of 1 and printed in a 16 bit tifs. 9 duplicates of the 101 patches were made and scrambled to reduce printer position variation which can be quite large and create slightly ragged steps.
The tif file was printed using standard color management selecting the ABW profile created with I1Profiler and my C++ program. The results printing in Perceptual Intent and Abs. Col. Intent are shown in the attached Matlab plot.
Average dE00 was .33 and would have been lower but I didn't follow my usual sequence of printing an initial page then tossing it.
Still, the very smooth performance of ABW is very evident and cleaner than I can get using color printing. The Abs. has nice ending plateaus at L*=3 and L*=95 which is the paper's black ink point and white point respectively.
The Perc. plot goes smoothly from end to end intercepting the black and white points. Rel. Col. with BPC (not plotted) would produce a very straight line from end to end.
Post by: GWGill on April 09, 2019, 01:51:17 am
I1Profiler, or Argyll's colprof, would just see the patch set as another RGB/Lab dataset and make a profile.Sorry, that's not what I was asking. I was wondering if Epson's driver in AWB mode expects monochrome ICC profiles (i.e. ICC profiles with one device channel). [ Assuming of course that Epson's driver in AWB has a slot for ICC profiles. ]
Post by: Doug Gray on April 09, 2019, 01:59:38 am
Sorry, that's not what I was asking. I was wondering if Epson's driver in AWB mode expects monochrome ICC profiles (i.e. ICC profiles with one device channel). [ Assuming of course that Epson's driver in AWB has a slot for ICC profiles. ]Epson, and other drivers will convert RGB values that are not equal to B&W. Probably with some standard algo weighting the RGBs. I'm not aware of any documentation on it. Since this profile is designed for B%W images all the values exiting the CME will be within about 1 bit from each other and are usually the same. Thus whatever the driver does mapping different RGB levels to B&W doesn't affect anything.
Post by: Doug Gray on May 05, 2019, 06:55:32 pm
The primary conclusion is that the Packed RGB approach provides much less value than I had expected. The differences between just a straight linear RGB set and packed linear sets of similar sizes are very small. However, what has shown up in spades is that getting good performance on near neutrals is very dependent on the patch set and benefits from larger, near neutral patch sets. Further the tracked profiles, where an initial profile is made with a larger number of neutrals (100 to 250), then a set of patches with tracking neutrals and standard set of color patches, produces the best result with excellent accuracy on both color and near neutrals.
I no longer generate a patch set. Print it. Then validate it by printing known colors. This is inadequate as the printer variation overcomes differences in patch size effectiveness once the overall size exceeds about 1000 patches. This is what I have done to effectively eliminate the page variation that has obscured earlier results.
To compare various patch sets, I concatenate them so I have all the different sets I wish to compare. Then I randomize them so that each patch set is distributed over all the printed pages. Thus, any differences due to warm up, or printer cartridge shaking, or even paper variation are distributed.
When the pages are scanned, the results are de-randomized and each set corresponding to specific patch sets being tested is extracted and saved as a CGATs file which then is plopped into I1Profiler's measurement tab to create a profiles for that set.
Then, a similar process is done for measuring color accuracy. The basic set for this is a combination of colors, measured and extracted from an XRite ColorChecker SG but only color patches are used, the neutrals are excluded. To test the neutrals, a set of L* values from 3 to 95, with a* and b* set to 0 is used.
For testing each profile created from the previous process, the Lab colors are then converted to device space and concatenated and randomized then printed on 1 or more pages depending on the number of profiles being tested. The scanned results are then descrambled and statistics gathered on actual measured Lab values.
While warm up, occasional ink cartridge "shaking" or other changes in the printer can and do cause changes, these effects apply to all the tested patch sets so I get good relative repeatability (around .02 on dE averages) even while seeing between .1 and .3 dE variation due from print to print.
More specifically, I can easily get an average dE of .45 to .60 from one print to another using the same profile. But using the same profile within the scrambled sets, I see that variance drop to a difference of around .02.
The main result is that I see the total patch size, not whether they are packed, as being the largest determiner of color profile accuracy and the spacing and arrangement of near neutrals the big factor for B&W or near neutral accuracy. The automatically generated patch sets I1Profiler produces are pretty good for colors. They increase the RGB spacing irregularly, for instance, increasing the spacing of one channel at a time and filling in whatever is left with near neutrals. The break points where the number of near neutrals is greatest produces the better near neutral profile performance though the dE2k of the near neutrals still exceeds that of the colors. I have various near neutral sets that produce materially better results in this region and generating tracking profiles produces better results yet with much fewer added near neutrals.
This will result in a substantial re-write of my code which gave undue advantage to packed RGB patch sets.
Post by: Doug Gray on May 06, 2019, 02:52:15 pm
Post by: rasworth on May 06, 2019, 03:23:23 pm
Richard Southworth
Post by: Doug Gray on June 01, 2019, 01:34:16 am
Each of these ave dE00 values were from measuring a 301 patch set of dispersed, in gamut, LAB values. The same values were used for all profiles created from the different patch sets.
The packed and standard profile patch sets were all concatenated, randomized, printed and scanned then de-randomized to make CGATs files which were then used to create profiles for their patch set. These profiles were used to create device space RGB patch sets with Abs. Col. conversion. Randomized, printed, and scanned then de-randomized and statistics gathered.
For patch sizes below about 1500, packed grids produce lower deltaE 2000 results than straight linear grids, the I1Profiler default. However, the advantages are greatest at small patch counts. For instance, the packed grid 5x5x5 with an inset 4x4x4 for a total of 189 patches produced an ave. dE00 of .60 while the traditional linear grid of 6x6x6 (216 patches) was dE00 of .73
However, increasing the grid size up to about 3000 resulted in asymptotically approaching dE00 of .30 whether the patches were packed or traditional. There was no measurable difference and increasing the patch count over 6000 actually slightly increased the dE00 from .30 to .32
Turns out the 855 packed grid, 8x8x8 with an inset 7x7x7 with added near neutrals outperforms the I1Profiler 957 patch, one page iSis default on my three printers for both glossy and matte. It's not printer specific so I've added it as an attachment.
Post by: rasworth on June 01, 2019, 11:04:39 am
To verify I understand I'm summarizing:
1. Your attached target patch set, consisting of a main 8x8x8 and packed 7x7x7 and 100 or so added near neutrals is a good (.6 avg. dE00) overall profile generating target set.
2. Better neutral performance can be obtained using the 8x8x8/7x7x7 together with 100 tracked neutrals.
Richard Southworth
Post by: Doug Gray on June 01, 2019, 11:46:45 am
Doug,Yes, the target set has better performance than the I1iSis 957 patch set which fits on a single US letter size page. For my 3 printers it improves accuracy on my 3 printers by about 30% of what can be achieved by using a large number, 3000 or so, of patches. Increasing the I1Profiler patch set generator to about 1300, exclusive of near neutrals, produces roughly the same improvement. However, for day to day work with an iSis, the single page set is attractive. I posted a detailed comparison of this v the standard iSis 957 patch set as well as an extended patch set from I1Profiler using 2 pages, 1914 patches.
To verify I understand I'm summarizing:
1. Your attached target patch set, consisting of a main 8x8x8 and packed 7x7x7 and 100 or so added near neutrals is a good (.6 avg. dE00) overall profile generating target set.
Quote
It generally produces better neutrals than the I1Profiler's 957 iSis patch set. but not always. Depends on the printer an paper. However, when used with tracking neutrals it is always much better on neutrals regardless of the patch set size. At least with more than a few hundred color patches. Without sufficient color patches, >100 or so, the tracking neutrals gets squirrely.
2. Better neutral performance can be obtained using the 8x8x8/7x7x7 together with 100 tracked neutrals.
Quote
Richard Southworth
Post by: Doug Gray on August 28, 2019, 11:58:16 pm
Packed grids of the same size offer little advantage over conventional grids for larger patch counts over 1000 in terms of averge dEs. However, they signficantly reduce the dEs associated with the worst 20% of dEs and most especially, the worst 5%.
In particular, The average dEs of the packed grid with a count of 1729 patches is virtually identical to that of the conventional grid of 4096 patches for the worst 5% of dEs.
Detail
I return to evaluate packed v standard grid patch performance with a set of 3 packed grid patches and 3 standard grid patches using a very large (1940) set of independent color patches randomly distributed in the printer's RGB device space.
There are 3 conventional grid patch sets and 3 packed grid sets of similar sizes. They are:
Conventional: 343, 1728, and 4096 patch counts which are grids of 7x7x7, 12x12x12, and 16x16x16.
Packed: 341, 1729, 3925 patch counts which are grids of 6x6x6, 10x10x10, 13x13x13 each of which has an inset grid with the same spacing and patches centered in the cubes of its surrounding grid.
These 3 sets are close enough in patch counts to produce a good performance comparison of the two approaches.
High precision is obtained by using a very large set of patches together with the patch targets scrambled and combined into 14, 957 patch sheets. It's possible to find even tiny differences in performance as the standard deviation of the means decreases in proportion to the square root of the number of patches in a group.
Further, the large number of patches allows closer examination of the characteristics of outliers and not just total averages. When looking at outliers, like the 1% that are the worst, it's important to get enough patch samples to reduce the impact of paper/printer anomalous results which can easily produce dE's of 1 or more on exactly the same color.
Here are two chart pairs, one for Argyll created profiles and one for I1Profilers using the highest quality settings. The chart's X axis is percent of patches that are at or below the dE00 shown in the Y axis. The colors represent the patch size of charts, red for the smallest, black for the largest. Solid lines are from the packed, dashed from the conventional grids.
Each chart pair has the full range of 1% to 99% on the left, and an expanded range showing outliers 80% to 99% on the right.
A few points of note.
The packed, 341 chart shows that only about 1 in 10 printed colors exceeded .95 dE while the 343 patch conventional grid chart shows 1 in 10 exceeded 1.07 dE for I1Profiler profiles. Interestingly, the Argyll profiles showed worse performance 1.02 dE and 1.23 dE respectively.
For the large (4k) patch sets, the differences between I1Profiler and Argyll patches reversed. One in ten patches of the both the 3925 packed and 4096 conventional profile exceeded .60 dE however, for the Argyll profile 3925 packed patches dE dropped to .54.
Generally, the Argyll profiles reduced outliers the most for large patch sets and noticeably so for the larger packed, profiles.
The I1Profiler packed 1729 profile performed as well as the conventional, 4096 patch profile keeping the dE below 1.0 for 97% of the independent colors.
The packed 3925 patch Argyll profile printer 99% of all 1940 patches with a dE of 1.0 or lower.
Argyll
(https://t2l21w.dm.files.1drv.com/y4mdUB3r686b8ueagRSFloGli9QZG0h4zZGRr7Ud1IpIFpBodJoLTNiRWD8ZDbXTWi8Bu7GPUuTYF4CJ2Drw62n7BlFIBD8EXp-oelBres48jZE3FN2tzp_2bhZ6muk35dMu5F1Xxjtg1rOcb8aAKrIUyuxuRFSJk9k5CXSByLHK0SjTHt9iEBLFKzcO41bbh7t?width=660&height=413&cropmode=none)
I1Profiler
(https://6t0deg.dm.files.1drv.com/y4mzv0VtRsNQEEbzMgBfWSb8e0MadRFaqwO00kqaYxOiKBmMt98kpAcaRySRZNKE3fvKEGOmidyLOBiOTVH6b-6SAY9UR91bWEZKHoNen1riJ8z8jNuG6mVxAZtTWHsSe2GUyLC5kxZSbmVuWOfbOOvRAo-JyqWqV9I4QM--SQfuO91u4sw8Oh4_5zPDMZDCPT6?width=660&height=4134&cropmode=none)