Luminous Landscape Forum

Raw & Post Processing, Printing => Colour Management => Topic started by: Rhossydd on April 01, 2011, 07:09:53 am

Title: i1 Profiler monitor QA odd results ?
Post by: Rhossydd on April 01, 2011, 07:09:53 am
I’m getting some curious QA results from i1Profiler.
Using an i1pro spectro to calibrate (110cdm) and profile my PA271 with Basicolor 4.1.22 (aka Spectraview II in Europe). When I use the validation option I see Average DeltaE94 0.25, max 0.93, standard deviation 0.26. All pretty good I reckon.

Now if I try the display QA option in i1 Profiler with the same instrument it fails with average DE 3.09, maximum 4.88. Why the difference ?

Paul
Getting frustrated with X-Rite releasing the software, but not the licence codes!
Title: Re: i1 Profiler monitor QA odd results ?
Post by: Czornyj on April 01, 2011, 09:44:12 am
You're comparing apples to oranges - basICColor only validates the monitor profile - it measures same RGB patches that were used for display profile creation. Basically it just shows if the display behaves in the same way as when it was profiled.

QA in i1Profiler works in a different way - it calculates RGB values of color space of our display that should give best match to colorimetric values of CC24 patches. In this case gamut of the display may not cover all CC24 patches 100%, or the profile may not be accurate enough to give a perfect match.
Title: Re: i1 Profiler monitor QA odd results ?
Post by: Rhossydd on April 01, 2011, 11:36:37 am
You're comparing apples to oranges...... Basically it just shows if the display behaves in the same way as when it was profiled.
?? As I understand it it's measuring the accuracy of colours displayed on the screen.

Quote
QA in i1Profiler works in a different way - it calculates RGB values of color space of our display that should give best match to colorimetric values of CC24 patches. In this case gamut of the display may not cover all CC24 patches 100%, or the profile may not be accurate enough to give a perfect match.
The CC24 is comfortably within the PA271's gamut and the Basicolor validation suggests the profile is accurate to less than DE1.

Having studied the results more closely it seems that all the Profiler measured results have a lower than expected luminance value. It's looking like the QA module is taking X-Rite's default 120cdm luminance target and is expecting higher luminance values than I choose to work with, so thinks the results it's measuring are wrong.
I can't see how this can be changed, so maybe this module will only be of use if using it's own monitor profiling module.
Not a lot of use to those of us that use other profiling software to benefit from hardware calibrated displays.
Title: Re: i1 Profiler monitor QA odd results ?
Post by: Mark Paulson on April 01, 2011, 11:53:15 am
Getting frustrated with X-Rite releasing the software, but not the licence codes!


I called Xrite and got my code right then. I was entitled to a free upgrade and sent them an email with my proof of purchase a couple of days ago and called yesterday to ask some questions and they sent the code.
Title: Re: i1 Profiler monitor QA odd results ?
Post by: Rhossydd on April 01, 2011, 02:03:32 pm
I called Xrite and got my code right then. I was entitled to a free upgrade.....
Lucky you. I've spent a serious amount of money on GMB/X-Rite products in the past and want to upgrade, but I have to wait until they can be bothered to get the boxes out to dealers :(

Edit: I've just got an email notifying me of dispatch of my upgrade box.
"Next day", but I doubt I'll see it until Monday and it does . At least I've little on next week to allow me some time for evaluation. The only advantage of the mess of pre-release is that at least I'll have viewed all the tutorials, read the help files and been able to print out some charts and let them dry down before measurement.
Title: Re: i1 Profiler monitor QA odd results ?
Post by: gromit on April 01, 2011, 07:24:57 pm
?? As I understand it it's measuring the accuracy of colours displayed on the screen.

No. It's measuring how closely the profile generated models the actual displayed values ... to the accuracy of the instrument. It says nothing about how "accurate" the colours are.

Try calibrating with MultiProfiler and profiling (only) with basICColor display (or whatever).
Title: Re: i1 Profiler monitor QA odd results ?
Post by: digitaldog on April 01, 2011, 08:05:38 pm
If you have a PA, you don’t want to mess with i1P or any 3rd party software for calibration and profiling when you have the NEC software.
Title: Re: i1 Profiler monitor QA odd results ?
Post by: Rhossydd on April 02, 2011, 02:11:32 am
No. It's measuring how closely the profile generated models the actual displayed values ... to the accuracy of the instrument. It says nothing about how "accurate" the colours are.
You'll need to re-phase that, because to me how closely matched the colours are displayed is the same thing is how accurately they're reproduced.

Quote
Try calibrating with MultiProfiler and profiling (only) with basICColor display (or whatever).
Why ?
Title: Re: i1 Profiler monitor QA odd results ?
Post by: gromit on April 02, 2011, 09:10:54 pm
You'll need to re-phase that, because to me how closely matched the colours are displayed is the same thing is how accurately they're reproduced.
Why ?

Here's an analogy: You have a ruler and mark out 1m along a piece of wood. Next you cut the wood at the mark you made then measure the length with the same ruler. Yep 1m. This is validation. Except your piece of wood is 997mm long due to the inherent inaccuracy of the ruler.

The same with monitor calibration/profiling. Consumer grade instruments (colorimeters, spectros) are inherently inaccurate, made more so by light leakage when taking the measurements, dirt on the reference slate, off-vertical measurement with spectros etc. This is exacerbated by the different/larger gamut that colorimeters have to deal with these days. If you have a number of devices you'll notice that they all give slightly different results. I managed to achieve a dE of 0.15 for validation on my hand-picked Spectraview Reference PA271 with the best combination of software and hardware I had, but visually prefer the results I get (more neutral shadows, cleaner grayscale ramps) by using MultiProfiler alone. So that's what I use. It may be more satisfying to get your monies worth out of whatever software/instrumentation you bought, but at the end of the day it's the results that count.
Title: Re: i1 Profiler monitor QA odd results ?
Post by: Rhossydd on April 03, 2011, 04:10:15 am
Sorry gromit, but you're rather missing the point.
A spectro will have a degree of error both in absolute accuracy and repeatability, but that can be ascertained with testing and you can get a good idea of how accurate the instrument is in practice. For my main instrument I see better than 0.14 DE.
The data analysis module of Profiler will be able to help test and calculate these sorts of errors.

So once you know the degree of error possible for your instrument you can then go on to measure colours on different devices with a known degree of accuracy.

The problem (design failure?) of the QA module in Profiler is that it seems impossible to define what standards it's measuring to, (luminance, colour temp etc), if you haven't built a profile for it to measure. In other words, if you wish to use another software product to calibrate and profile your display with (eg Spectraview, Multiprofiler or the Eizo products) Profiler can't be used to independently validate the quality of the display. A facility that would be useful and I had thought was being included within the product.
Title: Re: i1 Profiler monitor QA odd results ?
Post by: gromit on April 03, 2011, 04:35:59 am
Sorry gromit, but you're rather missing the point.
A spectro will have a degree of error both in absolute accuracy and repeatability, but that can be ascertained with testing and you can get a good idea of how accurate the instrument is in practice. For my main instrument I see better than 0.14 DE.

The i1Pro isn't great for measuring darker values on monitors. I got better results overall from a Samsung branded i1D2 (came with their XL20 LED monitor) than any other device in my arsenal. The Optix XR was a good device in its day but unsuited to the PA271.

Irrespective of devices and validation results I run visual tests after calibration/profiling to test shadows/highlights for neutrality/separation, gray ramps for smoothness/cleanliness, gamut tests and real world images (skintones etc) to see if I'm happy with the results. MultiProfiler is excellent in these respects, and gives negligible difference in rendition to hardware calibration. Given the flexibility in changing white/blackpoint on the fly it's what I use. People place too much confidence in instruments (most of which cost peanuts to manufacture) and bogus numbers instead of using their eyeballs IMHO.
Title: Re: i1 Profiler monitor QA odd results ?
Post by: Rhossydd on April 03, 2011, 04:51:29 am
People place too much confidence in instruments (most of which cost peanuts to manufacture) and bogus numbers instead of using their eyeballs IMHO.
I guess if that's your opinion, colour management isn't really for you.
Title: Re: i1 Profiler monitor QA odd results ?
Post by: gromit on April 03, 2011, 05:57:22 am
I guess if that's your opinion, colour management isn't really for you.

Given a number of solutions for monitor calibration/profiling (including separating the two) I'll simply choose the one that gives the best visual results, that's all.

Try calibrating with MultiProfiler, even if you still use an instrument for profiling. You might find this beneficial!
Title: Re: i1 Profiler monitor QA odd results ?
Post by: Rhossydd on April 03, 2011, 06:49:44 am
Try calibrating with MultiProfiler, even if you still use an instrument for profiling. You might find this beneficial!
I tried it and didn't find beneficial when I first got the PA
Title: Re: i1 Profiler monitor QA odd results ?
Post by: Czornyj on April 03, 2011, 02:49:17 pm
?? As I understand it it's measuring the accuracy of colours displayed on the screen.
The CC24 is comfortably within the PA271's gamut and the Basicolor validation suggests the profile is accurate to less than DE1.
The difference is, that while profiling you're measuring some RGB patches (255.0.0, 0.255.0, 0.0.255 - etc.), and then you're measuring the same patches during validation. You don't really know about the accuracy of the whole rest 16,7mln R,G,B values...

Having studied the results more closely it seems that all the Profiler measured results have a lower than expected luminance value. It's looking like the QA module is taking X-Rite's default 120cdm luminance target and is expecting higher luminance values than I choose to work with, so thinks the results it's measuring are wrong.
I can't see how this can be changed, so maybe this module will only be of use if using it's own monitor profiling module.
Not a lot of use to those of us that use other profiling software to benefit from hardware calibrated displays.
I think you're right. I've also noticed, that it has a different opinion than SpectraviewII, basICColor display/Spectravie profiler concerning the luminance value and wtpt... XRGA?
Title: Re: i1 Profiler monitor QA odd results ?
Post by: Czornyj on April 03, 2011, 03:02:37 pm
correction - QA likes something, it recognizes as 160cd/m^2 luminance level.
Title: Re: i1 Profiler monitor QA odd results ?
Post by: Rhossydd on April 03, 2011, 03:05:10 pm
correction - QA likes something, it recognizes as 160cd/m^2 luminance level.
Curious, the default is 120
Title: Re: i1 Profiler monitor QA odd results ?
Post by: Rhossydd on April 03, 2011, 03:12:11 pm
The difference is, that while profiling you're measuring some RGB patches (255.0.0, 0.255.0, 0.0.255 - etc.), and then you're measuring the same patches during validation. You don't really know about the accuracy of the whole rest 16,7mln R,G,B values...
Obviously measuring every possible value is impractical.
The choice of using the CC24 values is actually more useful than Basicolor's values* as they are more representative of real world image values. Which why the difficulty(impossibility?) of using profiler's validation module would have been desirable.

*you can just look at the less extreme(ie real world) colours' validation values and make your own mind up. It still looks good.

It's also disappointing to see how poor the printer profile validation routine is, just print a CC24 out and get an overlay to put over it. Where's the ability to actually measure a chart and compare it empirically for validation ?
Title: Re: i1 Profiler monitor QA odd results ?
Post by: Czornyj on April 03, 2011, 04:48:12 pm
Curious, the default is 120
L*a*b works well in certain viewing conditions, so maybe the LCD must be @160cd/m^2 to display these colors correctly? Unfortunately, I have limited knowledge on advanced color science, but considering Stevens & Hunt effects the luminance of the display has visual impact on perceived color and contrast (see page 5, 27)
http://www.cis.rit.edu/fairchild/PDFs/AppearanceLec.pdf
Title: Re: i1 Profiler monitor QA odd results ?
Post by: digitaldog on April 03, 2011, 05:30:21 pm
Sorry gromit, but you're rather missing the point.
A spectro will have a degree of error both in absolute accuracy and repeatability, but that can be ascertained with testing and you can get a good idea of how accurate the instrument is in practice. For my main instrument I see better than 0.14 DE.

I agree with gromit and the ruler analogy. Without a higher grade, known reference instrument, you can’t possibility known how well the i1 did in measuring the colors. If it were doing such a superb job, why would you be running this variation process over again unless you were trying to figure out if something happened within the process (like the puck falling off the display)? And that’s about the only useful functionality of such ‘validation’ to come back from the process, see it looks butt awful and then wanting confirm numerically it was butt ugly. But why its butt ugly isn’t defined. Given that the software producing this validation often sends colors within colorspace of the display that will produce good values, the process is more a feel good function than anything useful. Now you have a reference grade spectroradiometer and software you can use to examine both its measurements and the i1? That could be useful.

Quote
No. It's measuring how closely the profile generated models the actual displayed values ... to the accuracy of the instrument. It says nothing about how "accurate" the colours are.

Exactly.
Title: Re: i1 Profiler monitor QA odd results ?
Post by: Rhossydd on April 04, 2011, 02:25:05 pm
I agree with gromit and the ruler analogy. Without a higher grade, known reference instrument, you can’t possibility known how well the i1 did in measuring the colors.
So where does the i1 pro fit in the range of absolute accuracy for measuring colour ?
Presumably there's data in existence as to how good they absolutely can be, how consistent they are from reading to reading, how much they vary from sample to sample and how they're likely to change over their lifetime.
Title: Re: i1 Profiler monitor QA odd results ?
Post by: digitaldog on April 04, 2011, 02:28:59 pm
So where does the i1 pro fit in the range of absolute accuracy for measuring colour ?
Presumably there's data in existence as to how good they absolutely can be, how consistent they are from reading to reading, how much they vary from sample to sample and how they're likely to change over their lifetime.

That’s an impossible question to answer. Compared to a $20K spectroradiometer? Compared to other devices in its price range? Which colors within color space? And its not just the instrument that’s a factor here. Its the software and entire process of calibration and profiling that makes validation using one instrument for the process less than effective.
Title: Re: i1 Profiler monitor QA odd results ?
Post by: Rhossydd on April 04, 2011, 02:50:03 pm
That’s an impossible question to answer.
No it isn't.
Just in the same way you can compare your ruler to a known standard, you can get it to measure a known colour.

Maybe a 20k instrument might be accurate to 0.01 DE and consistent to 0.005 from measurement to measurement and only drift no more than 0.2DE in two years. That might be necessary for industrial process control.
On teh other hand, i1 Pros might only get accurate to 0.6 DE*, accurate to within 0.15DE from reading to reading* and drift about 1DE over two years, that migth still be a valuable standard if you were trying to see how accurate a colleague's laptop display had been calibrated or how a print has faded in daylight.

This data must be known, even if it isn't in the public domain.
We might be pleasantly surprised if we knew the facts.

* these numbers would fit the data I've collected from my own i1 Pros
Title: Re: i1 Profiler monitor QA odd results ?
Post by: digitaldog on April 04, 2011, 02:59:10 pm
No it isn't.
Just in the same way you can compare your ruler to a known standard, you can get it to measure a known colour.

The standard hasn’t been defined. Nor the method for analyzing colors through the display system and profile (for validation).

Quote
Maybe a 20k instrument might be accurate to 0.01 DE and consistent to 0.005 from measurement to measurement and only drift no more than 0.2DE in two years.


For what colors? All?

Quote
This data must be known, even if it isn't in the public domain.

Such data is usually specified. You can find some data for the EyeOne Pro. Does your instrument produce this? For all colors one might measure? That’s not defined.
Quote
Measurement geometry: 45°/0° ring illumination optics, DIN 5033
Light source: Gas filled tungsten (Type A)
Physical filters: No or UV cut (Filters not exchangeable)
Inter-instrument agreement: Average DE*94 0.4, max. DE*94 1.0 (Deviation from X-Rite manufacturing standard at 23°C for single measurement mode on 12 BCRA tiles (D50,2°)
Short-term repeatability: DE*94 <= 0.1 (D50,2°), with respect to the mean CIELab value of 10 measurements every 3 seconds on white
Data format: Spectral radiance (mW/nm/m2 /sr); Luminance Y (cd/m2)
Measurement range: 0.2 ... 300 cd/m2
Short-term repeatability: x,y: +/- 0.002 typical (CRT 5000°K, 80 cd/m2)
Title: Re: i1 Profiler monitor QA odd results ?
Post by: Rhossydd on April 04, 2011, 03:52:41 pm
The standard hasn’t been defined.
? not sure what you mean by this. There's enough standards around in colour science to work to. It just has to be stated.
Quote
For what colors? All?
You can just specify a range for which any standards chosen to apply, eg a known colourspace like ColormatchRGB . Just like saying a car speedometer might be accurate to +/- 2mph from 20mph to 80mph
Quote
Such data is usually specified. You can find some data for the EyeOne Pro. Does your instrument produce this?
See you can find these sort of things out...................... in minutes ;-)
Actually my i1s might fall slightly outside that criteria. I'm seeing a higher error rate with the older spectro, not surprising seeing it's nine years old and hasn't been back to X-Rite for re-calibration. The newer one is pretty close to that spec, but without access to the same reference samples it would be bold to make too many claims for it.
What is apparent is that error rates are rarely going to exceed visible tolerances. So using them for real world verification should be acceptable.
Title: Re: i1 Profiler monitor QA odd results ?
Post by: digitaldog on April 04, 2011, 03:57:10 pm
? not sure what you mean by this. There's enough standards around in colour science to work to. It just has to be stated.

Go ahead.

Quote
Just like saying a car speedometer might be accurate to +/- 2mph from 20mph to 80mph

So you believe that a Spectrophotometer, and a car, have the same degree of accuracy over the same span (in your example, its the same from 20-80,mph?).

Quote
Actually my i1s might fall slightly outside that criteria. I'm seeing a higher error rate with the older spectro, not surprising seeing it's nine years old and hasn't been back to X-Rite for re-calibration.


A procedure that some recommend once a year....

Quote
The newer one is pretty close to that spec

You got that data how?

Quote
but without access to the same reference samples it would be bold to make too many claims for it.

Indeed.
Title: Re: i1 Profiler monitor QA odd results ?
Post by: Rhossydd on April 04, 2011, 04:21:05 pm
Go ahead.
I'm not the expert here. I'm just saying that it ought to be possible.
Quote
So you believe that a Spectrophotometer, and a car, have the same degree of accuracy over the same span (in your example, its the same from 20-80,mph?).
It's perfectly possible to say over what range of speed a speedometer will work to a specified accuracy. Why can't a similarly appropriate set of variability over a known range be specified for a spectro ?
You can insert any other form of measuring device and variable, It's what the science of measurement is based on.
Quote
A procedure that some recommend once a year....
Yes, it's not ideal, but I'm pretty certain that not many owners make the investment in annual re-calibration of their i1. The clue was that when I asked X-Rite they didn't immediately know how much it would cost or how long it would take, rather a give away that they're not dealing with that service very frequently.
It would be fascinating to know how the ones they do re-calibrate have drifted over time and if they still meet the criteria you've quoted, I wonder if they check ?
Quote
You got that data how?
Consistency of readings can be done by repeatedly measuring the same targets and comparing the results. I've done it a few times when I've been interested enough and had some spare time.
i1 to i1 is harder as I've only access to two instruments, but again doing a similar test to above you can start to get a rough idea of what's going on.
I'm not defending it as definitive data, but it's more than I've read of anyone else doing.
Title: Re: i1 Profiler monitor QA odd results ?
Post by: gromit on April 04, 2011, 05:31:43 pm
What is apparent is that error rates are rarely going to exceed visible tolerances. So using them for real world verification should be acceptable.

Try this. Create an image the width of the screen at 100% with the same gamma you're calibrating to and fill with a gradient from black to white. Hold down the shift key to ensure the fill is horizontal. (Bill Atkinson's "LAB Grays Test Image" and "Grays Test Image" are excellent for this purpose, if you can find them.)

Now calibrate and profile with your i1Pro. Look at the image above and make an assessment of how smooth the ramp is, how neutral through the range it is. (You may want to turn off Open GL as this has a tendency to screw things up.) Now calibrate/profile to identical target settings with MultiProfiler and examine the ramp again. Are there differences? Which is better? Could it be that, for neutrals at least, the eye is more discerning than the instrument/process you place so much faith in? Doing a test like this will answer these questions for yourself.
Title: Re: i1 Profiler monitor QA odd results ?
Post by: Rhossydd on April 05, 2011, 03:51:46 am
Try this..... make an assessment of how smooth the ramp is
Been there done that, but that's testing a different parameter of monitor performance. It doesn't tell you if colour is accurate.

Do you not see that colour accuracy might be important to some users ?
Title: Re: i1 Profiler monitor QA odd results ?
Post by: gromit on April 05, 2011, 04:04:16 am
Been there done that, but that's testing a different parameter of monitor performance. It doesn't tell you if colour is accurate.

As expounded above, neither will your i1. The gray ramp is just one of a number of tests to run. Personally, I've found negligible difference in colours between MultiProfiler and the best hardware calibration I can do.
Title: Re: i1 Profiler monitor QA odd results ?
Post by: Rhossydd on April 05, 2011, 04:15:33 am
As expounded above, neither will your i1.
No, the previous discussion has just highlighted that it's possible that an i1 may not give absolute accuracy. However it's also possible it will give give extremely accurate results and, if performing to it's manufactured specification, ought to be more accurate in colour assessment than the human eye.

You might not value anything more than your own display 'looking' right and maybe matching your prints, but others will benefit from being able to make objective comparisons of display colour accuracy. Being able to accurately measure the output of a display can, over time, can also help detect deteriorations in the output quality that wouldn't be noticed by eye alone.
Title: Re: i1 Profiler monitor QA odd results ?
Post by: digitaldog on April 05, 2011, 10:09:16 am
Being able to accurately measure the output of a display can, over time, can also help detect deteriorations in the output quality that wouldn't be noticed by eye alone.

That’s trending, a useful function some products provide (and again assume the instrument is consistent in delivering its data). An inaccurate but consistent instrument can provide trending to the user. Trending doesn’t tell us anything about accuracy, that’s a different metric.
Title: Re: i1 Profiler monitor QA odd results ?
Post by: gromit on April 05, 2011, 07:04:40 pm
However it's also possible it will give give extremely accurate results and, if performing to it's manufactured specification, ought to be more accurate in colour assessment than the human eye.

Photography/printmaking is a visual medium, not a branch of number theory like you/many here seem to think. If the results look good and you didn't waste any/much paper getting there, where's the harm in this? People obsess too much about accuracy, numbers, gamut volume etc, with typical soulless results. You'd be better off developing your visual skills (which will immediately tell you if something is amiss) rather than treating the whole exercise as a science project.
Title: Re: i1 Profiler monitor QA odd results ?
Post by: Rhossydd on April 06, 2011, 05:02:04 am
Photography/printmaking is a visual medium, not a branch of number theory like you/many here seem to think.
If you won't understand that part of photography and print making, and in particular it's process control, relies on the ability to measure and replicate results why bother reading forums on the science of photography ?
You've added nothing of value to this thread. Don't waste our bandwidth.
Title: Re: i1 Profiler monitor QA odd results ?
Post by: gromit on April 06, 2011, 07:35:08 am
If you won't understand that part of photography and print making, and in particular it's process control, relies on the ability to measure and replicate results why bother reading forums on the science of photography ?

You just need to get things into perspective. Colour management is a tool, not an end in itself. The only reason I contributed to this thread was to correct your understanding of what monitor validation is, and its shortcomings, something you still seem to be struggling with.