Pages: 1 2 [3]   Go Down

Author Topic: Green/blue color cast with scanner profiles  (Read 13337 times)

BrianToth

  • Newbie
  • *
  • Offline Offline
  • Posts: 43
Re: Green/blue color cast with scanner profiles
« Reply #40 on: July 15, 2018, 05:43:56 pm »

The uselessness of scanner profiles

I figured it was time to circle back to this thread. This was my first posting to this forum, out of frustration and desperation after months of trying to figure out what I was doing wrong. Then I took a step back and made sure my fundamentals were correct before continuing on. For anyone that's interested, I took what I learned and all my trial and error techniques and finally put together a concise analysis of some of my scanner profiles. I'm sure to the pros out there that this is nothing new, but I found it to be an extremely clear way of showing the problem I'm facing and might help some other amateur like me not waste too much of their time.

Applying an ICC profile back to the scanned image that created the profile

Let's start with the classic IT8 target that's bundled with a lot of higher-up scanner packages. I have three of them, one from LaserSoft, two from Wolf Faust (one Fuji, one on Kodak). They all give me the same results FYI. This example uses one from Wolf:


(Click for larger image.) On the left are the RGB values of the scanned target using Epson Scan with No Color Management. (I sampled the RGB values from the scanned image as a color list.) I created an ICC profile using i1Studio and applied it to the RGB values with an absolute rendering intent, resulting in the LAB values in the next column. Moving on are the LAB values from the IT8 reference file provided with the target. The final column are the dE2000 values between the profiled image and the reference values. Green are <= 1dE, yellow up to 2.5dE, etc.

Here's the similar results from a ColorChecker 24 and a Digital ColorChecker SG. (Note that I made my own reference files because there were a few significant deviations between my targets and the bundled i1Studio reference files.)

All this does is confirm that the profiling software can correct for the scanned target itself, but that's something. Presumably, that means my equipment and software are working and that my workflow isn't just wrong.


Applying a profile from one target to another

This basically fails. I'll share two that I thought were interesting because they kinda backup my original observations with the color shifts.

First, a scanned IT8 with a profile from the DCCSG. There are obvious color shifts in the greyscale patches.


Second, the DCCSG with the IT8 profile. It's harder to see but the greys and blacks took on a greenish cast. The LAB values would seem to confirm it. Plus the dE values are way out there anyway.


So, I'm sure in hindsight this should be obvious. All three targets I'm working with have different spectral characteristics. This leads to my prints…

Applying the profiles to a printed photo

I took an image of the ColorChecker 24 from BabelColor that was already in LAB colorspace and printed it using an absolute intent to a custom paper profile also made with the i1Studio. I measured the print with my i1Studio spectrophotometer and did the same tests. The only profile that did a good job was a custom target of 962 patches that I made with Argyll on the same paper. The other targets weren't good enough.

The ColorChecker profile couldn't even correct the printed ColorChecker:


Custom Argyll target:

For all I know this could be even better if I made sure to have the same print settings.

Conclusion of the testing

So this seems to prove what was said before by Doug, Graeme, and others that are much more knowledgeable about this stuff. But I'm happy that I proved it to myself. That said, now I'm kinda annoyed. Scanner profiling is marketed as a simple solution to getting your colors right and it's clearly not. I'd feel ripped off if it weren't for the fact that the i1Studio has worked for print profiling and display profiling. Glad I didn't spend big bucks on third party scanner profiling software.  I did spend a ton of time with Argyll and got similar results to the i1Studio in the end. But Argyll is awesome and I couldn't have done these tests without the other tools it provides like its spotread tool – because the i1Studio doesn't offer that capability.  You can see all the test results here.
Logged

BrianToth

  • Newbie
  • *
  • Offline Offline
  • Posts: 43
Re: Green/blue color cast with scanner profiles
« Reply #41 on: July 15, 2018, 06:24:51 pm »

I maybe be naďve, but how can scanners be expected to be profiled into yielding accurate color?  For example, here is the light in my Epson V600 scanner:



If this can be profiled into giving accurate color, then why are we wasting time searching for lights that provide accurate color rendering?  Why don't we just grab whatever LED bulbs are in our local hardware store and profile our cameras just like scanners are profiled?

Sorry if this is an elementary question but I've had cognitive dissonance ever since I measured my V600's LED.


I forgot that you posted this, but for the last few months my assumption was that the color shifts I was seeing had to be influenced somehow by the LEDs. Doug made a post regarding the flare of his v850 affecting his profiles, and other topics I read seemed to indicate this could be an issue.

The past couple weeks I've been reading about going the copy stand route of photographing my old photos. I started looking into lighting solutions and came across discussions of profiling and "spiky" LEDs, and better "full spectrum" lights.  Then I came across this webinar video hosted by Robin Myers and Doug Peterson that seemed to explain this topic very well.

Full spectrum high CRI light bulbs don't seem to be all that expensive, so why are the LEDs in the scanners so uneven? Would I be more likely to archive good reproductions using the camera method where I can control the lighting myself? Just some thoughts I've been having lately.

(Also, how were you able to analyze the light of your scanner? Did you just hold your spectrophotometer in place while the scanner light moved under it? I don't have an android device to use the ArgyllPRO app.)
Logged

BrianToth

  • Newbie
  • *
  • Offline Offline
  • Posts: 43
Re: Green/blue color cast with scanner profiles
« Reply #42 on: July 15, 2018, 06:29:53 pm »

If you are getting the same results with a range of different charts and profiling packages, then that points to either a workflow issue outside the profiling process, or a spectral mismatch between the media you are scanning and the charts. Lacking a spectral characterization of your scanner, the only way of guaranteeing optimal results using test charts and tri-stimulus profiling is to have test charts based on identical paper and colorants to the media you wish to scan.

Is it possible (for me) to create a spectral characterization of the scanner? When I read my targets with Argyll I see that it can include spectral measurements… not sure what to do with that info though. Is there a source for more information about this method? Thanks.
Logged

GWGill

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 595
  • Author of ArgyllCMS & ArgyllPRO ColorMeter
    • ArgyllCMS
Re: Green/blue color cast with scanner profiles
« Reply #43 on: July 15, 2018, 08:19:05 pm »

Is it possible (for me) to create a spectral characterization of the scanner? When I read my targets with Argyll I see that it can include spectral measurements… not sure what to do with that info though. Is there a source for more information about this method? Thanks.
Yes, but it's not at all easy. One method that has been used with variable success, is to use a test chart (or combination of different test charts) made with a range of independent colorants (so that the spectral characteristics are varied), and then try and figure out what scanner spectral response best explains the measured results. This is a rather under constrained problem, and various tricky mathematical/algorithmic methods are used to try and end up with a sensible result. Typically though the accuracy is moderate at best, and very poor at worst. If you go searching with the right keywords, you will find several dozen academic papers on this type of approach. I don't know of any successful commercial products though.

The other approach is more traditional - spectrally characterize the components of the scanner. Light source is relatively easy, but the sensor is not - the thorough approach is a monochrometer used with a broad band light source to scan through the spectrum and measure the sensor output. In theory the manufacturers of the scanners should have done that, but they don't seem to publish it, not will every example of a particular scanner be the same.

There are other approaches that could be used, but they also rely on some special equipment - i.e. a scan of a linear variable bandpass interference filter. Such filters are not cheap.
Logged

Doug Gray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2173
Re: Green/blue color cast with scanner profiles
« Reply #44 on: July 15, 2018, 08:20:32 pm »

Is it possible (for me) to create a spectral characterization of the scanner? When I read my targets with Argyll I see that it can include spectral measurements… not sure what to do with that info though. Is there a source for more information about this method? Thanks.

No. Not possible. Further, there is no way to tell what the color filter spectral response used by the scanners is. Perhaps they have been modified to sort of invert the LED spectral bumps and produce something more D50 like: However, I doubt it.

My results are somewhat similar. Interestingly, using the scanner profile I generated from a large printed target produces the almost the same average dE00s as that that built in to the V850 scanner and the custom profile generated from the IT8 target with I1PRofiler.

There is a slight improvement when the  reflectance glare is removed.

XRite "Classic" Colorchecker scanned v I1Pro2 measured
dE00:  2.91,  Epson Standard, ICC mode
dE00:  3.00,  Scanned using I1P IT8 profile, ICC mode
dE00:  3.01,  Scanned using 1914 printer patch profile
dE00:  2.74,  Scanned using 1914 printer patch profile, refl corrected

Note that the reflectance factor is quite small compared to my tests where patches are either surrounded by black or white.

Also, I believe the good results scanning the same IT8 image used to create a profile overstates the actual, even when considering that the IT8 is unchanged spectrally. The effects of reflected light will be the same. The way to truly test this is to do a scan with randomly re-arranged IY8 patches. I would expect them to have significantly higher errors. I see this using printed targets where I can easily print the same patches re-arranged.
Logged

Mark D Segal

  • Contributor
  • Sr. Member
  • *
  • Offline Offline
  • Posts: 12512
    • http://www.markdsegal.com
Re: Green/blue color cast with scanner profiles
« Reply #45 on: July 15, 2018, 08:48:18 pm »

I have completed a substantial amount of research about exactly this set of issues and the article is complete, awaiting final comment from several peer reviewers before publication on this website. I expect you'll see it within the next two weeks. So I won't say much here just now, except to note that I too have round-tripped the scanner profiling targets as very much like Brian did as PART of the verification routine. While the profile is built from these patches, that is not the main reason for using them in a round-trip test. There are three other reasons: firstly, we have the file reference numbers for those patches from which to calculate dE values, secondly they are most likely to be within the gamut of the profiles they help to create and thirdly these reference values are also used by the profiling software vendors' own internal validation routines, allowing us to compare outcomes between their tests and ours - to some extent.

I think the results Brian got in the CTP round-trips using the profiling targets are very good. I also think like him that the other results he got leave a fair bit to be desired, so the question is why. There could be a number of reasons. One of the more important ones may be that a large difference between the profiling target media and the profile testing media could create differences of reflectance that create larger dE issues. Another could be the fidelity of the reference values for the test media.

It is also necessary to break the analysis and the discussion into at least two components: one being reflective targets and the other transmissive. While the principles of profile verification are essentially the same, there are differences in the details of application largely having to do with reference values and how they are derived.

Finally, I have observed that scanner profiling is more finicky than printer profiling. More can go wrong and the outcomes tend to show less accuracy. The real issues are the sizes of the inaccuracies, how easily they can be compensated in a post scan editing workflow and whether they really matter to the acceptability of the final results. The smoothness of tonal transitions is something I pay a fair bit of attention to, because while several dE of colour inaccuracy may not amount to a big deal, jagged movements up a tone curve could be an altogether different matter, depending on their size and shape.

Stay tuned.
Logged
Mark D Segal (formerly MarkDS)
Author: "Scanning Workflows with SilverFast 8....."

Doug Gray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2173
Re: Green/blue color cast with scanner profiles
« Reply #46 on: July 15, 2018, 09:36:48 pm »

Yes, but it's not at all easy. One method that has been used with variable success, is to use a test chart (or combination of different test charts) made with a range of independent colorants (so that the spectral characteristics are varied), and then try and figure out what scanner spectral response best explains the measured results. This is a rather under constrained problem, and various tricky mathematical/algorithmic methods are used to try and end up with a sensible result. Typically though the accuracy is moderate at best, and very poor at worst. If you go searching with the right keywords, you will find several dozen academic papers on this type of approach. I don't know of any successful commercial products though.

The other approach is more traditional - spectrally characterize the components of the scanner. Light source is relatively easy, but the sensor is not - the thorough approach is a monochrometer used with a broad band light source to scan through the spectrum and measure the sensor output. In theory the manufacturers of the scanners should have done that, but they don't seem to publish it, not will every example of a particular scanner be the same.
Taking apart a scanner and measuring it's spectral response would seem to be more straightforward and more accurate. I'm surprised there are as many studies on this (various spectrally reflectance colors) as you indicate Graeme. Has their been a change since "white" LEDs became ubiquitous? And any idea how that has effected what color filters are used in scanners for their RGB response?

And what about the issue of large area flare crosstalk? This, at least, is something that can be corrected from the scanned RGB values unlike spectral metameric failure.
Quote

There are other approaches that could be used, but they also rely on some special equipment - i.e. a scan of a linear variable bandpass interference filter. Such filters are not cheap.
Logged

Doug Gray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2173
Re: Green/blue color cast with scanner profiles
« Reply #47 on: July 15, 2018, 10:10:43 pm »

I have completed a substantial amount of research about exactly this set of issues and the article is complete, awaiting final comment from several peer reviewers before publication on this website. I expect you'll see it within the next two weeks. So I won't say much here just now, except to note that I too have round-tripped the scanner profiling targets as very much like Brian did as PART of the verification routine. While the profile is built from these patches, that is not the main reason for using them in a round-trip test. There are three other reasons: firstly, we have the file reference numbers for those patches from which to calculate dE values, secondly they are most likely to be within the gamut of the profiles they help to create and thirdly these reference values are also used by the profiling software vendors' own internal validation routines, allowing us to compare outcomes between their tests and ours - to some extent.

I think the results Brian got in the CTP round-trips using the profiling targets are very good. I also think like him that the other results he got leave a fair bit to be desired, so the question is why. There could be a number of reasons. One of the more important ones may be that a large difference between the profiling target media and the profile testing media could create differences of reflectance that create larger dE issues. Another could be the fidelity of the reference values for the test media.

It is also necessary to break the analysis and the discussion into at least two components: one being reflective targets and the other transmissive. While the principles of profile verification are essentially the same, there are differences in the details of application largely having to do with reference values and how they are derived.

Finally, I have observed that scanner profiling is more finicky than printer profiling. More can go wrong and the outcomes tend to show less accuracy. The real issues are the sizes of the inaccuracies, how easily they can be compensated in a post scan editing workflow and whether they really matter to the acceptability of the final results. The smoothness of tonal transitions is something I pay a fair bit of attention to, because while several dE of colour inaccuracy may not amount to a big deal, jagged movements up a tone curve could be an altogether different matter, depending on their size and shape.

Stay tuned.
Yup. That sort of thing is far worse than larger dE errors that occur smoothly. The reflected light errors do not produce abrupt, jagged color changes and are much less objectionable, if even visually noticeable, than things like color shift on darker neutrals compared to lighter neutrals.

The larger errors associated with reflected light errors (wide area crosstalk) materially impact the use of a scanner to calibrate printers or measure density but have much less effect on the visual appearance of scans. Reflected light affects widely separated colors much more than adjacent color patches and it's much harder to see dE differences when widely separated. And it has no impact on the smoothness of color transitions.
Logged

GWGill

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 595
  • Author of ArgyllCMS & ArgyllPRO ColorMeter
    • ArgyllCMS
Re: Green/blue color cast with scanner profiles
« Reply #48 on: July 16, 2018, 04:17:05 am »

I'm surprised there are as many studies on this (various spectrally reflectance colors) as you indicate Graeme.
Most such papers relate to spectrally characterizing cameras using test charts. Naturally this is applicable to scanners too (assuming that it's possible to measure the scanner light source.)
Logged

Doug Gray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2173
Re: Green/blue color cast with scanner profiles
« Reply #49 on: July 16, 2018, 07:53:24 pm »

BrianToth,

I just ran similar experiments on my CCSG. Measured with an I1Pro2.

The V850 had a very similar shift in the neutrals with the b* averaging about -5. using the IT8 chart profiled by I1Profiler.

If I used the profile generated by Argyll from my printed patch set the average of the neutrals decreased a bit but was still blue shifted with the b* running about -4.

Interesting in that scanning neutral colors on printed patches produces very good results with a dE less than 1 on both the colored  and neutral printed patches.

It's pretty clear that the V850 LED and whatever color filters  they use are a long, long way from L/I.

Also, and rather curiously, the printed patch scanner profile is better than the IT8 scanner profile when processing a scanned CCSG though both show significant metameric error.

Presumably the IT8 profile would work best when scanning things like color prints on Kodak paper, specifically the Endura paper the IT8 uses. But I have no way to verify that since one would need an independent color patch set different from the IT8 but on the same paper.

On the positive side, I can scan my prints, fixed for reflection, and get very good results when using my printed patch set and Argyll. Colors match with an average dE00 of well under 1.  But if I scan anything else, things are off. Especially neutrals which is interesting since actual CC neutrals are quite spectrally smooth, So the error is in the printer's neutrals which are just lumpy enough when combined with the really lumpy scanner LED to produce a material shift. Interesting the same, slightly worse, shift occurs with the IT8 profiles.
« Last Edit: July 16, 2018, 07:59:45 pm by Doug Gray »
Logged

Doug Gray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2173
Re: Green/blue color cast with scanner profiles
« Reply #50 on: August 12, 2018, 02:18:44 am »

I ran a comparison of 3 scanner profiles against a measured, printed set of patches completely distinct from the patches used to create the profiles. The profiles were an IT8 I1Profiler profile, and Argyll 2871 patch set profiles The test patches measured with the three profiles were a set of 957 colors distributed evenly across the printer's gamut. The IT8 based profile using I1Profiler was the first test and showed an average dE00 of 2, which is not too bad. Especially considering the spectral characteristics of the printer and IT8 chart are fairly different. Also it's only a few hundred patches and read with a different spectrophotometer than the printed patches.

IT8 chart supplied with V850 and used with I1Profiler to create a profile:
mean dE00:  2.02, mean dE00 best 90%:  1.79  mean dE00 worst 10%:  4.09  Max: 6.22

The expanded large, Argyll patch set performed quite well, dropping the average dE00 below 1.
mean dE00:  0.83, mean dE00 best 90%:  0.72  mean dE00 worst 10%:  1.83  Max: 3.69

Modeling and removing the internal scanner reflection crosstalk with scannerreflfix.exe improved things further.
mean dE00:  0.47, mean dE00 best 90%:  0.39  mean dE00 worst 10%:  1.14  Max: 3.33


The errors associated with scanner internal reflections here are much lower than in the specialized tests where a color was surrounded by a larger area of white or black. That can introduce errors of up to about 6 L*. on near whites.

Reflected light errors, on average, are normally low since most patches have a reflectivity of only about 20% and error is proportional to the % of light reflected and most things are averaged by the re-reflected light from surrounding patches. Errors associated with the additional uncertainty from re-reflected light can be reduced somewhat by using a large number of patches. The bumpiness gets better averaged.

Re-reflected light error can not be corrected using a profile and must be removed with a separate process that estimates the re-reflected light's contributions and removes them.


Update: Ran the same three measurements using Baryta luster paper with very little OBAs. The above data was from Costco Glossy which has very large amounts of OBAs with b* running almost -14 when read with M1. However, all my work, and the above measurements, is done with M2 which eliminates the effect of uV and results in very close agreement with non OBA containing media.

mean dE00:  2.32, mean dE00 best 90%:  2.09  mean dE00 worst 10%:  4.36  Max: 6.35 - using IT8 I1Profiler Profile
mean dE00:  0.74, mean dE00 best 90%:  0.64  mean dE00 worst 10%:  1.68  Max: 3.66 - using 2871 patch printed chart to profile scanner
mean dE00:  0.56, mean dE00 best 90%:  0.48  mean dE00 worst 10%:  1.29  Max: 2.91 - as above but corrected for re-reflection

Conclusion: The V850 scanner illuminant does not contain uV. Consequently, all measurements or self constructed charts such as printed charts used to create scanner profiles should either have no OBAs or be scanned by a spectro using M2 (uV cut).
« Last Edit: August 12, 2018, 10:56:59 pm by Doug Gray »
Logged
Pages: 1 2 [3]   Go Up