Pages: 1 2 [3] 4 5 ... 7   Go Down

Author Topic: Generating a Kodachrome profile from an IT8 target  (Read 52839 times)

crames

  • Full Member
  • ***
  • Offline Offline
  • Posts: 210
    • http://sites.google.com/site/clifframes/
Re: Generating a Kodachrome profile from an IT8 target
« Reply #40 on: May 21, 2011, 12:42:39 am »

I'll upload some of the reference scans in 16-bit if you want to play around.

Try as I might, I cannot get a Gamma 1.0 scan of a high-contrast slide to look as good in the shadows as a Gamma 1.8 scan. And in my experience, it's the shadow area that is very important for a slide to look good on screen. Whether it's my scanner or Argyll I don't know. Will post the results for all to see as soon as I am confident of the results. There's no particular reason I chose gamma 1.8. It was just one of a range of values I was experimenting with.

Please upload them. I am wondering about the defects you are seeing, as I don't have a problem with linear scans.

Quote
I've been wondering if there is an optimum gamma (other than 1), so I've been playing around with some equations to work out how closely a gamma curve matches the L* curve. A close match would mean that when profiling the scan in Lab coordinates, the errors would be minimized. I have attached a plot showing the gamma 2.2 and the L* function, and a third curve showing the errors between the two. By plugging in a range of gamma values, it turns out that the optimum match between gamma and L* occurs around gamma 2.5.

There is validity to that approach, but I'm not sure how it applies for different kinds of Argyl/Coca generated profiles. It might not help a measurement-based 3D look-up table type of profile, but it could help a matrix-shaper profile. See the discussion in the Hardeberg thesis: Acquisition and reproduction of colour images, Chapter 3 (3.2.3.5), where he reduced perceptual error by using an exponent of 1/3 on the scanner values.
Logged
Cliff

crames

  • Full Member
  • ***
  • Offline Offline
  • Posts: 210
    • http://sites.google.com/site/clifframes/
Re: Generating a Kodachrome profile from an IT8 target
« Reply #41 on: May 21, 2011, 02:18:46 am »

Guy,

Since you are experimenting, here is a profile you might want to try:

https://sites.google.com/site/cliffpicsmisc/home/profiles/KChromeCoolscanT3250.icm?attredirects=0&d=1

It was made with Coca using your target scan (16-bit gamma 1), but using a Q60 reference file that I modified to have a tungsten 3250K white instead of the usual D50.

To use it, assign it to a Kodachrome scan, then convert to a working space of your choice using the ABSOLUTE COLORIMETRIC INTENT. This should give you the Kodachrome scan as though illuminated with tungsten projector light.

Of course, you should view the image in a dark surround to complete the simulation. If you project the image with a video projector, I wonder how close it will look to the actual projected slide?
Logged
Cliff

guyburns

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 99
Re: Generating a Kodachrome profile from an IT8 target
« Reply #42 on: May 22, 2011, 07:16:56 am »

There is validity to that approach, but I'm not sure how it applies for different kinds of Argyl/Coca generated profiles. It might not help a measurement-based 3D look-up table type of profile, but it could help a matrix-shaper profile. See the discussion in the Hardeberg thesis: Acquisition and reproduction of colour images, Chapter 3 (3.2.3.5), where he reduced perceptual error by using an exponent of 1/3 on the scanner values.

Cliff: if you keep giving me articles to read, I'll never finish testing. The Hardeberg thesis is very informative. He modified the linear RGB values from the scanner by applying an exponent of 0.33, trying to approximate the relationship between L* and Y/Yn which has the same exponent. But a closer correlation comes at ~0.4 (gamma = 2.5) if the curves I have generated are correct (see attachment). Maybe there are good reasons why he chose 3.0 instead of 2.4.

For my own reference I have summarised Hardeberg's Chapter 3, in which he explains how he went about generating a profile from an IT8 target. Page references are to Hardeberg, but in what follows there are no quotations.

Summary
Hardeberg found the most accurate method of generating a profile by the methods he tested was to:

1. Linearise the RGB scanner values;
2. Apply gamma correction of 1/3 to the RGB values; and
3. Transform via a third-order polynomial to Lab space. [p51]

That mouthful means:

1 – Before the real processing begins, it is best to ensure the RGB data from the scanner is linear in case of non-linearities caused by stray light, sensor "dark" current, gamma correction of unknown amount applied by the scanning software, and other non-linearities. However, for the two devices tested by Hardeberg, no linearisation was required by one, and only "a correction for the black offset" was required by the other. It seems doubtful to me that any software could attempt to linearise raw RGB output without damaging the data – unless for a very simple case like "black offset". In practice, linearising the RGB data is of no concern (and is not feasible for most users anyway).

It seems to me the main reason Hardeberg includes this step is to make sure that gamma correction hasn't already been applied. i.e. you don't want to use gamma corrected RGB values as input for steps 2 and 3, because his step 2 applies gamma correction. So with his method of profiling, the data must start as linear as possible, otherwise you'll end up with gamma correction upon gamma correction.

I wonder if this is where the idea started that profilers should have linear input data? Yes, profilers do require linear RGB data – if your next step is to apply gamma of 1/3 (as does Hardeberg), to ensure the most accurate profiles in Lab space. But if the profiling software does not apply gamma correction to the RGB data, then Hardeberg's results clearly show that for the most accurate profiles, the RGB data should be gamma corrected before it enters the profiler.

2&3 – If you are transforming from one data set to another, you will get the most accurate results if the two sets start off as close as possible. That means, if your RGB data is linear, you should use the XYZ colour space for your transformation when profiling, because the Y component of XYZ is linear. The problem is that, as Hardeberg explains, the XYZ colour space is not linear with respect to colour, but only with respect to luminance (if I understand him correctly). So although it might appear to be best to use linear RGB values as input to an XYZ profiler, because of the way humans see colour, such profiling (according to Herdeberg's results) won't be as good as transforming to an Lab space – which is reasonably linear with respect to colour.

But… the Lab colour space is not linear with respect to absolute luminance, so it is not linear with respect to scanner RGB values. And transforming between the two will result in errors. Let's put some values into this discussion, in particular, some of the Grayscale values GS0-GS23 (first figure is relative luminance as measured by a linear RGB scanner, the second the L* value of the IT8 target):

73 –>    88
65 –>    84
59 –>    81
  3 –>    20
  1.2 –> 10
  0.3 –> 2.8

The transformer has to work out an equation that relates the two sets of figures. If it is expecting a cubic relationship, or models the relationship cubicly, the fit will be quite good. The fit will be even better if the profiler is presented with this set of numbers (I just made them up) which are derived from the first set by gamma correcting:

89 –>    88
83 –>    84
81 –>    81
21 –>    20
  9.8 –> 10
  2.4 –> 2.8

Transforming the second set will result in better accuracy than the first. This is what Hardeberg has shown.

So, if you know your profiler works in Lab space and applies gamma correction before profiling – then it is best to present it with linear data. However, if it works in Lab space and does not apply gamma correction, then the profile will be more accurate if you massage the RGB figures so that they are close to the Lab numbers to start with. But how does a user know how the internals of their profiler works?

This is all in theory. I haven't seen much colour difference between any of the dozens of targets I scanned, then profiled using the four settings in Coca. But I have seen luminance differences in the deepest blacks of a profiled slide. And that just happens to be one of the areas in which I want the best rendition of my slides - the deep, moody blacks.

Gamma correction has been shown by Hardeberg to theoretically give better results for his test conditions (though there is not much difference between Methods 1 and 3, see Results below). And I've seen an improvement in some of my slides when I gamma correct a target, but I'm not convinced yet how real the improvement is.


Profile Methods tested by Hardeberg
To generate profiles, Hardeberg used different IT8 charts, with and without preliminary gamma correction of scanner RGB values, and three polynomial orders (linear, square and cubic) for the RGB/XYZ and RGB/Lab transform. He tried three general methods:

1. Transforms (first, second and third order) to XYZ space
2. Transforms (first, second and third order) to Lab space
3. Gamma correction of RGB data, followed by polynomial transforms to Lab space.

The main drawback with method 1 is that the XYZ colour space is very poorly correlated to visual colour differences. A transform directly to Lab values (method 2), instead of XYZ values, should give better results since distance in CIELAB space corresponds quite well to perceptual colour differences.

Method 3 involved an exponent of 1/3 (identical to a gamma correction of 1/3) being applied to the scanner RGB values before the transformation to Lab space. The use of cubic root was motivated from considering the Lab transformations, which involves cubic-root functions on the linear XYZ tristimulus values. [p44-46]

Results
Table 3.1 shows that method 3 is the best solution for generating a profile with minimum errors, as measured by the mean colour differences. Figures below show first, second and third order results:

Method 1…   5.1, 2.1, 1.6
Method 2… 19.4, 7.3, 4.6
Method 3…   5.5, 1.4, 1.0

From the testing results, transforming directly from linear RGB values to Lab is not a good solution, whereas applying a gamma correction of 1/3 gives very good results, especially when third-order polynomials are used. [p46-47].
« Last Edit: May 22, 2011, 07:23:49 am by guyburns »
Logged

crames

  • Full Member
  • ***
  • Offline Offline
  • Posts: 210
    • http://sites.google.com/site/clifframes/
Re: Generating a Kodachrome profile from an IT8 target
« Reply #43 on: May 22, 2011, 11:54:09 am »

Cliff: if you keep giving me articles to read, I'll never finish testing.

I thought you might like that one! ;D

Quote
The Hardeberg thesis is very informative. He modified the linear RGB values from the scanner by applying an exponent of 0.33, trying to approximate the relationship between L* and Y/Yn which has the same exponent. But a closer correlation comes at ~0.4 (gamma = 2.5) if the curves I have generated are correct (see attachment). Maybe there are good reasons why he chose 3.0 instead of 2.4.

Sorry, another article but it's short. Bruce Lindbloom did an analysis to find the best gamma to approximate L* http://www.brucelindbloom.com/index.html?CompandCalcHelp.html#BestGammaForLab and concluded:

Quote
If you go to the trouble of mathematically finding the "best" gamma, you will find that its value depends on your definition of "best."

    If best means minimizing the largest error, then the best gamma is 2.1723.
    If best means minimizing the RMS (root mean square) error, then the best gamma is 2.3243.

and settled on 2.2 as a good compromise.

Quote
For my own reference I have summarised Hardeberg's Chapter 3, in which he explains how he went about generating a profile from an IT8 target. Page references are to Hardeberg, but in what follows there are no quotations.

An excellent summary, Guy.

I wonder how much gamma actually effects the accuracy of the profiles generated by Coca and others. It would help to have a way to evaluate the accuracy of profiles made with all the different methods. There was a handy profile checker included with Little CMS http://www.littlecms.com/ which has been spun off into LPROF http://lprof.sourceforge.net/. It gives dE and other color errors for each target patch. Argyl has a command line profcheck command http://www.argyllcms.com/doc/profcheck.html.

In my (limited) experience, most profilers will give low errors of dE 1 or less throughout most of the scanner range. But, no surprise, the largest errors, particularly with Kodachrome are in the highest densities. The Dmax and lowest gray-scale patches are not well resolved by the desktop scanners we are working with. Varying the gamma, it seems to me, is not going to help the profiling software to linearized these darkest tones. If anything, gamma is going to push them even closer together, making them even less distinguishable? Maybe this results in more random dithering of the dark tones when passed through the profile, the random noise appearing smoother?

Anyway, one profiling package I used, SCARSE http://www.scarse.org/ seemed to give smoother dark tones from Kodachrome. It's author Andrei Frolov told me that he put extra effort into keeping shadow detail intact, and tested extensively with Kodachrome while developing the last version 0.4.

So all profiling software is not created equal. When it comes to Kodachrome, the extra-high densities and non-neutral gray-scale put extra demands on the profilers.

Still itching to get a look at your 16-bit reference scans, since I still haven't actually seen the problem.



Logged
Cliff

joofa

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 544
Re: Generating a Kodachrome profile from an IT8 target
« Reply #44 on: May 22, 2011, 01:27:50 pm »

the XYZ colour space is very poorly correlated to visual colour differences

I think early colorimeters were quick to deduce this conclusion  - a practise that is continuing to this day.

Joofa
Logged
Joofa
http://www.djjoofa.com
Download Photoshop and After Effects plugins

guyburns

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 99
Re: Generating a Kodachrome profile from an IT8 target
« Reply #45 on: May 22, 2011, 10:47:14 pm »

Cliff: I'll answer your last few posts in a day or two. But I'll have to read that new article first.

I haven't uploaded my reference scans yet. I want to make absolutely sure that the problem in the shadows is real and is not some artifact of my monitor, eyeballs, room lighting, light entering the scanner or other variables. So I intend rescanning the IT8 target and reference slides at gammas 1.0, 1.5, 2.0, 2.2, 2.4 – then I'll upload.

Another link to varying gamma while scanning a target: http://lprof.sourceforge.net/help/lprof-help.html. Apart from the hyperbole "huge loss of details in shadows", it seems as if the author knows his stuff.

Gamma: On most scanners you can select the gamma to be used for scanning the image. In general you should use a gamma between 2.2 and 3.0. A Gamma 2.2 has the additional benefit of being close to the sRGB gamma, and this means the uncorrected Image will "look nice" on an "average" monitor. It is also near to perceptual gamma. Gamma 2.4 has the additional benefit of being closest to perceptual space, and this is a very good reason to use this value. Less that 2.2 (and of course the infamous 1.0) can generate huge loss of detail in shadows, only to give a slight bettering of highlights. Don't use this unless your are using 16 bits per sample, and even in such case, don't do it unless you know what are you doing! Gammas around 2.4 are best for flat bed scanners and film scanners with limited dynamic range.  With high dynamic range film scanners values closer to 3.0 may be best.  Hutch Color, for example, recommends a gamma of 2.8 for high dynamic range scanners. But for flat bed scanners more than 2.4 (up to 3.0) looses some highlight detail with no gains in shadow detail.
Logged

guyburns

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 99
Re: Generating a Kodachrome profile from an IT8 target
« Reply #46 on: May 23, 2011, 12:15:00 am »

Cliff: I like the Bruce Lindbloom site. Wonderful stuff. I can't image any colour site more authoratative that his. I was wondering how Bruce RGB got its name (it's one of the options in my NikonScan software), and I've also been wondering why there was a discontinuity in the L* function at L* = 8 which I noticed when I was plotting derivatives. As Bruce explains at length, http://www.brucelindbloom.com/index.html?LContinuity.html, it's because the figure of 903.3 (which I took from Hunt) should actually be  24389/27.

I'm also wondering (I do a lot of wondering) whether sRGB is a better fit to the L* function than a pure gamma, but Bruce's calculator doesn't allow me to plug in sRGB.

Talking about sRGB, since all my Kodachrome scans will only be viewed digitally (no printing envisaged) via BluRay at home or the local cinema, I'm assuming the best colour space for editing would be sRGB – so as not to run into the problem of out-of-gamut colours.
Logged

crames

  • Full Member
  • ***
  • Offline Offline
  • Posts: 210
    • http://sites.google.com/site/clifframes/
Re: Generating a Kodachrome profile from an IT8 target
« Reply #47 on: May 23, 2011, 01:50:07 pm »

Another link to varying gamma while scanning a target: http://lprof.sourceforge.net/help/lprof-help.html. Apart from the hyperbole "huge loss of details in shadows", it seems as if the author knows his stuff.

Gamma: On most scanners you can select the gamma to be used for scanning the image. In general you should use a gamma between 2.2 and 3.0. A Gamma 2.2 has the additional benefit of being close to the sRGB gamma, and this means the uncorrected Image will "look nice" on an "average" monitor. It is also near to perceptual gamma. Gamma 2.4 has the additional benefit of being closest to perceptual space, and this is a very good reason to use this value. Less that 2.2 (and of course the infamous 1.0) can generate huge loss of detail in shadows, only to give a slight bettering of highlights. Don't use this unless your are using 16 bits per sample, and even in such case, don't do it unless you know what are you doing! Gammas around 2.4 are best for flat bed scanners and film scanners with limited dynamic range.  With high dynamic range film scanners values closer to 3.0 may be best.  Hutch Color, for example, recommends a gamma of 2.8 for high dynamic range scanners. But for flat bed scanners more than 2.4 (up to 3.0) looses some highlight detail with no gains in shadow detail.

Like some of the previous references, this advice is geared towards optimizing 8-bit scans. Note the relaxation of the gamma requirement when using 16-bits.
Logged
Cliff

crames

  • Full Member
  • ***
  • Offline Offline
  • Posts: 210
    • http://sites.google.com/site/clifframes/
Re: Generating a Kodachrome profile from an IT8 target
« Reply #48 on: May 23, 2011, 02:16:44 pm »

Cliff: I like the Bruce Lindbloom site. Wonderful stuff. I can't image any colour site more authoratative that his. I was wondering how Bruce RGB got its name (it's one of the options in my NikonScan software), and I've also been wondering why there was a discontinuity in the L* function at L* = 8 which I noticed when I was plotting derivatives. As Bruce explains at length, http://www.brucelindbloom.com/index.html?LContinuity.html, it's because the figure of 903.3 (which I took from Hunt) should actually be  24389/27.

Lindbloom's color space is BetaRGB. BruceRGB is a different space, created by Bruce Frasier.

Quote
I'm also wondering (I do a lot of wondering) whether sRGB is a better fit to the L* function than a pure gamma, but Bruce's calculator doesn't allow me to plug in sRGB.

It's true that both L* and sRGB use a 2-piece TRC, with a linear segment for the darkest tones, but I don't know the answer to your question.

Quote
Talking about sRGB, since all my Kodachrome scans will only be viewed digitally (no printing envisaged) via BluRay at home or the local cinema, I'm assuming the best colour space for editing would be sRGB – so as not to run into the problem of out-of-gamut colours.

I believe that Blu Ray uses "Rec. 709", which has the same white point and primaries as sRGB, but possibly a different tone curve (TRC).
Logged
Cliff

guyburns

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 99
Re: Generating a Kodachrome profile from an IT8 target
« Reply #49 on: May 23, 2011, 09:44:48 pm »

Lindbloom's color space is BetaRGB. BruceRGB is a different space, created by Bruce Frasier.

There are too many Bruces in the colour world. When I read somewhere that Lindbloom had created a colour space, I assumed it was Bruce RGB. Thanks for the correction.

Attached is a comparison of sRGB and L*, using the exact specifications for each. Of all the gamma-type curves, sRGB has the best approximation to L* in the shadows.
« Last Edit: May 24, 2011, 06:53:50 am by guyburns »
Logged

crames

  • Full Member
  • ***
  • Offline Offline
  • Posts: 210
    • http://sites.google.com/site/clifframes/
Re: Generating a Kodachrome profile from an IT8 target
« Reply #50 on: May 24, 2011, 04:07:41 am »

There are too many Bruces in the colour world.

 :D

Quote
Attached is a comparison of sRGB and L*, using the exact specifications for each. Of all the gamma-type curves, sRGB has the best approximation of L* in the shadows.

I think you might be on to something.

I made some test profiles using LPROF to see how a gamma 2.2 and an sRGB TRC would affect the errors compared to linear.

To apply the TRCs, I first made 2 variations of the standard sRGB profile, using the Photoshop Color Settings Custom Profile tool. The first is a profile with the sRGB white point and primaries, but a gamma 1.0. The second was the same but with a gamma 2.2 (which Photoshop automatically names "Simplified sRGB"). I opened the linear scan and assigned the sRGB gamma 1 profile, then Convert to Profile to the sRGB gamma 2.2 profile. I suppose that this resulted in only the gamma 2.2 TRC being applied, since the white point and primaries are the same. For the sRGB TRC I did the same, but converted to the standard sRGB profile.

Linear
MEAN_DE   0.479439
MAX_DE   7.25239
MIN_DE   0.0111697
STD_DE   0.791264
Lab dE:
GS18   0.85846
GS19   2.15729
GS20   2.73401
GS21   4.51088
GS22   6.33844
DMAX   7.25239

Gamma 2.2
MEAN_DE   0.475655
MAX_DE   6.95842
MIN_DE   0.00871406
STD_DE   0.762017
Lab dE:
GS18   0.89821
GS19   2.14864
GS20   2.50815
GS21   4.37009
GS22   6.36931
DMAX   6.95842

sRGB TRC
MEAN_DE   0.3986
MAX_DE   2.47964
MIN_DE   0.00954745
STD_DE   0.370155
Lab dE:
GS18   0.71167
GS19   1.86969
GS20   2.0874
GS21   1.0714
GS22   1.12813
DMAX   1.61505

As you can see, preconditioning the target scan with the sRGB TRC significantly improves the error, especially in the very darkest patches. Also, I would say that the application of gamma 2.2, alone, does not make much of a difference. The the linear segment is the key to the improvement.

There is a small problem though - in order to use the profile you first have to apply the sRGB TRC to the scan. Maybe Argyl, with it's almost infinite options, has a way to incorporate the preconditioning within the profile itself.

Definitely worth pursuing IMHO.
« Last Edit: May 24, 2011, 04:12:37 am by crames »
Logged
Cliff

guyburns

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 99
Re: Generating a Kodachrome profile from an IT8 target
« Reply #51 on: May 24, 2011, 07:37:48 am »

There is a small problem though - in order to use the profile you first have to apply the sRGB TRC to the scan. Maybe Argyl, with it's almost infinite options, has a way to incorporate the preconditioning within the profile itself.

Wouldn't scanning into sRGB space achieve the same thing?

QUES 1: Where did all those multi-decimal numbers in your post come from? Data analysis from within LPROF?

It's good to see the improvement where I was hoping it would be – in the shadows – but I now think the reason my shadows had poor detail has less to do with the profiling and more to do with the density range of my scanner or the quality of my target. For Gamma 1, there is barely a difference between GS20-23 (i.e. those patches are virtually "blocked"), but there is a reasonable difference when scanned at Gamma 1.8 and above. I can't explain that, because I assumed that for the higher gammas the scanner would derive them from the Gamma 1 figures. These are the mean RGB figures, measured in a small area of each patch in Photoshop > Histogram, Gamma 1 first, then Gamma 1.8:

GS19 … 2.25,  18.19
GS20 … 1.43,  14.82
GS21 … 1.32,  13.44
GS22 … 1.33,  12.24
GS23 … 1.33,  12.42

What intrigues me is – for my scanner, the Gamma 1.8 figures are not derived directly from Gamma 1.0. If they were, GS22 would be higher than GS21. This leads me to ask two more questions:

QUES 2: What scanner do you use?

QUES 3: What are the RGB values of GS20-23 patches for one of your IT8 scans using Gamma 1.0?

Logged

crames

  • Full Member
  • ***
  • Offline Offline
  • Posts: 210
    • http://sites.google.com/site/clifframes/
Re: Generating a Kodachrome profile from an IT8 target
« Reply #52 on: May 24, 2011, 08:31:51 am »

Wouldn't scanning into sRGB space achieve the same thing?

It would depend on the scanner software. It's possible that the scanner software would apply more than just the sRGB TRC, and also do a matrix transformation to the sRGB primaries as well as a chromatic adaptation to the sRGB white. I tried to avoid that extra manipulation and apply only the TRC.

Quote
QUES 1: Where did all those multi-decimal numbers in your post come from? Data analysis from within LPROF?

That's from the error report that LPROF conveniently generates after making a profile.

Quote
It's good to see the improvement where I was hoping it would be – in the shadows – but I now think the reason my shadows had poor detail has less to do with the profiling and more to do with the density range of my scanner or the quality of my target. For Gamma 1, there is barely a difference between GS20-23 (i.e. those patches are virtually "blocked"), but there is a reasonable difference when scanned at Gamma 1.8 and above. I can't explain that, because I assumed that for the higher gammas the scanner would derive them from the Gamma 1 figures. These are the mean RGB figures, measured in a small area of each patch in Photoshop > Histogram, Gamma 1 first, then Gamma 1.8:

GS19 … 2.25,  18.19
GS20 … 1.43,  14.82
GS21 … 1.32,  13.44
GS22 … 1.33,  12.24
GS23 … 1.33,  12.42

What intrigues me is – for my scanner, the Gamma 1.8 figures are not derived directly from Gamma 1.0. If they were, GS22 would be higher than GS21.

I think you are limiting your measurements to 8 bits because of the way Photoshop displays the histogram. If you Select a region of the target and Filter, Average, you can read the average in the Info window set to 16-bit for more precision. Then you can see the variation in the linear scan values at the lowest levels. Also, see the LPROF measurement reports below which use the full bit depth.

So I still think it is true to say the the Linear scan is the Mother of All Scans - all of the others are derived from it. Unfortunately profiling software is not necessarily able to take advantage of additional information that might be available in a linear scan, and have been optimized instead for gamma adjusted scans which must be the more common type.

Quote
This leads me to ask two more questions:

QUES 2: What scanner do you use?

I have a Polaroid Sprintscan 4000, which gives only 12-bits. But note that I'm not using my scans for this, I'm using the target scans you provided.

Quote
QUES 3: What are the RGB values of GS20-23 patches for one of your IT8 scans using Gamma 1.0?

I would have to dig that out for my scanner - it would be an interesting comparison. Here is what LPROF reports for your 16-bit linear and your 16-bit Gamma 1.8.

« Last Edit: May 24, 2011, 08:52:17 am by crames »
Logged
Cliff

crames

  • Full Member
  • ***
  • Offline Offline
  • Posts: 210
    • http://sites.google.com/site/clifframes/
Re: Generating a Kodachrome profile from an IT8 target
« Reply #53 on: May 25, 2011, 09:19:19 am »

Ok, for comparison here is data from my Sprintscan.

There is a problem with flare in the darkest patches, a lot of it due to the bright line separating GS22 and Dmax. This is after cleaning the mirror and lens.

Your Nikon also shows flare affecting those patches, but with less structure than the Polaroid. The advantage of the 2 extra bits in the Nikon gives you is also visible as better smoothness.

The flare is causing the DMAX patch to scan lighter in some of the channels than GS22. Interestingly, the Nikon has a bump in the red channel at GS22 that is not present in the Polaroid.

So the Kodachrome target is tougher to scan than it should be because of the lighter areas surrounding the darker GS gray scale patches. I think it's fair to assume that in real scans the darkest patches will be surrounded with dark patches of similar level, so that there would be less flare in those parts of the image. Or should I say that anomalies in the darkest tones caused by the profile will be more visible due to a lack of simultaneous-contrast effects - that puts more demands on the accuracy of the profile than we are getting because of flare in scanning the target.

An approach to improve the target scan and therefore, hopefully, the profile in the darkest tones would be to reduce the flare in the target. One way would be to overlay a mask on the target slide to block the lighter structures surrounding the patches. Seems extremely fiddly to make such a tiny mask, though. A more practical approach would be to measure the darkest patches by hand and editing the measurement file with flare-free estimates.

Time for more experiments - you better get right on it, Guy!

There are a million worms in this can you have opened.....

attached:

Polaroid Sprintscan GS and DMAX measurement snippet
Polaroid SS dark patch image, lightened
Nikon dark patches, lightened
Logged
Cliff

guyburns

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 99
Re: Generating a Kodachrome profile from an IT8 target
« Reply #54 on: May 25, 2011, 12:14:26 pm »

Thanks Cliff for all the data. I'll pour over it in the next few days. I've just about got this profiling to a level as good as it's going to get for my setup (Nikon Coolscan V ED, Kodachrome and Argyll via Coca). It's now fairly clear that blacks are a challenging area when profiling. There are several reasons for this in my setup:

• Light entering the scanner via the slide hole contributes a small amount of light to the deepest blacks. One measurement indicated about 0.2 RGB units (not yet double checked). I now cover the hole with black cloth when scanning, just in case.

• I have to use the Kodachrome setting because of ICE (dust removal). I'd rather not use it because the profiling removes Nikon's attempt at colour correction anyway. The problem is that the Kodachrome setting throws a light yellow wash over the entire slide, bleaching GS21, 22, and 23 to similar L* values (not yet double checked). I suspect that this yellow wash is applied after gamma is applied in NikonScan. This may explain some of the benefit of applying gamma – the yellow wash becomes a much smaller percentage of the higher gamma-corrected values of GS21-23. Instead of being similar, they are spread apart, allowing Argyll the space to match them to the target values.

• Human visual sensitivity (the L* curve) for shadows is higher than for highlights. Blacks have to be scanned much more finely.

• Kodachrome (and other slide types?) throws an additional burden on the scanner because the Density/Exposure curve exhibits a gamma of ~1.5, throwing the shadows deeper into darkness. Although this doesn't directly affect profiling accuracy, this increased gamma means the histogram of a typical slide will be weighted towards the black end (quite heavily in many cases) – and these shadow areas will require their levels stretched apart (increased contrast) to bring them into the light. Stretching the shadows without introducing errors, requires scanner and profile to have better accuracy that the L* curve alone would indicate.

• For GS20-23 scanned with low gammas (1.0 -?), my scanning/profiling process is not accurately transforming the scanner RGB values into IT8 L* values. I suspect these is an optimum value of gamma for my setup of around 1.8. To test this, I'll have to scan the target at various gammas to see which gives the closest match.
Logged

guyburns

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 99
Re: Generating a Kodachrome profile from an IT8 target
« Reply #55 on: May 25, 2011, 12:37:20 pm »

That white streak between GS22 and 23 has been annoying me for a while. To fix the problem of the streak casting a flare on GS22 and 23, would there be any problem with working out the average L* in a representative small area in the centre of each, and then filling the whole of both patches with that average value?
Logged

crames

  • Full Member
  • ***
  • Offline Offline
  • Posts: 210
    • http://sites.google.com/site/clifframes/
Re: Generating a Kodachrome profile from an IT8 target
« Reply #56 on: May 25, 2011, 09:13:18 pm »

• For GS20-23 scanned with low gammas (1.0 -?), my scanning/profiling process is not accurately transforming the scanner RGB values into IT8 L* values. I suspect these is an optimum value of gamma for my setup of around 1.8. To test this, I'll have to scan the target at various gammas to see which gives the closest match.

It was your target scan that was used in the above post that showed error results for those patches. With the sRGB TRC, respectable DE errors of 2 or less are achievable in those patches, with your equipment.

Are you still using the 8-bit target scans to make profiles?
Logged
Cliff

crames

  • Full Member
  • ***
  • Offline Offline
  • Posts: 210
    • http://sites.google.com/site/clifframes/
Re: Generating a Kodachrome profile from an IT8 target
« Reply #57 on: May 25, 2011, 09:16:55 pm »

That white streak between GS22 and 23 has been annoying me for a while. To fix the problem of the streak casting a flare on GS22 and 23, would there be any problem with working out the average L* in a representative small area in the centre of each, and then filling the whole of both patches with that average value?

That could work. Maybe use a value with an amount of flare subtracted from the average.

Or paint over the white streak with some ink.
Logged
Cliff

guyburns

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 99
Re: Generating a Kodachrome profile from an IT8 target
« Reply #58 on: May 25, 2011, 11:25:31 pm »

It was your target scan that was used in the above post that showed error results for those patches. With the sRGB TRC, respectable DE errors of 2 or less are achievable in those patches, with your equipment. Are you still using the 8-bit target scans to make profiles?

Always been 16-bit except when initially testing. I was being kind to Argyll when I mentioned a problem in my "scanning/profiling process". My first draft of the post you quote from was:

For GS20-23 scanned with low gammas (1.0 -?), Argyll does not accurately transform the scanner RGB values into IT8 L* values. I suspect these is an optimum value of gamma for my setup of around 1.8 to get around this Argyll problem. To test this, I'll have to scan the target at various gammas to see which gives the closest match.

Then I thought: Well, I can't be sure it's Argyll; the error might be occuring somewhere else in the entire scanning/profiling process. So I changed the wording.

And a note to myself and anyone else reading all previous posts: figures given for average RGB values may be in error by as much as 0.5 RGB units because of the incorrect way Photoshop calculates Histogram > Average – it uses 8-bits instead of the maximum available.

Now for a rant. Don't you just love the way PS reports 16-bit L* values as 0-32768 when L* should range from 0-100? Can you imagine someone wanting a more accurate percentage reading of some measurement, say, 67%, so they are told a 16-bit accurate figure is 21953%. Ridiculous. And another thing about 16-bit L* and Photoshop: the values are actually 15-bit, not 16-bit.
Logged

crames

  • Full Member
  • ***
  • Offline Offline
  • Posts: 210
    • http://sites.google.com/site/clifframes/
Re: Generating a Kodachrome profile from an IT8 target
« Reply #59 on: May 26, 2011, 12:43:46 am »

Then I thought: Well, I can't be sure it's Argyll; the error might be occuring somewhere else in the entire scanning/profiling process. So I changed the wording.

Or the problem might be the in the choice of Argyll options that the Coca front end is using.

I mentioned SCARSE before. My correspondence with its author some years ago gives some insight into the choices that profiler writers have to make. He found problems with the curve fitting of the tranfer curve for Kodachrome, because the code assumed that the gray-scale should be neutral, as it is in most "normal" slide films. Of course, if you force it to be neutral it won't match the targets and Lab delta E goes out the window. He had to alter the code to do a robust non-linear fit to all the data to accommodate Kodachome's non-neutrality. I can imagine similar problems with other profilers, but with the authors never taking Kodachrome's perculiarities into account.

Moral of the story, profilers that work well with normal slide films are not necessarily going to work well with Kodachrome. Testing is needed. I wonder how well the commercial profilers do? I have inCamera, which makes extremely accurate profiles, but the original version I have has 8-bit limitations that make it unsuitable for my uses with Kodachrome. (I understand that the newer version doesn't have the 8-bit limitations.) LPROF seems to do a good job, especially with the sRGB TRC. I believe there is a Mac binary version available.

Quote
And a note to myself and anyone else reading all previous posts: figures given for average RGB values may be in error by as much as 0.5 RGB units because of the incorrect way Photoshop calculates Histogram > Average – it uses 8-bits instead of the maximum available.

Now for a rant. Don't you just love the way PS reports 16-bit L* values as 0-32768 when L* should range from 0-100? Can you imagine someone wanting a more accurate percentage reading of some measurement, say, 67%, so they are told a 16-bit accurate figure is 21953%. Ridiculous. And another thing about 16-bit L* and Photoshop: the values are actually 15-bit, not 16-bit.

A lot of "gotchas" to be found along the way, to be sure.
Logged
Cliff
Pages: 1 2 [3] 4 5 ... 7   Go Up