Generating a Kodachrome profile from an IT8 target

[crames]

– for example, the D-H curves show a minimum density of ~0.21 (0.62 relative luminance), yet my scanned slides go much lighter than that.

You shouldn't get lighter than dmin if you Convert-to-Profile from the scanner profile using Absolute. Are you using the same exposure that you used when scanning the profile target?

Below are the Input/Output figures I tried to put into Curves. The first column is the input, the second is Green output, and the third is Red output. Because PS doesn't allow an input of less than 4 into Curves, I combined the first four settings into one for green and red: 4,7, then rounded the others. Note how the differences disappear at Input = 157 (equal to density of 0.21, the lightest shown on the D-H curves). So the corrections only apply to half the brightness range of the slide. Didn't seem right when I calculated the figures, and it certainly didn't remove the blue cast.

I'm going to give it a try. Maybe there is a way to do it without using Curves.

[crames]

Guy, are you still with me?

After some difficulty trying to invert them, it seemed worthwhile to check if the profiles are giving us anything that resembles the published characteristic curves. On my last couple of rolls of K64 I shot a few frames with a ColorChecker and a Kodak Q-13 gray-scale. This let me generate some plots after applying the various profiles. The procedure was to assign the profile, then convert to a modified gamma 1.0 Prophoto RGB working space with Absolute intent. The plots are displayed in a way that allows comparison to the published Kodachrome characteristic curves that were recently posted.




The first plot is the raw scan without profile.

The second plot shows the gray-scale after applying the Argyll Lab CLUT profile, which is the "best behaved" of all the Kodachrome profiles I have tested for my Polaroid SS4000 scanner.

The third plot shows a result that is typical for most of the other profiles. Note how the red curve crosses over red and blue, giving a changing color-cast along the tonal range.

The big surprise is the fourth plot. For the heck of it, I wanted to see how bad it would be to use an Ektachrome profile. It's better than any of the Kodachrome profiles! Particularly in the upper-left part of the curves, which are the highest densities. This profile makes it very easy to edit, needing only white and black point setting and some gentle curves (and/or gamma adjustments) to achieve a neutral gray-scale along the whole range of tones.

The problem with using the Ektachrome profile is that, although the gray tones can easily be made neutral, the non-gray colors can be inaccurate. The solution would be to make a Kodachrome profile from an Ektachrome profile, a "Pseudo-Chrome" profile using the method described by Hutcheson.

[guyburns]

Cliff, thanks for the four curves. I'll play around with Ektachrome and the Hutch method to see what comes of it. And maybe I should choose Lab clut as my preferred profile, instead of XYZ (see Knockout Rounds, below).

You shouldn't get lighter than dmin if you Convert-to-Profile from the scanner profile using Absolute. Are you using the same exposure that you used when scanning the profile target?

For a Gamma 1.0 scan (see Ref 03 in the clouds and snow), 16-bit, raw RGB values are R>28,000 and G&B > 30,000, well above what the D-H curves indicate I should be getting as the brightest scan from Kodachrome. And when the profile is applied, those values increase. I'm not sure anything sensible can be gained by a person of my limited colour knowledge trying to correct a scan by using the Kodachrome D-H curves, because I don't know how those curves relate to an actual scan.

Guy, are you still with me?

Yes, still here playing around, though I can see an end in sight. At some stage I have to move away from testing and start actual scanning. I've been generating profiles for Agfa, Velvia and Ektachrome and testing them based on the Kodachrome results. A well-exposed Velvia RVP50 slide is so easy to scan and edit compared to Kodachrome. In some cases (see Gil 06: http://www.mediafire.com/?o1y5c7cpstedj3n, one of several Velvia slides I've borrowed from a photographer mate), the unedited profiled scan is superior to my attempt at editing it. I've never come across that when editing Kodachrome, which seems to always need editing.


Preliminary Overall Results
Targets tested: Kodachrome, Ektachrome, Fuji (Vevlia, Sensia, Provia, Astia), Agfa. I scanned all my IT8 targets at G1.8 with the Coolscan V ED scanner, then made a second "corrected" copy of each target by averaging 40% of certain GS patches and applying that to the whole patch. For Kodachrome I altered GS15-GS23. Alterations for the other films varied, depending on how much flare from the surrounds was present. All films except Kodachrome showed an increase in density from GS22 to GS23; only Kodachrome showed a slight reversal. Because of this reversal, for the corrected version of Kodachrome I replaced GS23 with the colour value of unexposed Kodachrome (Lab 56, 21, 12); for the uncorrected version, I replaced GS23 with a copy of GS22 (so that the "uncorrected" scan wasn't corrected very much).

Knockout Rounds
All S+M profiles when applied to the target showed colour changes in certain colour patches compared to Lab and XYZ (which appeared identical). So S+M was knocked at round 1. For both XYZ and Lab, the difference between "Uncorrected" and "Corrected" was minimal, in most cases undetectable, the only difference being a lightening of the darkest GS patches. Because the "corrected" versions should theoretically give better profiles, and because the differences between "uncorrected" and "corrected" were minimal, the "uncorrected" versions were knocked out in round 2.

That left the "corrected" versions of XYZ and Lab to play off in the final. The difference came down in XYZ's favour because of the way it retained the contrast in certain grainy patches (typically the patches GS17-19) i.e. the XYZ profile kept the grain intact whereas Lab smoothed out the grain. Originally I choose Lab because of this, but after further thought I realised the Lab had the lower contrast in the darkest regions (thus smoothing the grain), so I opted for XYZ as the best.


Summary
The best profile from the Nikon Coolscan was obtained by:

1. Using a gamma of 1.8 while scanning at 4000 dpi;

2. Resampling in PS to 2000 to reduce file size for archiving;

3. Correcting certain GS patches to reduce the problem of flare, and in the case of Kodachrome alone, to remove the density reversal between GS22 and 23.

4. Resampling to 1000 for input into Coca for profile generation. I did not alter the Box Shrink parameter from the default 3.5 to my theoretical best of 7.5, because the patch correction of step 3 would have minimized the effect of flare.

5. Editing will be in the IT8 profile space with 2-4 Curve layers applied. There are several reasons for not converting to a wider gamut space. I have arrived at this tentative decision after a few hundred test edits, but the reasons are not yet final:

(a) Testing seems to indicate that editing in a wider-gamut space makes editing more difficult. I'm not convinced that this is a real phenomenon (i.e. a change in editing procedure might fix the problem), and until I work out why this might be the case, this finding is open to change.

(b) Editing in the IT8 profiled space by applying Curve layers is non destructive. Converting to another profile alters the colour numbers and the process can't be exactly reversed. By staying in the IT8 profile space and editing only by Adjustment Layers, the colour numbers are always only one step removed (the gamma 1.8 step) from what the scanner sees on the slide. This is a significant space saving when archiving, because I won't have to archive the original scan as it is non-destructively incorporated in the edited scan.

(c) All my scans are destined for Rec 709 output (effectively sRGB) on a digital projector. I don't require a wide gamut.


Additional Tests
1. One of my long-standing photographer mates wants to learn how to scan his Kodachrome slides, so he sent me some slides to play around with and comment on. I asked for "difficult" sides, and he complied. Check out my thoughts at: http://www.mediafire.com/?ymh90cvds5c3w2j

[crames]

Cliff, thanks for the four curves. I'll play around with Ektachrome and the Hutch method to see what comes of it.

The Ektachrome profile might be the solution to your KC shadow issues. Your Velvia profile might work, too. In the Ektachrome log plot, above, look at how the three channels track nearly perfectly right down to DMAX. Even the wayward red channel has been brought into line. I haven't tried an Ektachrome profile with gamma, yet, since I have only, ever made linear scans with my scanner.

And maybe I should choose Lab clut as my preferred profile, instead of XYZ (see Knockout Rounds, below).

If not your Ektachrome profile. I was using Lab clut only because XYZ does not work as well for linear scans. For gamma >1 scans, I think XYZ is usually better.

For a Gamma 1.0 scan (see Ref 03 in the clouds and snow), 16-bit, raw RGB values are R>28,000 and G&B > 30,000, well above what the D-H curves indicate I should be getting as the brightest scan from Kodachrome. And when the profile is applied, those values increase. I'm not sure anything sensible can be gained by a person of my limited colour knowledge trying to correct a scan by using the Kodachrome D-H curves, because I don't know how those curves relate to an actual scan.

Hmm, my scanner also gives RGB values for DMIN in the same range as yours. I think it's just the scanners giving a little extra exposure. The effect would be to shift the characteristic curves down, so that DMIN is closer to zero density, an equal subtraction of density from the three channels.

All transparency films seem to have a DMIN in the same ballpark. (For that matter, so do negative films.) In that case it would make sense for scanner manufacturers to calibrate so that the maximum RGB numbers are assigned to a density closer to film DMIN, instead of to density = 0 (no film in the scanner).

All that matters is that the highest RGB values in a scan do not exceed the values for DMIN in the Q60 profiling target. If they do exceed DMIN, then the profile can be adjusted by one of the Hutcheson techniques.

If you Convert to Profile using Absolute Intent, the lightest white on the slide will be L* = 88, which is how DMIN is defined for the Q60 target. It gets darker, not lighter. Then just adjust lightness to a preferable level while editing.

Preliminary Overall Results
Targets tested: Kodachrome, Ektachrome, Fuji (Vevlia, Sensia, Provia, Astia), Agfa. I scanned all my IT8 targets at G1.8 with the Coolscan V ED scanner, then made a second "corrected" copy of each target by averaging 40% of certain GS patches and applying that to the whole patch. For Kodachrome I altered GS15-GS23. Alterations for the other films varied, depending on how much flare from the surrounds was present. All films except Kodachrome showed an increase in density from GS22 to GS23; only Kodachrome showed a slight reversal. Because of this reversal, for the corrected version of Kodachrome I replaced GS23 with the colour value of unexposed Kodachrome (Lab 56, 21, 12); for the uncorrected version, I replaced GS23 with a copy of GS22 (so that the "uncorrected" scan wasn't corrected very much).

Did you see a visible improvement with the corrected Kodachrome?

Knockout Rounds
All S+M profiles when applied to the target showed colour changes in certain colour patches compared to Lab and XYZ (which appeared identical). So S+M was knocked at round 1. For both XYZ and Lab, the difference between "Uncorrected" and "Corrected" was minimal, in most cases undetectable, the only difference being a lightening of the darkest GS patches. Because the "corrected" versions should theoretically give better profiles, and because the differences between "uncorrected" and "corrected" were minimal, the "uncorrected" versions were knocked out in round 2.

That left the "corrected" versions of XYZ and Lab to play off in the final. The difference came down in XYZ's favour because of the way it retained the contrast in certain grainy patches (typically the patches GS17-19) i.e. the XYZ profile kept the grain intact whereas Lab smoothed out the grain. Originally I choose Lab because of this, but after further thought I realised the Lab had the lower contrast in the darkest regions (thus smoothing the grain), so I opted for XYZ as the best.

Matches my results, exactly.

5. Editing will be in the IT8 profile space with 2-4 Curve layers applied. There are several reasons for not converting to a wider gamut space. I have arrived at this tentative decision after a few hundred test edits, but the reasons are not yet final:

(a) Testing seems to indicate that editing in a wider-gamut space makes editing more difficult. I'm not convinced that this is a real phenomenon (i.e. a change in editing procedure might fix the problem), and until I work out why this might be the case, this finding is open to change.

(b) Editing in the IT8 profiled space by applying Curve layers is non destructive. Converting to another profile alters the colour numbers and the process can't be exactly reversed. By staying in the IT8 profile space and editing only by Adjustment Layers, the colour numbers are always only one step removed (the gamma 1.8 step) from what the scanner sees on the slide. This is a significant space saving when archiving, because I won't have to archive the original scan as it is non-destructively incorporated in the edited scan.

(c) All my scans are destined for Rec 709 output (effectively sRGB) on a digital projector. I don't require a wide gamut.

Earlier in the thread I showed that sRGB and Adobe RGB can't contain all of the colors of the Kodachrome Q60 target. But chances are your images don't contain colors as saturated as those on the Q60, so you might not have to worry about clipping colors in those smaller color spaces.

Although you don't necessarily have to convert to a wide-gamut space, I think it's important to convert to some space. Otherwise, by not converting at the beginning and editing in the profile space, all your edits to the image are, in effect, modifying the scan before applying the profile, and the scan will no longer match the conditions under which the profile was made. The edits invalidate the profile. You are also giving up the advantages of a standard working space such as knowing that R=G=B is a neutral color.

That's the theory - but if it's working for you, why not?

Additional Tests
1. One of my long-standing photographer mates wants to learn how to scan his Kodachrome slides, so he sent me some slides to play around with and comment on. I asked for "difficult" sides, and he complied. Check out my thoughts at: http://www.mediafire.com/?ymh90cvds5c3w2j

I will check them out!

Try using an Ektachrome profile with your Kodachrome scans and let me know what you think.

[crames]

Additional Tests
1. One of my long-standing photographer mates wants to learn how to scan his Kodachrome slides, so he sent me some slides to play around with and comment on. I asked for "difficult" sides, and he complied. Check out my thoughts at: http://www.mediafire.com/?ymh90cvds5c3w2j

Well done, Guy. That's a great article!

[SergeyT]

I think you would have had a better success with making a scanner profile vs. a "specific film" profile.

If shadow details was your concern why not to try making a *scanners* profile based on the HCT target instead? It has the grays located far away from the saturated colors and most importantly the darkest patch far way from any white. So that would overcome your scanners flare problem and solve the dark tones issue as well. I find using the colprof with "-ax" producing the best profiles.

Profile evaluation should be done by comparing appearance of a scanned slide( with the color profile applied) to the slide's appearance on a light-table with a color corrected light temp.

If they look alike then the job with profiling is done well.
It should not matter if the scan looks bluish on the screen for as long as it looks the same (bluish) on the light table.
All required color correction should be done to the scanned image after its conversion into a working RGB (ProPhotoRGB).

Hope this helps,
SergeyT.

[guyburns]

I thought this thread was long buried. Well, at least I was hoping it was.

Successful profiling assumes a good quality scanner and an accurate target. The HCT target you mention may well be superior to the other targets, but if the scanner is sus, what's the point? And as reluctant as I am to say this, the Coolscans have problems (and I'll include the 5000 and 9000 here, in addition to my V ED, although that may change once I see the test scans from the upmarket models).

You ever had to change your mind about something that you had always assumed? It can be a long, slow process to come to a different viewpoint. I'm that way with my two Coolscan V EDs. Give me another month or so, and I'll probably come around to accepting that a flatbed scanner – to me the idea used to be anathema – that a flatbed without profiling (the Epson V700), can give as pleasing an image as a dedicated slide scanner with profiling. Often the image is more pleasing. I still shake my head about that.

The Science & (Black) Art of Scanning Kodachrome is going to be a good read when I finish it.

[WombatHorror]

I thought this thread was long buried. Well, at least I was hoping it was.

Successful profiling assumes a good quality scanner and an accurate target. The HCT target you mention may well be superior to the other targets, but if the scanner is sus, what's the point? And as reluctant as I am to say this, the Coolscans have problems (and I'll include the 5000 and 9000 here, in addition to my V ED, although that may change once I see the test scans from the upmarket models).

You ever had to change your mind about something that you had always assumed? It can be a long, slow process to come to a different viewpoint. I'm that way with my two Coolscan V EDs. Give me another month or so, and I'll probably come around to accepting that a flatbed scanner – to me the idea used to be anathema – that a flatbed without profiling (the Epson V700), can give as pleasing an image as a dedicated slide scanner with profiling. Often the image is more pleasing. I still shake my head about that.

The Science & (Black) Art of Scanning Kodachrome is going to be a good read when I finish it.

Are you sure? I'm getting results that seem decent enough to me with a Coolscan 9000 and scanning K64 (using Silverfast calibrated with a K64 IT8 slide).
I should say that the 9000 is noticeably better IMO than the V at K^4. The type of semi-diffuse lighting it uses doesn't overdue the grain, bubbles, scratches quite as badly as the more direct lighting in the V and it doesn't have the halation flaring when you get say really bright sky right next to a dark tree line and it seems to be able to dig a bit deeper into the shadow detail. Although I'm probably not being as picky as to getting everything 100% exact from exact tone curve to every last shade without needing to adjust a single thing.

actually these are from some early attempts so they may not be the best samples but I don't have much scanned stuff online yet and I hadn't yet figured out my current sharpening/detail settings yet:

[guyburns]

Hi Larry. Thanks for the comments and the scanned images.

Are you sure? I'm getting results that seem decent enough to me with a Coolscan 9000 and scanning K64 (using Silverfast calibrated with a K64 IT8 slide).

I'm as certain as I can be that, taken overall (sharpness, contrast, colour, lack of flare, and general "feel"), that the Epson flatbed unprofiled gives results that are as good as a profiled Coolscan V ED in most cases. Sometimes better, sometimes inferior. I have come to that conclusion after detailed side-by-side comparison, results of which I will make available when my testing is finished.

This photographer (http://www.gerardhorsman.com/) seems to think the Epson is pretty good. That's what he uses to scan his 6x17 Velvia images. He's just about to hold an exhibition in Hobart showcasing his 2 metre-wide images. He lives close by, and I checked out his process a few weeks ago. The fellow I went with was so impressed, that within a few days he went out and bought a top-of-the-line 27" iMac and the Epson scanner (same as Gerard's setup). Then he gave them to me to set them up for him. It's great to have mates who let you have $3000 of new equipment for a fortnight. That's when I began testing the Epson. I'm not spruiking any particular scanner though. I simply want to find out which one does the best job across a variety of slides.

The slides I have chosen as reference slides are challenging to scan (that's why I chose them). The scans you have posted appear to me to not present too many challenges, and because of that I'd say that most scanners would do a pretty good job on them. My aim is different. If an idea I have comes to fruition, I'll be travelling all over the place scanning slides of a landscape now submerged, and those slides may be Kodachrome, Agfachrome, Anscochrome, Ektachrome in all sorts of conditions. And I may only get one shot at them. Before I embark on this project, I have to be confident that no matter what type of slide a person hands me (overexposed, underexposed, covered in dust, seriously colour-faded), that I know I will come away with the best possible scan given the limitations of my equipment (whatever that turns out to be). Kodachrome, as such, is just one part of my testing.

My theory is: if I can scan challenging slides, the ordinary slides will be easy.

Five of my test slides are in Canada now, to be scanned on a 5000. Then when I get them back, they're off to Germany for the once over on a 9000. If the latter doesn't come off, are you willing to scan them for me? You can gmail me at gdburns.