A free high quality resampling tool for ImageMagick users

[NicolasRobidoux]

Alan:
Another thing: With sharpening = 0, the "downsampling" scheme uses EWA Quadratic instead of RobidouxSoft. So, sharpening = 1 (or .0001) should give a signficiantly different result.
On the other hand, note that the Keys Cubic RobidouxSoft is not very far from the cubic B-spline, which in some sense is not "very" far from the Quadratic B-spline.

[Jack Hogan]

LOL. As people say; A dog is man's best friend ...

Jack, I'm not sure where your perceived loss of local contrast comes from, but maybe it is the absence of artifacts (seeing that many people like Luminance aliased Sigma Foveon conversions)?

Can't quite put it into words, Bart.  I can't tell whether my perception of loss of local contrast is due to the shadow-end of transitions being raised or whether it's the highlight-end that, having been boosted, overwhelms the shadow-end.  Keep in mind that my meter of comparison is PS bicubic, as that's what my monitor shows when viewing the full sized image at less than 100% (close to 25% in this case).

Perhaps the forum could help me with the right words.  This is a typical situation: a landscape downsized about 3.5:1 to 1000 pixel height.  The following animated GIF (with all its limitations) shows alternating 1 second views of the same capture downsized with simple bicubic in PS and with v1.2.2 using the downsample routine and 100% sharpening.  It's pretty clear which is which.



Click this link to download or see the image at full size in your browser ( make sure it doesn't resize it: ctrl-zero in Chrome).

First comment is that I find 100% too crispy (see for instance grass bottom right) - not a big deal, I could just as easily use 50% sharpening.  The second is that to me it looks as if saturation and local contrast have been lost by v1.2.2.  See for instance the grass to the left of the blue boot and shadows around the DoG.  Similar effect at 50%.  It's enough to make me prefer bicubic, which can't be right

Jack

[Bart_van_der_Wolf]

Can't quite put it into words, Bart.  I can't tell whether my perception of loss of local contrast is due to the shadow-end of transitions being raised or whether it's the highlight-end that, having been boosted, overwhelms the shadow-end.  Keep in mind that my meter of comparison is PS bicubic, as that's what my monitor shows when viewing the full sized image at less than 100% (close to 25% in this case).

Hi Jack,

That's a very good example of some of the differences between resampling algorithms. In fact, the shadows are as dark as they were at the original size, and the highlights are also preserved better. However, our eyes play a trick on our perception due to the smaller scale (we are more sensitive to the lighter tones when our eyes can't resolve detail, and we imagine luminance differences based on direction of light and we invent edge enhancement, AKA Mach effect). One remedy would indeed be to reduce sharpening to 50, which restores original sharpness/detail without boosting it. That still leaves some of the perceptual effect due to scale.

You could also try reducing sharpening to 0 or less, which will allow to manually dial in deconvolution radius and amount. By choosing a larger radius, say 1 - 2, instead of approx. 0.5, the sharpening will turn a bit more into a local contrast enhancement.

It would be interesting to see a crop of the grass at the original size, and compare it with a zoomed in version of the down-sampled result with sharpening set to 50. I would expect the mean/median value of the histograms to be close to each other, while a bicubic would be darker and with low contrast and lacking subtle color differentiation.

When the reduced sharpening doesn't help enough, there would be of course also be other means to address our over-zealous perception (e.g. Topaz Detail with a Magenta-Green luminance shift for darker Greens), but it would be nice if we can keep the workflow simple.

Perhaps the forum could help me with the right words.  This is a typical situation: a landscape downsized about 3.5:1 to 1000 pixel height.  The following animated GIF (with all its limitations) shows alternating 1 second views of the same capture downsized with simple bicubic in PS and with v1.2.2 using the downsample routine and 100% sharpening.  It's pretty clear which is which.

I'd say human perception is playing tricks on us, but maybe we can find a way to address that as well. It is actually tying in with my 'Blend-if' desire  ...

Cheers,
Bart

[NicolasRobidoux]

Maybe the "downsample" scheme should have default sharpening set to 50 instead of 100?
P.S. Because we are using EWA lanczoses to upsample, not downsample, the default sharpening for the "generic" scheme should be very low. If I have time, I'll figure what corresponds to the LanczosSharp found in Adam Turcottte's thesis. Pretty close to zero. Alan Gibson's tests http://im.snibgo.com/resamphm.htm) support this choice.

[Jack Hogan]

It would be interesting to see a crop of the grass at the original size, and compare it with a zoomed in version of the down-sampled result with sharpening set to 50. I would expect the mean/median value of the histograms to be close to each other, while a bicubic would be darker and with low contrast and lacking subtle color differentiation.

Here are the histograms in an animated GIF

Jack

[NicolasRobidoux]

RE: Jack Hogan's excellent comparison:
With sharpening at 50 or so, I would like to know if making the two deconvolution parameters the same better preserves the color histograms. (Set them both to the linear light value.)
I would also like the answer to the same question with sharpening = 1 (0, really, but this gives EWA Quadratic through linear light, a completely different scheme.)
P.S. I wonder if we have to be more careful in our colorspace conversions. (No time...)
P.S.2 I also wonder whether I need to have another go at blending approaches that are symmetrical w.r.t. black<->white interchange.
P.S.3 I bet symmetric breaking is what's what's going on. Back to the drawing board.
P.S.4 But then, the HVS is asymmetrical. :|
P.S.5 Not obvious how to improve things. In this example, the very dark shadows are actually slightly fuller with LWGB: some form of contrast is increased, just not "symmetrically". Bad Idea: Instead of weighting gamma 1 with the luminance, weight it with luma? (In which case the question of which gamma to use becomes more important: Which gamma is best?)

[NicolasRobidoux]

At this point the best suggestion I have (off the top of my head) to fix the "things get lighter near high amplitude high frequency" is likely to have only a small effect:
1) Use the same deconvolution filter params for both gammas.
2) Instead of using gamma 1 and gamma 3, use gamma 1.5 and gamma 3. (So luminance becomes luma 1.5.)
Hopefully Bart will think of something more effective.

[NicolasRobidoux]

Possibly a red herring, but here is:
Looking at http://blog.kasson.com/?p=7225, I am a bit surprised to see how much dark haloing is seen with the EWA Keys with LWGB deconvolution method.
Are the results blended correctly? (I vaguely recall that it's quite hard to completely get rid of the dark halos, so maybe this is expected.)
-----
Wish I had time to do my own tests...
P.S. Jack Hogan: Thank you for bringing the "lightening effect" to our attention.

[NicolasRobidoux]

Here is another idea:
Set the parameters of the two deconvolutions so that they are the same.
First, check that the "lightening effect" is not there if the two gammas are the same (=1).
Then, push the second gamma up until there is noticeable lightening.
Hopefully, both the light and dark halos are still reasonably small.
This is a really pragmatic solution that takes into account that escaping zero sums is hard.

[NicolasRobidoux]

If this has not been done already:
First, it needs to be ascertained that the issue is not with the colour conversion.
Then, it needs to be ascertained whether there is only lightening. Comparing with the downsampling results of PS or LR is not a reliable indicator B/C I'm pretty sure they don't use linear light (like, for example, nip2, when proper import and colorspace conversion has been performed).

[NicolasRobidoux]

Bart:
(I think we discussed this before.)
Are you sure that -auto-level really helps?
P.S. What I am trying to do is simplify the method as much as possible so that we figure out whether "tonality" is preserved, and how to make things better if not. Also, complex ImageMagick scripts sometimes reveal quirks in the methods' implementations, and simple -> fewer surprises.
P.S.2 Also see Alan Gibson's comment on proper use of color profiles with ImageMagick in http://www.luminous-landscape.com/forum/index.php?topic=91754.msg764826#msg764826 and his script for how to cleanly handle Adobe RGB. I find it very plausible that the LWGB changes the lightness of high amplitude high frequency patches, but we need to be careful not to mix such artifacts (if they are significant) with shortcomings of the toolchain w.r.t. color space handling. FWIW, this is why http://www.imagemagick.org/Usage/filter/nicolas/ warns that the suggested ImageMagick calls assume sRGB input and output. The color space handling should be tackled separately.

[NicolasRobidoux]

For downsampling at the least, I think that the only way to get this to work "perfectly" will be to have the blending be locally determined so as to better preserve local averages. This is likely not to be a trivial thing to put together.

[alain]

...
P.S. I wonder if we have to be more careful in our colorspace conversions. (No time...)

Is a color space conversion needed or is it "just" linearisation that's needed?  If it's just a gamma correction does it matter if it's the 100% correct gamma that's used.
I know of gamma sRGB (close to 2.2, but a little bit special in the low values) ,1.8 (prophoto) 2.2 (adobe and most others).  It could be that gamma 2.2 is good enough for all resamplings...

PS. color space conversions do more that just a gamma correction.

[NicolasRobidoux]

Is a color space conversion needed or is it "just" linearisation that's needed?  If it's just a gamma correction does it matter if it's the 100% correct gamma that's used.

Ideally, the input image should be converted to linear light before being pushed through the resampling pipeline, and the final result converted out of linear light into the desired output color space. If both the input image and output images are sRGB, this is trivial with ImageMagick. As Alan Gibson pointed out in his version of the script, if what's coming is a gamma space (like Adobe RGB), and the output is a gamma space with the same primaries, it's again easy. Things start being more complicated when one deals with color profiles. Making one single tool that automatically, accurately and correctly deals with all possibilities is tricky.

[Bart_van_der_Wolf]

Possibly a red herring, but here is:
Looking at http://blog.kasson.com/?p=7225, I am a bit surprised to see how much dark haloing is seen with the EWA Keys with LWGB deconvolution method.

As in the earlier test in this thread (reply 164) on your suggested CGI vertical edge, it is virtually impossible to avoid halos. With a tiny amount of blur, less than a typical lens produces, the halos are almost gone. Real life camera images are more blurry (Sigma Foveon aliased raw conversions aside), due to lens aberrations, defocus, diffraction, and demosaicing. Also, not all edges are perfectly aligned with the pixel grid, some edges land dead in the middle of the pixel itself, reducing its intensity to halfway between the luminance on both sides.

Cheers,
Bart

[NicolasRobidoux]

Bart:
(Again off the top of my head. No experimenting.)
I am pretty sure that -auto-level is a mistake.
The "lightening" of high frequency patterns will be minimized if the blending alpha varies slowly.
Using -auto-level normalizes it in a way that makes it vary more rapidly.
I realize the attraction of "normalizing" in a "scaling independent" way.
However, in the present case, I would put money down that most of the time it makes things worse.
Nice images are not scaling invariant.

[NicolasRobidoux]

As in the earlier test in this thread (reply 164) on your suggested CGI vertical edge, it is virtually impossible to avoid halos. With a tiny amount of blur, less than a typical lens produces, the halos are almost gone. Real life camera images are more blurry (Sigma Foveon aliased raw conversions aside), due to lens aberrations, defocus, diffraction, and demosaicing. Also, not all edges are perfectly aligned with the pixel grid, some edges land dead in the middle of the pixel itself, reducing its intensity to halfway between the luminance on both sides.

Point taken. The goal is to resample real, off the camera, digital photographs. Results obtained with anything else are backs of the envelope.

[NicolasRobidoux]

Bart:
In the KISS dept, I think my best suggestion so far to minimize the "lightening" effect is to use gamma 1.5 or 2 together with gamma 3 in the deconvolution step. That is: raise the lower gamma so as to "balance" the dark (which currently are visible) and light (which currently are invisible) halos. (And get rid of -auto-level and use the same deconvolution with both gammas.)
Note: I am not talking about the "generic" scheme here. I am talking about the "downsampling" scheme, for which I think that EWA should be done in linear light but the deconvolution should be done in a pair of perceptual spaces chosen to balance and minimize the dark and light halos. There is no reason to stick to gamma 1 except possibly to compute the luminance, although I suspect that using the luma derived from the lowest gamma (whether it is 1 or higher), when deconvolving, is just as good.
P.S. Grrr. Do we lose something perceptually significant by deconvolving out of linear light?

[Bart_van_der_Wolf]

If this has not been done already:
First, it needs to be ascertained that the issue is not with the colour conversion.
Then, it needs to be ascertained whether there is only lightening. Comparing with the downsampling results of PS or LR is not a reliable indicator B/C I'm pretty sure they don't use linear light (like, for example, nip2, when proper import and colorspace conversion has been performed).

I agree, and that's why I asked for a crop of the original and of the down-sampled image (to be blown up with nearest neighbor to the original size for comparison) and histogram mean or median of the histogram compared. We may be comparing a darker duller Lightroom conversion with a better luminance preserving LWGB one. There may also be a difference between profiles (e.g. AdobeRGB working space versus sRGB destination space).

For that purpose, I did a comparison (see attachment) of a crop of a larger file, in Adobe RGB space, downsampled that to 25% with LWGB with sharpening (amount 50, and 100), and upsampled that with Nearest Neighbor (pixel replication). The Histogram Mean / Median of the original is 106.97 / 104, of the amount 50 sharpened it is 108.55 / 106, and of the amount 100 sharpened it is 108.45 / 106.

That isn't to say that there is no lightening due to the linear gamma resampling (there is some), but it's not necessarily as strong as Jack's example shows.

Cheers,
Bart

[NicolasRobidoux]

What would not surprise me at all is that LWGB slightly darkens dark textures and slightly lightens light textures. In other words, it may also have built-in frequency dependent contrast enhancement.
I'm not particularly concerned about it screwing up mid-tones. (Famous last words?)