Hmm... Too wordy? Wrong forum?
Hmm... Too wordy? Wrong forum?
Hi Luke,
It's hard to comment on something you have noticed but I have not, because I use different cameras. Theoretically, at the amount of downsampling ("even to web size") you mention, the only factors can be the MTF curves of both your capture systems combined with the downsampling method used. An optical low-pass filter will barely have any influence on the spatial frequencies that remain after downsampling.
Proper downsampling will eliminate as much as possible all spatial frequencies that cannot be reliably represented in the downsampled image, and do so before downsampling. The filter used for that can make a difference on how exactly the transition from wanted to unwanted detail is shaped, a Lanczos windowed Sinc filter is generally considered being close to a best compromise.
A free commandline utility like ImageMagick allows to do that (the default filter for downsampling is Lanczos), and the result can even be improved by adding a Gamma linearization before the downsampling, in the same Convert command.
I'd be a bit surprised if there were significant differences visible between two almost identical images taken with the same lens and two different bodies. I do agree that it's hard to beat the look of a well downsampled image, almost without any of the possible aliasing artifacts.
Theoretically, at the amount of downsampling ("even to web size") you mention, the only factors can be the MTF curves of both your capture systems combined with the downsampling method used. An optical low-pass filter will barely have any influence on the spatial frequencies that remain after downsampling.I do believe that the lack of OLPF can cause aliasing into low frequencies (even "DC") that may be visible not matter what the downsampling factor is?
Thanks for the recommendation. I have had a hard time finding a convenient Laczos filter that I can incorporate into my workflow. I was hoping ImageJ had it, but it doesn't seem to. Irfanview does it, but only at 8 bits. Too bad there isn't a photoshop plugin that can invoke ImageMagick calls.
To recap:
The image from a high-resolution sensor with H megapixels, after being downsampled to L megapixels (L < H), appears to retain more high-frequency information than an image captured with a native L megapixel sensor when both are examined at L megapixel resolution.
Hypothesis: This would be explained by a difference in the total MTF between the two systems, especially approaching the high-frequency cutoff.
I do believe that the lack of OLPF can cause aliasing into low frequencies (even "DC") that may be visible not matter what the downsampling factor is?
That's most likely what you're experiencing, a higher MTF response near the Nyquist frequency.
One thing that has pleased me about high-resolution sensors is the way in which high-frequency detail appears to be preserved, even after downsampling to prints size, or even to web size. The difference between my D3x and my D3s was striking even at small sizes. I can see it in skin, hair, leaves, textures, etc. I have no experience with the larger sensors, but have noticed the extra detail present in many images posted here.
But what are the factors that explain that? I had a hypothesis -- that the cutoff of the low pass filtering was different in each case. I ventured that this was not just a function of the OLP filters in each respective camera (or the lack thereof in some cases), but also a function of the total regimen of oversampling followed by downsampling. Somewhere up near the Nyquist limit, the D3s starts to fragment. In comparison at the same output size (12MP or less in this case), the D3x would produce good high-frequency detail. This was illustrated in a quick experiment that Lloyd Chambers did with these cameras, photographing the label on a soup can and noting the smoothness of the text in the D3x capture when downsampled to 12MP, in comparison to the fragmented text D3s capture (at it's native resolution).
In an exchange with Bob Newman over at DPR, Bob suggested that I need look no further than the difference between the OLP filters on these two cameras. The D3x does of course have a more gentle filter considered as a function of sampling frequency. But I wonder if Bob's claim is true? I asked Bob if a D3s with no OLP filter would produce output as detailed -- or more so -- than a D3x image downsampled to the same resolution. Bob believes the answer is yes.
Is that correct? I had surmised that there were more factors involved with this. For example, I had always thought of a downsampling filter as being in essence a LPF, with a cutoff that might vary somewhat in slope, depending upon the method. I feel that surely I am getting additional benefits from supersampling and other processing that manifests itself in the retention of detail after downsampling. But I don't have either the empirical or analytical tools at hand to resolve this question.
Among those of you here who have investigated in this area, do you have either an empirical or analytical answer to this, or just a good hypothesis?
To combine comments on OLP filters with mine about demosaicing algorithms: does it make sense that to avoid aliasing, the OLPF should be limited to the lowest spatial resolution of each color, meaning to the resolution of the 6 million red (and 6 million blue) pixels in a 12MP Bayer CFA camera? If so, green resolution gets squashed down to that level too, suggesting that downsampling to anything above about half the camera's pixel count will have resolution limited by the OLPF/sensor combination rather than the downsampled pixel count.My guess would be that the OLPF is a compromise between the spatial resolution of each color channel (slightly blurry for the green, slightly aliasy for red/blue).
To combine comments on OLP filters with mine about demosaicing algorithms: does it make sense that to avoid aliasing, the OLPF should be limited to the lowest spatial resolution of each color, meaning to the resolution of the 6 million red (and 6 million blue) pixels in a 12MP Bayer CFA camera? If so, green resolution gets squashed down to that level too, suggesting that downsampling to anything above about half the camera's pixel count will have resolution limited by the OLPF/sensor combination rather than the downsampled pixel count.Sounds like you are suggesting two possible sources for increased image fidelity, both of which seem plausible. They seem to involve (1) Offset of R->R pixels, B->B pixels, and variously, G->G pixels, and (2) Blur radius of OLPF
Hi,
MTF using Imatest based on D3X and D3S based on test images from The Imaging Resource.
First case: Both imported by default setting in LR
Second case: Both exported as JPEG in Lightroom, scaled to D3S image size using Lightroom, default settings.
Best regards
Erik
Erik,
Thanks for all those graphs.
What do the red lines marked with suffix "(corr)" signify?
Do the results showing sharpening with bicubic down sampling go with the comment in the thread on "N7 downsampled vs M9" http://www.luminous-landscape.com/forum/index.php?topic=60303.msg486061#msg486061 about the hazards of comparing downsampled to unsharpened images?
Or is it instead that by passing through JPEG conversion first, as in your first set of graphs, some sharpness advantage of the D3X files is lost?
Either way, it makes me realize how difficult resolution and sharpness comparisons are due to the intervention of different demosaicing algorithms and such.
P.S. I found the following reference explaining "(corr)". Those curves are with some standardized sharpening, which tries to compensate for the different sharpening in images from different sources. So this does seem the best comparison to use, and it also indicates the sharpening in bi-cubic down sampling from the D3X, since this sharpened red curve is no higher than the unsharpened curve.
http://www.imatest.com/docs/sharpness_comparisons/
This really helps to address some key questions about the benefits or otherwise of high resolution sensors.
I too was curious about MTF50(CORR). I see Imatest has a reference page on measuring sharpness, but I couldn't find reference to it there. Perhaps I needed to drill down further.
http://www.imatest.com/docs/sharpness/
I'm focusing on the TIF test, which doesn't involve questions about JPG artifacts and conversion.
It seems that the downsampled D3x image has significantly higher MTF than the native D3s image. If I'm taking MTF50(CORR) to be the relevant comparison, there seems to be some advantage to the D3x image just beginning around 1000 LP/PH, and becoming somewhat significant by 1500 LP/PH. I had not expected to see such a broad difference, but more of a difference up around the Nyquist frequency of the D3s sensor. Am I reading it correctly? It makes me wonder how other factors of conversion, downsampling method, sharpening, etc, figure into the results.
Do I also read correctly that the edge profile is sharper on the D3x, but also has a "ring," possibly an artifact of bicubic sharper? I wonder how that would fare using Lanczos windowing?
This is really useful data, Erik, and I'm looking forward to seeing your interpretation.
Luke,
Regrading comparing the TIFF and JPEG cases I would say that JPEG artifacts is a red herring. It is a very high quality JPEG conversion and I don't think possible JPEG artifacts upset Imatest.
I hope that other posters, more knowledgeable than me will chime in. Here is what I saw:
1) The JPEG images:
[...]
Here I can see that the two conversions are very similar. In this case I wouldn't expect to see benefits from the higher resolution of the D3X nor significant artifacts.
2) Bicubic sharper
[...]
In this case I see two key differences, one is the higher MTF at medium frequencies in the scaled down image. The other one is that MTF around Nyquist is rather high in the downscaled image. According to most sources MTF exceeding 20-30% at Nyquist frequency would cause problems with aliasing.
3) Aliasing
[...]
Both images are here at same size, the D3X image downsampled using bicubic sharper. Note that "Bicubic sharper" added some visible halos around the numbers.
Hmm. Erik, is it your feeling that
(1) all of the increase in MTF in the bicubic sharpening case is explained wholly by the sharpening?
(2) the downsampled JPG conversion accurately represents the RAW image such that Imatest results are not being skewed by compression artifacts?
For example, among things various people have mentioned as a part of their regimen: would one deconvolve (to solve for the OLPF) as a standard sharpening before downsampling, would one blur slightly to reduce antialiasing before downsampling, would one use say a Lanczos windowed SINc to downsample? What would be optimal?
Finally, there's one thing that an Imatest target doesn't capture, and that is color gradation and fidelity, the enhancement of which adds to the perception of detail.
Bart, what would you consider to be the optimal pre-blur radius for downsampling?
Deconvolution sharpening before downsampling doesn't help, and could possibly hurt the result. Preblurring is mandatory to reduce aliasing artifacts, and Lanczos is considered to be close to optimal.This is interesting. I would have thought that the following steps would be closer to optimal:
This is interesting. I would have thought that the following steps would be closer to optimal:
1. Use deconvolution to obtain an array properly lowpass-filtered according to Nyquist (i.e. maximize flatness in the passband below fs/2, maximize attenuation above fs/2).
...
... MTF around Nyquist is rather high in the downscaled image. According to most sources MTF exceeding 20-30% at Nyquist frequency would cause problems with aliasing.I would think that this aliasing worry only applies to the Nyquist frequency of the signal before down sampling. But I appeal to the local signaling processing authorities for advice on this!
Erik,Not sure if this is what you are asking, but generally when you are doing resampling from rate r1 to rate r2, the lowpass filter needs to have sufficient attenuation at and above min(r1/2,r2/2).
About your comment thatI would think that this aliasing worry only applies to the Nyquist frequency of the signal before down sampling. But I appeal to the local signaling processing authorities for advice on this!
Not sure if this is what you are asking, but generally when you are doing resampling from rate r1 to rate r2, the lowpass filter needs to have sufficient attenuation at and above min(r1/2,r2/2).OK, I see that ... but is it not possible, starting with a signal at a higher sampling rate r1, to filter with a near brick-wall filter in the digital domain, cutting off at r2/2, so that the guideline of MTF < 20-30% does not apply? I am guessing that this 20-30 guideline relates to the limitations of analogue low-pass filtering (as with OLPF's), not to the greater flexibility possible with DSP. Though it probably does not matter for the case at hand, where no MTF curve looks vaguely like a brick-wall, and all of them have significant signal above f(Nyquist).
OK, I see that ... but is it not possible, starting with a signal at a higher sampling rate r1, to filter with a near brick-wall filter in the digital domain, cutting off at r2/2, so that the guideline of MTF < 20-30% does not apply? I am guessing that this 20-30 guideline relates to the limitations of analogue low-pass filtering (as with OLPF's), not to the greater flexibility possible with DSP. Though it probably does not matter for the case at hand, where no MTF curve looks vaguely like a brick-wall, and all of them have significant signal above f(Nyquist).Talking only about linear, space-invariant filtering here.
In these tests, two types of filter frequency characteristic were used. One was a practical implementation of a well-known sharp-cut filter (the ITU Rec. 601[1] channel-filter), and the other a softer filter that avoids ringing or overshoots (the Modified Raised Cosine filter).
...
Each of these two filters consists of 43 taps, and may be called a prototype. The Rec. 601 prototype filter has a rise-time (the 10 – 90 % edge rise-time) of 2 pixels and the Modified Raised Cosine prototype filter has a rise-time of 4 pixels.
...
the filtering process was modified to include a conversion to light-proportional signals by raising the unfiltered signals to a power, filtering and then raising the result to the reciprocal power. The power chosen was 2.35
...
the ‘ringing’ on edges caused by the Rec. 601 filtering serves to give a subjective impression of increased sharpness which allows the rise-time to be increased to give the same degree of apparent sharpness,
...
The most important result is that, when using the sharp-cut “Rec. 601” filter, the average visible limit of edge rise-time lies in the range 0.997 to 1.111 minutes of arc, with a confidence of 95%. Interestingly, the generally accepted figure for visual acuity is approximately 1 minute of arc.
Another point: from a 24MP Bayer CFA sensor, I think that the pixel spacing relevant to the Nyquist frequency for green is the spacing of the green pixels, which is sqrt(2) times pixels width (and for R and B the Nyquist frequency is lower) while for 12MP down-sampled output with full RGB data at each point, the relevant spacing is the width of the new, bigger pixels ... which is sqrt(2) times the original pixels, and so equals the spacing of green pixels. So it seems that the Nyquist frequency is no lower for the downsampling 12MP full RGB output than for the 24MP Bayer CFA input, and is if anything higher for the R/B "colour" information, where aliasing is most noticeable.If you have aliasing at the capture stage, you may struggle with it later on no matter how much you downsample. Imagine a classical western-movie with wagon-wheels that seem to roll backwards or stand still due to the 24fps capture rate. If the movie is "resampled" to 12fps or 60fps, those problems will still be a problem, since the violation of Nyquists sampling theoreme means that the information is ambiguous.
Those of you who are photographers and want image detail (with or without downsampling) on real images, which you capture with your camera (and not some junky nerd lab rat who can't think beyond decades old and outdated MTF charts ;D), can use my freely available detail measure plugin for Photoshop CS3 or Photoshop Elements 8, from the website link in my signature. This tool will let you run detail measure on the whole image or an area selection using the PS marquee tool, as shown below, which is useful for finding detail measures in different parts of an image:(http://djjoofa.com/data/images/jidm.jpg)
Note that detail measure is reported in [0-1] range with higher numbers indicative of more detail. I ran the JDM on a whole image shown above with various downsmapling methods in PS and got the following:
Origianl image: 0.0835
Downsampling using:
- Nearest Neighbor: 0.1156
- Bilinear: 0.0879
- BiCubic: 0.0750
- BiCubic Smoother: 0.0617
- BiCubis Sharper: 0.0853
You can also download the free Raw Import plugin from the same download page and play around with detail measure (and/or other analyses) on separate R,G1,G2,B channels using the Raw Import plugin. And, free yourself of the issues such as which MTF chart to use, distance to an MTF chart, slanted this or that, sensor size, image size, size normalized or not, hey wait I can't see the lines, oops the light is not uniform, blah, blah, blah ...
Hope you find it useful.
Sincerely,
Joofa
Hi,
So your tool finds that "bicubic sharper" yields more detail. But it is real detail or fake detail?! In the case I tested it clearly produced visible haloes.
Best regards
Erik
I ran the JIDM on a whole image shown above with various downsmapling methods in PS and got the following:
Origianl image: 0.0835
Downsampling using:
- Nearest Neighbor: 0.1156
- Bilinear: 0.0879
- BiCubic: 0.0750
- BiCubic Smoother: 0.0617
- BiCubic Sharper: 0.0853
Joofa
This is interesting. I would have thought that the following steps would be closer to optimal:
1. Use deconvolution to obtain an array properly lowpass-filtered according to Nyquist (i.e. maximize flatness in the passband below fs/2, maximize attenuation above fs/2).
2. Use a proper resampling filter (I consider the analog prototype filter shape to be integral to the resampling process). Lanczos order2/3 or some other similar passband/stopband/spatial-ringing trade-off.
3. Sharpen to taste (and according to deficiencies in output medium).
Erik,
About your comment that I would think that this aliasing worry only applies to the Nyquist frequency of the signal before down sampling. But I appeal to the local signaling processing authorities for advice on this!
In actual practice there isn't something like a brick-wall filter, it would also introduce ringing. We're forced to seek a compromise.Sure, but it the lense/OLPF/... introduce significant loss of signal in the passband, it seems sensible to fix it "close to the problem". I meant brick-wall in the loosest possible form, and should probably have said "some nice lowpass shape with passband < fs/2"
As I have been saying, a Lanczos windowed Sinc filter is close to optimal.This is common knowledge, but what are the conditions for this optimality? Is it only "looks good to me"? What is implied about the source image (gamma?) Or can one insert subjectively motivated constraints into something like the remez algorithm and see the lanczos popping out?
Yes, but be careful because you'll (re)introduce stairstepping (=aliasing artifacts).According to some photographers, aliasing is much preferred over Nyquistian sampling. I can see why this can be true if you have complete control over the image chain and are willing to fiddle with pixels until it looks "good".