A 34MP Fx Sensor and Diffration Limit

[Christoph C. Feldhaim]

And there is a point of sensor resolution against f-stop where additional megapixels do not add more information and are simply redundant information filling up your memory card or harddisk space.

This is where the misconception begins.

Allright - I should have been more exact.
So  let me correct:
And there is a point of sensor resolution against f-stop where additional megapixels do not add more relevant information and are simply redundant information filling up your memory card or harddisk space.

In the end we will land at self testing lens/sensor or lens/film combos and actually see what we get, since the theory is so complex, that we simply not yet have a satisfactory model which integrates lens rendering characteristics, F-Stop, sensor/film characteristics, eyesight, subject [sic!] and so on to produce a usable number called "Image Quality Index".

[Christoph C. Feldhaim]

In theory, there is no point where no more information is added when increasing the pixel count. More pixels will always yield more information. Of course, the returns will diminish quickly, so there's a point at which theory and practice will diverge for practical intents and purposes. But that point definitely is NOT at 1 pixel per Airy disk. It rather is somewhere in the 5 × 5 to 10 × 10 pixel region, or maybe even higher, depending on your intents and requirements.

Yes - in theory - if I'm interested in getting great images of Airy discs even 100x100 is maybe not enough....

Just for fun - lets take your 5x5 to 10x10 constraint:
If we split the image diagonal in 1500 to 3000 and use your suggestion we'd end at a range of 1500*5 to 3000*10 pixels diagonal which would result in a 7500 to 30.000 pixel diagonal.
This would be a range of sensors from 4160*6240=26 Megapixels to 16641*24962=415 Megapixels.
In 24x36 mm format  this would result in a pixel pitch between 1.4 and 5.8 Micron.
But what lenses with what apertures would we need here

Other way round:
F 1.4 for example results in an Airy disc (ring 1) of 1.4*1.35 (at 550 nm light)=1.89 Micron
The diagonal on 24*36 is 43.3 mm by 1.89 Micron is 22892.
A diagonal of 22892 * 10 (10 to get the best Airy disk images ) = 228920 results in a sensor of 126982*190473=24187 Megapixel with a pixel pitch of 0.189 Micron.

Huh ... I'm getting dizzy ...

Now if we could just generate a market for that ....

Cheers
~Chris

[ErikKaffehr]

Hi,

A full frame sensor with 34 MP corresponds to a 13.2 MP APS-C sensor (Canon) or 15.1 MP-APS-C sensor (Nikon, Sony, Pentax). So we have plenty of cameras to compare with. I got the impression that to reach maximum resolution the lens should not be stopped down beyond f/5.6-8 (let's say f/6.3). This is based on MTF test done by the Swedish periodical Foto using Imatest.

The loss of sharpness with diffraction is gradual and can to some extent be compensated by correct sharpening.

Best regards
Erik

Hello :

Anyway to predict what which f/stop Diffraction Limit would be reached for a 34 MP FX Sensor ? Are there any such mathematical formulas that can provide this information ?

Thanks,

Jai

[bjanes]

In that case sensel pitch is of paramount importance, because a diffraction pattern diameter that exceeds the sensel pitch by a certain amount will cause degradation.

Your earlier remark:also misses the point that a diffraction pattern that's significantly smaller than the sensel pitch cannot be resolved by the sensor (especially in the presence of an OLPF), the diffraction pattern diameter is too small. Yet you suggest otherwise.

Well stated, Bart. Here are my thoughts for comment:

It is always good to compare theory with actual results in a real world system. Photozone.de has an abundance of test data for various lens and camera combinations using Imatest MTF50, which is a good measure of perceived image sharpness. Here are the data for the Zeiss Macro Planar 50 mm f/2.8 lens on the Nikon D200. One might prefer data for the D3x, but the pixel pitches of the two cameras are similar. The size of the diffraction spot for green light (wave length = 540 nm) is also shown (from Roger Clark's site).

Two times the pixel pitch is often regarded as the critical value for the diffraction spot, since demosaicing of the Bayer array interpolates from the RGGB quartet of the sensor.

With the lens wide open, there is some loss of MTF due to lens aberrations, and system MTF improves as the lens is stopped down to reduce aberrations. MTF is maximal when the diffraction spot is approximately the size of the pixel. There is no really significant loss of MTF until the diffraction spot is approximately double the pixel size.

One should remember that MTFs multiply and the system MTF is the product of the individual MTFs (lens, sensor, demosaicing software). Since MTF is always less than 1.0 in a real world situation, MTF can never be greater than that of the lowest MTF in the chain, but a higher MTFs in the other components can improve system MTF.

[Bart_van_der_Wolf]

Two times the pixel pitch is often regarded as the critical value for the diffraction spot, since demosaicing of the Bayer array interpolates from the RGGB quartet of the sensor.

Hi Bill,

For large output, where the resulting pixels wiil need to be interpolated to get that large output with adequate PPI, I use a more critical rule of thumb for the onset of visible degradation at the pixel level, 1.5x the sensel pitch. It is based on my experience with a number of DSLRs and lenses (and a P/S compact), and the principle that the diagonal spacing of the pixels is 1.41x the horizontal/vertical sensel pitch, and the fact that the peak of one sensel's diffraction pattern approx. coincides with the first minimum of its diagonal neighboring sensel. Another reason why 2x might be a bit tolerant, is because the Bayer CFA samples a proportion of luminosity at each (1x pitch) sensel position.

Of course this cannot be more than a rule of thumb, because we usually are dealing with multispectral light (not just 555 nm, although it's important for visual acuity), we can have the influence of an OLPF, and we often do not have a perfectly circular aperture. Having said that, and as also demonstrated by your example, 1.5x the sensel pitch (@ f/6.8 ) is close to the optimum of that lens/sensor combination.

Of course some of the losses can be restored in postprocessing, and when we can relax out output size requirements, the considerations will change, and we can start worrying about optimal downsampling instead of diffraction

Cheers,
Bart

[01af]

... seems to indicate that you are missing the simple fact that the choice for large megapixel sensor might be inspired by the need for large output.

What a silly statement!  

Of course I am aware of that simple fact.

Large output means that the per-pixel microcontrast needs to be as good as one can reasonably get (within the DOF limitations one sets).

Sure.

In that case sensel pitch is of paramount importance, because a diffraction pattern diameter that exceeds the sensel pitch by a certain amount will cause degradation.

Diffraction will always cause some degradation. The smaller the aperture, the more degradation through diffraction. At any pixel pitch.

Your earlier remark:also misses the point that a diffraction pattern that's significantly smaller than the sensel pitch cannot be resolved by the sensor ...

The question is not, can the sensor resolve the diffraction pattern? The question is, can the diffraction pattern reduce sharpness? And the answer is, yes it can, even when one single Airy disk is smaller than one single pixel.

Maybe you have some actual examples that unambiguously show this physics defying phenomenon?

Before making snide remarks about the defying of physics, it would be useful to understand the physics in the first place, wouldn't it? You seem to believe in a system with two resolution-limited subsystems where the output from the first is the input to the second, the resulting system's resolution was equal to the smaller of the two. Not so.

Be R1 the first subsystem's resolution limit (lens), R2 the second subsystem's (sensor). Then for the resulting system resolution Rtotal, the following equation holds:

1/Rtotal  =  1/R1 + 1/R2

As you can easily see, the system resolution will increase when increasing either subsystem's resolution—even when increasing the one that was higher before. And the system resolution will drop when decreasing either subsystem. It's not the weaker subsystem that's defining the limit for the whole system. Instead, it's both ... or all then there are more than two subsystems. In that case, the equation above gets extended like this:

1/Rtotal  =  1/R1 + 1/R2 ... + 1/Rn

Please derive your own conclusions from this with regard the the interaction of pixels and Airy disks. Real life is more complex than you think. And we're still far from the very bottom of all this ...

[01af]

MTF is maximal when the diffraction spot is approximately the size of the pixel.

Actually, the MTF is maximal where the gain in lens performance through reduced lens aberrations gets outweighted by the loss through diffraction. This happens near f/4 for the image center and around f/5.6 at the edges. That's typical values for a top-class 35-mm-format lens. On another camera with a lower or a higher pixel pitch, the maximum for this lens will appear at the very same f-numbers (albeit with different absolute MTF50 numbers).

[bjanes]

Be R1 the first subsystem's resolution limit (lens), R2 the second subsystem's (sensor). Then for the resulting system resolution Rtotal, the following equation holds:

1/Rtotal  =  1/R1 + 1/R2

As you can easily see, the system resolution will increase when increasing either subsystem's resolution—even when increasing the one that was higher before. And the system resolution will drop when decreasing either subsystem. It's not the weaker subsystem that's defining the limit for the whole system. Instead, it's both ... or all then there are more than two subsystems. In that case, the equation above gets extended like this:

1/Rtotal  =  1/R1 + 1/R2 ... + 1/Rn

Please derive your own conclusions from this with regard the the interaction of pixels and Airy disks. Real life is more complex than you think. And we're still far from the very bottom of all this ...

Your formula for system MTF is outdated and only applies to MTFs around 10%, which are too low to be of much use in terrestrial photography. Norman Koren briefly discusses a more general approach which can be used with more useful MTFs. One must use a Fourier transform to convert from the spatial to the frequency domain and then multiply the frequency components and then perform an inverse transform back to the spacial domain using a convolution. I have never done this myself, but Bart could likely explain the process in more detail.

A bit of advice: be careful in your criticism of Bart, who is one of the most knowledgeable contributors to this forum. He is usually correct, but I don't think he needs to prove himself to you.

Regards,

Bill

[bjanes]

Actually, the MTF is maximal where the gain in lens performance through reduced lens aberrations gets outweighted by the loss through diffraction. This happens near f/4 for the image center and around f/5.6 at the edges. That's typical values for a top-class 35-mm-format lens. On another camera with a lower or a higher pixel pitch, the maximum for this lens will appear at the very same f-numbers (albeit with different absolute MTF50 numbers).

You seem to forget about the resolution of the sensor. Diffraction takes place independently of the sensor, but resolution by the lens beyond what can be resolved by the sensor is of little use.

Regards,

Bill

[Bart_van_der_Wolf]

The question is not, can the sensor resolve the diffraction pattern? The question is, can the diffraction pattern reduce sharpness? And the answer is, yes it can, even when one single Airy disk is smaller than one single pixel.

In theory, sure. You probably/hopefully are aware that there are limits as to what difference can be observed by a human, and of the limits of image reproduction? How about in practice, as in helpful to the OP? Any example(s) where the diffraction pattern diameter was significantly smaller than the sensel pitch that you wish to share?

Before making snide remarks about the defying of physics, it would be useful to understand the physics in the first place, wouldn't it?

Snide remarks, or misplaced condenscending remarks? You seem to prefer the latter.

You seem to believe in a system with two resolution-limited subsystems where the output from the first is the input to the second, the resulting system's resolution was equal to the smaller of the two. Not so.

You are mistaken in what I believe. The MTFs of the optical chain components multiply (it's the convolution of the Point Spread functions and the original signal that defines the MTF), which also has an impact on the limiting resolution, that's all very well understood. FYI, I've been using the slanted edge method of MTF determination, amongst others, even before it was a formal ISO standard procedure. This is not new territory. For those who are new to Digital Signal Processing, I can recommend the free The Scientist and Engineer's Guide to Digital Signal Processing, By Steven W. Smith, Ph.D., it's relatively easy to read (student level, relatively low on math content).

However, the sampling density sets a hard upper limit (Nyquist frequency) for spatial resolution (in the context of this thead, we're not talking about supersampling/drizzling techniques), yet you find it is humorous to consider any sensel pitch. How much difference then would you say a bit of diffraction has on resolution when the area sensors cannot resolve the diffraction pattern and other small detail in the first place, with or without OLPF? Or do you view aliasing as detail?

Please derive your own conclusions from this with regard the the interaction of pixels and Airy disks. Real life is more complex than you think. And we're still far from the very bottom of all this ...

The suspense is killing ...

Cheers,
Bart

[01af]

Sigh.

It's so simple.

Consider a lens. A real-world lens made for the purpose of photography. It has a maximum and a minimum aperture. It has an optimal aperture that is somewhere between the minimum and the maximum. The so-called optimal aperture is where the lens' resolution is at its optimum, and that is where decreasing lens aberrations and increasing diffraction losses balance each other out.

Use this lens on a camera. Different cameras have different sensors, or will be loaded with high-speed or low-speed films. Hence, they have different resolution limits. Still, the lens' optimal aperture will always be the same. The absolute resolution of the final image depends on both the resolution of the lens and the resolution of the image-recording medium (sensor or film). Now for the crucial point: No matter what the image-recording medium's resolution limit may be—for any given recording medium, the optimum will be just where the lens' optimum is. As simple as that.

[Policar]

Sigh.

It's so simple.

Consider a lens. A real-world lens made for the purpose of photography. It has a maximum and a minimum aperture. It has an optimal aperture that is somewhere between the minimum and the maximum. The so-called optimal aperture is where the lens' resolution is at its optimum, and that is where decreasing lens aberrations and increasing diffraction losses balance each other out.

Use this lens on a camera. Different cameras have different sensors, or will be loaded with high-speed or low-speed films. Hence, they have different resolution limits. Still, the lens' optimal aperture will always be the same. The absolute resolution of the final image depends on both the resolution of the lens and the resolution of the image-recording medium (sensor or film). Now for the crucial point: No matter what the image-recording medium's resolution limit may be—for any given recording medium, the optimum will be just where the lens' optimum is. As simple as that.

Except with film, where dispersion causes a decrease in resolution with wider apertures.

With a 34MP FX sensor, digital will in theory equal large format.  If f4 is the diffraction-limited aperture, as it appears to be according to the calculations earlier in this thread, that's equivalent to around f16 on large format, field of view/depth of field-wise.  It's also the widest "normal" operating aperture for large focus work.  Past this point and with the best film, large format is mostly diffraction-limited; now it will be diffraction-limited on FF digital, too.

Velvia, the sharpest color film in common use, also drops past 100% mtf at 20lp/mm.  Digital equals or exceeds 100% mtf to about 70% of its claimed resolution.  34 megapixels on 36mmx24mm equates to 200 pixels/mm=100 pixel pairs/mm=70lp/mm (really).  36mm*70lp/mm=2520 line pairs on a 34MP FX bayer sensor; 127mm*20lp/mm=2540 line pairs on 4x5 velvia.  Yes, film will have some barely perceptible detail past this point (whereas digital won't) but most of that will be decimated by diffraction the majority of the time, anyway.

34MP FX=4x5 velvia, both formats diffraction-limited at the same depth of field--but four stops faster on the digital camera even at the same ISO (f4-->f16).  Bye bye, large format!

[Bart_van_der_Wolf]

The absolute resolution of the final image depends on both the resolution of the lens and the resolution of the image-recording medium (sensor or film). Now for the crucial point: No matter what the image-recording medium's resolution limit may be—for any given recording medium, the optimum will be just where the lens' optimum is. As simple as that.

Thanks for clarifying. However, you now (in the first sentence quoted above) bring the sensor resolution (sensel pitch or sampling density in this thread's context) into the equation for "absolute resolution". It seems we can, finally, agree on that.

When we disregard the sensor, like in your second sentence quoted above, residual lens aberrations which reduce with narrower apertures, and diffraction increasing with narrower apertures, tends to find an optimum in MTF response somewhere between the extreme aperture settings. There was never a disagreement there.

The combination of both sensor and lens however, more analytically the multiplication of their MTFs in linear gamma space, will have its own optimum (when we only look at the limiting resolution), and it is even spatially variant (corners are usually worse than the optical centre). It's hard to speak of a global or even absolute optimum, because different MTF shapes will benefit different spatial frequencies, hence the simplification to limiting resolution for a specific position (centre) in the image plane.

Denser sampling (smaller sensel pitch, over-sampling) tends to bring the maximum system resolution closer towards the optical optimum (although with increasing dynamic range limitations), and coarser sampling density brings it closer towards the limiting Nyquist frequency. It is therefore important to also consider the role of the sensel pitch in the equation for total system resolution.

Cheers,
Bart

[01af]

However, you now [...] bring the sensor resolution [...] into the equation for "absolute resolution". It seems we can, finally, agree on that.

"However"? "Now"? "Finally"? Huh? I never said otherwise, and we never disagreed on that.

[...] residual lens aberrations which reduce with narrower apertures, and diffraction increasing with narrower apertures, tends to find an optimum in MTF response somewhere between the extreme aperture settings. There was never a disagreement there.

Right.

The combination of both sensor and lens however, more analytically the multiplication of their MTFs in linear gamma space, will have its own optimum [...].

Denser sampling [...] tends to bring the maximum system resolution closer towards the optical optimum [...], and coarser sampling density brings it closer towards the limiting Nyquist frequency. It is therefore important to also consider the role of the sensel pitch in the equation for total system resolution.

No, it's not. No matter what the sampling density is—for any reasonable given sampling density, the optimum is always where the lens' optimum is.

I think your misconception comes from the non-linearity of the relationship between the subsystems' individual resolution limits and the whole system's resulting resolution liimit. When stopping down beyond the lens' optimum then the system's resolution will drop—but in a non-linear way. The loss of resolution will be small as long as the Airy disks are smaller than the pixels, and it will take a significant drop when the Airy disks outgrow the pixels. So for practical intents and purposes, the pixel pitch does have some significance. Still the optimum is where the lens' optimum is.

See the attached image below. It assumes an f/2 lens with its optimum at f/4 (blue curve). It's used on two sensors; Sensor A with a pixel pitch of approx. 7.5 µm (yellowish green) and Sensor B with a pixel pitch of approx. 15 µm (blueish green). The green curves are supposed to show the resulting system resolution when the given lens is used on the respective sensor. The lens' diminishing performance hits the resulting resolution severely when the Airy disks outgrow the pixel size (see arrows "significant drop-off"). The curves are pretty flat from f/2 to the drop-off points, and even flatter for the lower-resolution sensor. But they're not perfectly flat, that's my point! There still is an optimum, and that's exactly where the lens' optimum is.

The relative flatness of the curves before the drop-off point makes it hard to locate the optimum in real-world images, while the drop-off point is fairly obvious ... at least for pixel peepers. That's the source of the misconception that the optimal f-stop depends on the pixel pitch (among other things). But it doesn't.

[bjanes]

See the attached image below. It assumes an f/2 lens with its optimum at f/4 (blue curve). It's used on two sensors; Sensor A with a pixel pitch of approx. 7.5 µm (yellowish green) and Sensor B with a pixel pitch of approx. 15 µm (blueish green). The green curves are supposed to show the resulting system resolution when the given lens is used on the respective sensor. The lens' diminishing performance hits the resulting resolution severely when the Airy disks outgrow the pixel size (see arrows "significant drop-off"). The curves are pretty flat from f/2 to the drop-off points, and even flatter for the lower-resolution sensor. But they're not perfectly flat, that's my point! There still is an optimum, and that's exactly where the lens' optimum is

You raise an interesting point, but how did you obtain the values for your plot? MTFs for lenses are usually plotted at two frequencies, perhaps 10 lp/mm and 30 or 40 lp/mm with the lens at maximal aperture and stopped down to f/8 or so. One can obtain MTF at multiple frequencies and apertures with Imatest, but the results are system MTF. MTFs for sensors are difficult to obtain.

And then did you calculate system MTF with your inappropriate equation or did you perform a Fourier analysis and deconvolution as discussed above. It would be interesting to see your data and calculations.

Regards,

Bill

[hjulenissen]

Seems to me you are discussing (?) what will be the total response of two lowpass filters in series, each having a cutoff frequency of fc1, fc2.

A general answer cannot be found, I think, without having more detailed information about the actual response (besides the -3dB point or whatever).

If both filters can be assumed to belong to a generic class of lowpass filters that can be fully described by its cutoff frequency, then the solution is possible. But it seems that optics cannot be simplified in that manner?

Optics, OLPF, micro lenses and the spatial integration carried out by pixel sites can probably be described by a linear space-variant filter/PSF reference system. The actual pixel sampling grid cannot, and seems to make the problem harder to describe in a simple manner.

-h

[Daniel Browning]

Whatever happened to the guys who said 6 megapixels are enough, haven't heard from them in a while.

Maybe they got new accounts and they are the ones now saying 12 MP is enough for anyone.

You can go back as far as 2003 and find the more brilliant folks, like Brian J. Caldwell (highly talented lens designer) have been saying that it's possible to get meaningful information from 24 MP in DX-sized sensors at f/8 or even f/11.

[Bart_van_der_Wolf]

Optics, OLPF, micro lenses and the spatial integration carried out by pixel sites can probably be described by a linear space-variant filter/PSF reference system. The actual pixel sampling grid cannot, and seems to make the problem harder to describe in a simple manner.

Indeed, it is not easy to model the system MTF with several poorly characterized components. However, one can approximate the effect on overall performance when one varies a single variable, e.g. the sensel pitch.

I assume most professional photographers will only invest in higher MP sensor arrays when they feel the need to produce larger output. Afterall, there is not much use for huge MP (and associated storage) solutions for web-publishing. Because most camera platforms pose physical limitations on the dimensions of the sensor array, the common approach is to increase sampling density, i.e. using a smaller sensel pitch, for a sensor array of a given size.

I've made a simulation of the sensel pitch effect on the MTF curve of a system with an imaginary perfect lens (no residual aberrations), with a fixed (perfectly circular) aperture of f/8 which causes diffraction , and a (square, 100% fill factor, sensel grid) sensor array without OLPF. I've varied the sensel pitch between 1 micron and 9 micron, which will have an effect on dynamic range, but I've only focused on the sensel pitch effect (due to diffraction) on resolution.

Without diffraction, the square exposure aperture of the sensel basically performs like a box filter with the size of a sensel and that produces a predictable MTF roll-off, and that shape does not change with sampling density (because each sensel is filtered the same). However, the diffraction pattern diameter for a given wavelength and aperture value has a given dimension and spans a variable number of sensels, depending on the sensel pitch.

In the attached file, I've shown the same f/8 diffraction pattern for 555 nm wavelength, but overlaid with grids with a different pitch. The 9 micron pitch grid shows that a single sensel position is almost the same size as diffraction pattern, but at a 1 micron pitch the same diffraction pattern is subsampled much more. Again, the diffraction patterns are the same, it's the sampling pitch that's different. This represents the effect of viewing each pixel at the same size (100% zoom on screen, or the same PPI in output). The denser sampling will produce more output pixels for the same image detail, so larger output at a given PPI but with lower per pixel micro-contrast.

I've used a simulated crop from a 24x36mm sensor array, and convolved the various sensel pitch versions with the f/8 diffraction pattern of 555 nm light. The results were evaluated with Imatest, and I'll attach several graphical outputs of some relevant MTF results in followup posts (due to the file number/size limitations). First I'll present a summary of various key numbers from the Imatest output in the second attachment. For an explanation of their meaning you can read about it on the Imatest website (http://www.imatest.com/docs/sharpness.html#optimum_aper).

Cheers,
Bart

[Bart_van_der_Wolf]

Here are some Imatest results in graphical form:
First attachment is for a 100% fill factor (FF=100) 1 micron sensel pitch (SP=1) sensor array with a lens at aperture number f/8 (F=8) and 555 nm light (555nm). The spatial frequencies are expressed in Cycles/pixel, where 0.5 Cy/Px equals the Nyquist frequency (beyond which no detail can be reliably resolved).

The following 3 attachments are for 2, 3, and 4 micron sensel pitches.

Cheers,
Bart

[Bart_van_der_Wolf]

And continued by the 5, 6, 7, and 8 micron sensel pitch versions.

Cheers,
Bart