Luminous Landscape Forum

Equipment & Techniques => Digital Cameras & Shooting Techniques => Topic started by: David Sutton on March 16, 2013, 03:59:01 am

Title: Is the problem of diffraction over-rated?
Post by: David Sutton on March 16, 2013, 03:59:01 am
Hello folks.
Short answer: probably.
I ran some tests and found with deconvolution sharpening most of my lenses are as good at f/22 as they are at f/8. I am somewhat nonplussed.
It's another layer of processing, but so what.
I've just written it up on my blog (http://davidsutton.co.nz/blog/). I'd appreciate any feedback. Sometimes what is clear in ones own mind becomes impenetrably wordy when written down.
Cheers,
David

Edit: I've just noticed see Erik has done similar test on his site (http://echophoto.dnsalias.net/ekr/index.php/photoarticles/49-dof-in-digital-pictures?start=2)
Title: Re: Is the problem of diffraction over-rated?
Post by: scooby70 on March 16, 2013, 08:24:08 am
For me... probably.

Add some sharpening and fiddle with the contrast and whatever else needs to be fiddled with and for most people it'll probably be fine  ;D
Title: Re: Is the problem of diffraction over-rated?
Post by: RFPhotography on March 16, 2013, 09:27:59 am
Is the problem of diffraction overrated?  Probably, yes. 

You produced your examples with no sharpening of the raw files.  My understanding is that the Detail panel in LR/ACR also uses a form of deconvolution.  Would you have been able to achieve even better results with some judicious sharpening at the raw conversion stage and would the f/32 image have been recoverable then?  Without raw sharpening, you're mixing the effects of diffraction, the AA filter and demosaicing.  Working to reverse the second two at the raw stage might help, mightn't it?
Title: Re: Is the problem of diffraction over-rated?
Post by: Bart_van_der_Wolf on March 16, 2013, 09:57:06 am
Hello folks.
Short answer: probably.
I ran some tests and found with deconvolution sharpening most of my lenses are as good at f/22 as they are at f/8. I am somewhat nonplussed.

Hi David,

I've been advocating the use of deconvolution sharpening for a long time, and there is a interesting thread with examples (http://www.luminous-landscape.com/forum/index.php?topic=45038.msg378541#msg378541) covering that subject on LuLa as well. However, there is something that many people overlook. When looking at the MTF curve of diffraction, it becomes clear that the spatial frequencies near the Nyquist frequency (the highest level of detail that can be reliably reconstructed from discrete samples), will suffer most loss of contrast. That means that when detail already has a low contrast (such as all high spatal frequency already does), it will be reduced to zero much earlier than higher contrast detail with the same spatial frequency. So low contrast microdetail will become unrecoverable, while some of the higher contrast microdetail may still be recoverable by deconvolution (having a low noise image helps to increase the chance of pulling it off).

So yes, you will be able to recover a lot of what seemed to be lost by using high quality deconvolution, but the low contrast micro detail may be lost forever (also in the focus plane, not only at the edges of the DOF zone) when stopping down too far. An aperture of f/22 is guaranteed to limit your average luminance microcontrast detail to being totally unrecoverable above 90% of your 40Ds maximum resolution! When you limit the narrowest aperture to f/20, then some lower contrast micro detail can theoretically be recovered all the way up to the maximum resolution of your camera (although in practice, the residual lens aberrations will still throw a spanner in the works, but at least you're not throwing away every possibility of recovery).

The challenges with deconvolution are:

I've made available a free tool (http://bvdwolf.home.xs4all.nl/main/foto/psf/SlantedEdge.html) to determine point a. for a given camera/lens/aperture combination. That will alow to determine the correct settings for Topaz InFocus (which you apparently used), or other deconvolution tools (e.g. RawTherapee offers Richardson Lucy deconvolution, even when using existing files, not only Raws), much more accurately than we generally can achieve by eye with trial and error. It can also reveal some interesting facts, such as when using tele-extenders or some zoomlenses. My tool will also allow, under its section 4. that operates independently from the 3 prior steps, to figure out what the diffraction limits for certain aperture sensel pitch combinations are.

Good deconvolution also benefits from good tools (preferably with tweakable regularization of noise amplification, and custom PSF input), and high computational accuracy (to avoid round-off errors creeping into the results).

Cheers,
Bart
Title: Re: Is the problem of diffraction over-rated?
Post by: theguywitha645d on March 16, 2013, 11:44:49 am
Yes, the problem is very much overstated. Part of this is because it is done at pixel level, meaning there is no difference to a 2MP sensor and an 80MP sensor nor the format, and that diffraction is always done in a comparative way where it would probably not be seen in a single image.
Title: Re: Is the problem of diffraction over-rated?
Post by: Bart_van_der_Wolf on March 16, 2013, 12:39:50 pm
Yes, the problem is very much overstated. Part of this is because it is done at pixel level, meaning there is no difference to a 2MP sensor and an 80MP sensor nor the format,

Hi,

Are you suggesting that an image from a 2MP sensor doesn't need more magnification for a given output size than the image from an 80MP sensor? Because that would give the same sized diffraction pattern on the sensor for a same size aperture and focal length.

Quote
and that diffraction is always done in a comparative way where it would probably not be seen in a single image.

Of course it is done in a comparative way, because otherwise we cannot show how much is lost to diffraction. Closing one's eyes doesn't make the light go out ...

Cheers,
Bart
Title: Re: Is the problem of diffraction over-rated?
Post by: Fine_Art on March 16, 2013, 02:11:30 pm
Bookmarked your slanted edge tool, thanks Bart!
Title: Re: Is the problem of diffraction over-rated?
Post by: Fine_Art on March 16, 2013, 02:33:07 pm
Deconvolution is not a panacea. It can put messy artifacts into areas that should be OOF. That is where PS layers and masking really pays for PS.
Title: Re: Is the problem of diffraction over-rated?
Post by: theguywitha645d on March 16, 2013, 03:02:18 pm
Hi,

Are you suggesting that an image from a 2MP sensor doesn't need more magnification for a given output size than the image from an 80MP sensor? Because that would give the same sized diffraction pattern on the sensor for a same size aperture and focal length.

Output size is a function of format, not pixel pitch and that is where evaluation at 100% is problematic. An image from a 35mm sensor is not impacted by diffraction at differing amounts because of pixel pitch. The viewer perception does not change and viewing distance and print size is not done in relation to pixel pitch--I make print to a size I want, say 16x20, I don't vary my print size because of how many pixels I have.

Quote
Of course it is done in a comparative way, because otherwise we cannot show how much is lost to diffraction. Closing one's eyes doesn't make the light go out ...

Cheers,
Bart

But if the loss can only be viewed under very tight constraints, then how significant is it? So what you are saying is the loss is overstated--my position?

Just because there is a loss, does not mean the loss would be perceived in normal condition. An image can still be sharp even though there are conditions that are "sharper." Then there is DoF. Which is greater and more significant to the final image, the loss of sharpness through diffraction or the loss through DoF? And this is where the comparison is meaningless. What I want is an image that looks sharp and the point where my system maximizes MTF is not the only point where a sharp images is made.
Title: Re: Is the problem of diffraction over-rated?
Post by: Bart_van_der_Wolf on March 16, 2013, 05:43:47 pm
Bookmarked your slanted edge tool, thanks Bart!

Hi Arthur,

You're welcome. It was discussed in a bit more detail here (http://www.luminous-landscape.com/forum/index.php?topic=68089.msg538932#msg538932).

Cheers,
Bart
Title: Re: Is the problem of diffraction over-rated?
Post by: Bart_van_der_Wolf on March 16, 2013, 06:08:30 pm
But if the loss can only be viewed under very tight constraints, then how significant is it? So what you are saying is the loss is overstated--my position?

That totally depends on the circumstances. That's why I won't say it is always overstated, because it sometimes is very relevant, especially in Photomacrography and with large format output, where the individual pixel quality counts. When the final image is shrunk, so are many issues ...

Quote
Just because there is a loss, does not mean the loss would be perceived in normal condition. An image can still be sharp even though there are conditions that are "sharper." Then there is DoF. Which is greater and more significant to the final image, the loss of sharpness through diffraction or the loss through DoF? And this is where the comparison is meaningless.

Not necessarily. It could just point to the need for a different technique, such as focus stacking (which will allow to maximize the per-pixel quality).

Cheers,
Bart
Title: Re: Is the problem of diffraction over-rated?
Post by: David Sutton on March 16, 2013, 06:22:53 pm
Thanks for your replies folks, and thank you Bart for the link.
Reading your comments, here are some more random thoughts.
While detail may be lost with at f/22 and deconvolution sharpening, if I can't see it it doesn't matter. YMMV
Sharpness is no longer on the top of my priorities. It is too often a substitute for content. The quality of the blur in the out-of-focus areas is more important to me.
Many people who worry about diffraction go out and photograph with the lens wide open. :o
If the camera is hand held all bets are off anyway.
When I have time I'll continue to take multiple photographs and focus blend, as this gives me a lot of control over the plane of focus holding the subject matter- deciding what is sharp and what isn't in that plane.
But otherwise if I want DOF f/22 will do fine as long as it doesn't drive my shutter speed down to around the half second area where the "thump" of the shutter on the 5DII shows up worst.
Title: Re: Is the problem of diffraction over-rated?
Post by: Bart_van_der_Wolf on March 16, 2013, 06:53:32 pm
Sharpness is no longer on the top of my priorities. It is too often a substitute for content.

Hi David,

You are correct, it shouldn't be a substitute. But diffraction should also not be a distraction, if it can be reduced or avoided. It's all subject dependent IMHO.

Quote
The quality of the blur in the out-of-focus areas is more important to me.

To me as well, ugly bokeh can seriously distract from the subject of focus (pun intended).

Quote
But otherwise if I want DOF f/22 will do fine as long as it doesn't drive my shutter speed down to around the half second area where the "thump" of the shutter on the 5DII shows up worst.

Yes, it's all a balancing act, also between artistry and technical skills. However, I'd suggest a comparison between f/18 or f/ 20 versus f/22. It will probably not change the DOF much, while the improvement in overall image quality (based on the 40D camera specs) is almost a given (because deconvolution will be more effective, and also because you can use a shorter shutter speed).

Cheers,
Bart
Title: Re: Is the problem of diffraction over-rated?
Post by: David Sutton on March 16, 2013, 08:06:25 pm
However, I'd suggest a comparison between f/18 or f/ 20 versus f/22. It will probably not change the DOF much, while the improvement in overall image quality (based on the 40D camera specs) is almost a given

Yes, that's a sensible suggestion.
Cheers
Title: Re: Is the problem of diffraction over-rated?
Post by: Jack Hogan on March 18, 2013, 10:30:21 am
The challenges with deconvolution are:
a. Finding the optimal model of the Point spread function (PSF) with which to deconvolve,[/li][/list]

The good thing about deconvolution is that PSFs can usually be broken down into a product of subPSFs which can then be applied separately to the image in any desired sequence - much like any real number can be broken down into a product of prime numbers and applied commutatively in a division or multiplication.  The other good thing is that in any natural situation there will typically be a decent-sized random (Gaussian) component to the total PSF - which I believe is what most deconvolvers target and try do undo.  Having gotten the easy Gaussian component of PSF out of the way first, the next challenge is figuring out the other specific PSF components, which I assume include fixed ones like for aperture, AA, pixel pitch and shape - and who knows what other variable ones.

Aside from the varying difficulty of determining each of these 'component' PSFs (given focal length and f/number it should be pretty easy to calculate the aperture's for instance), I suspect the key for good looking capture sharpening is determining the appropriate strength of each and not getting carried away with the sliders.  In addition I think though that we should also not get carried away with trying to calculate PSFs too precisely, because our lenses and systems are not perfect, they cannot be modelled perfectly and, last but not least, we are not dealing with monochromatic light.  So we need to maintain enough wiggle room in the system to accommodate our real life situation lest our PSFs invent stuff that was never there in the first place.

I've made available a free tool (http://bvdwolf.home.xs4all.nl/main/foto/psf/SlantedEdge.html) to determine point a. for a given camera/lens/aperture combination. That will alow to determine the correct settings for Topaz InFocus ...

Nice tool Bart.  I like InFocus and use it (with a light hand) on many of my landscapes.  How would you use the output of your tool to determine the correct settings for it?

Jack
Title: Re: Is the problem of diffraction over-rated?
Post by: xpatUSA on March 18, 2013, 12:34:41 pm
. . . . I think though that we should also not get carried away with trying to calculate PSFs too precisely, because our lenses and systems are not perfect, they cannot be modelled perfectly and, last but not least, we are not dealing with monochromatic light.

One of my favorite images is this one, which is worth a look whenever one is getting too obsessive about detail. It predates DSLR's, I believe.

(http://kronometric.org/phot/mtf/50mmlenses.gif)

A little difficult interprete at first glance because the frequency (X) axis is normalized for each f-number. It shows how close an f-number gets to the dashed theoretical diffraction-limited curve. Quite some surprises there.

Ted
Title: Re: Is the problem of diffraction over-rated?
Post by: Bart_van_der_Wolf on March 18, 2013, 02:12:52 pm
The good thing about deconvolution is that PSFs can usually be broken down into a product of subPSFs which can then be applied separately to the image in any desired sequence - much like any real number can be broken down into a product of prime numbers and applied commutatively in a division or multiplication.  The other good thing is that in any natural situation there will typically be a decent-sized random (Gaussian) component to the total PSF - which I believe is what most deconvolvers target and try do undo.

Hi Jack,

It gets even better, convolving several different waveforms will converge to a Gaussian shaped distribution, even if the underlying waveforms are non-Gaussian. This chapter (http://www.dspguide.com/ch7/2.htm) of a free book on Digital Signal Processing explains the differences between parallel and sequential filtering in a bit more detail (check out the section on the Central Limit Theorem at the end).

Quote
Having gotten the easy Gaussian component of PSF out of the way first, the next challenge is figuring out the other specific PSF components, which I assume include fixed ones like for aperture, AA, pixel pitch and shape - and who knows what other variable ones.

That may not even be necessary if the main component is close enough to a single Gaussian (which it often is, as my tool will show). In my experience, the combination of several Gaussians can sometimes add a tiny bit more accuracy, but the improvement is usually marginal.

Quote
Aside from the varying difficulty of determining each of these 'component' PSFs (given focal length and f/number it should be pretty easy to calculate the aperture's for instance), I suspect the key for good looking capture sharpening is determining the appropriate strength of each and not getting carried away with the sliders.  In addition I think though that we should also not get carried away with trying to calculate PSFs too precisely, because our lenses and systems are not perfect, they cannot be modelled perfectly and, last but not least, we are not dealing with monochromatic light.  So we need to maintain enough wiggle room in the system to accommodate our real life situation lest our PSFs invent stuff that was never there in the first place.

The "800 pound gorilla" in the room are sensels that do area sampling instead of point sampling (even the diffraction pattern itself starts to look a bit more Gaussian due to area sampling), in combination with some 'defocus'. Anything not in the plane of focus will have a defocus blur added to its signal. That means that as we get closer to the edges of the DOF zone, the influence of defocus will start to grow (demanding a larger radius Gaussian based deconvolution). Combining the  overall effect of diffraction and aperture sampling of our sensels, will basically turn all blur sources combined into a rather Gausian like blur.

Quote
Nice tool Bart.  I like InFocus and use it (with a light hand) on many of my landscapes.  How would you use the output of your tool to determine the correct settings for it?

The important notion is that InFocus combines 2 operations, first a generic (or deblur) deconvolution, and second an optional sharpening operation. One should probably not try and solve the entire blur issue with only a single deconvolution (because you have no influence on the 'strength' of the effect), they are supposed to work in tandem (although I'd rather prefer more control over the deconvolution process). The optimal deconvolution radius setting seems to correspond reasonably well with the Sigma radius that my tools determines, maybe dialing in a tad smaller radius can help avoid the generation of excessive artifacts. It is not obvious which deconvolution method (generic/deblur) would be best. Deblur seems a bit more aggressive, it seems to be more than just a different PSF shape. Many unsatisfied reactions are caused by using too large a deconvolution radius. Assuming the two operations are executed in sequence, the sharpening radius to use should then be smaller than the deconvolution radius, unless one tries to achieve creative sharpening.

Cheers,
Bart
Title: Re: Is the problem of diffraction over-rated?
Post by: Wayne Fox on March 18, 2013, 06:50:28 pm
if I can't see it it doesn't matter. YMMV

Just because you can't see it so you don't know you lost it, does that necessarily mean it doesn't matter?

I agree with your overall premise that you can stop down much further than the optimal f/stop and use sharpening to get a great image.  But like many other things, at some point it becomes a trade off ... acceptable loss of some information for the sake of better information such as depth of field.  But doing a focus stack at a more optimal setting certainly may preserve some detail as discussed by Bart ...  may being the operative word based on the scene itself.

For me if things are static and I'm not in a hurry so I can focus stack I will usually do it.  But sometimes that's not possible and I'm certainly not going to skip shooting something just because I can't shoot it at f/8 or f/11.  I know at f/22 the d800 is very soft, hard to believe I'm not losing something.  16 is substantially better and I'm very comfortable shooting there.  On the Phase gear, f/22 is similar to f/16 on the nikon and even f/32 isn't as soft as the d800 is at 22, so I will use those when I need to.
Title: Re: Is the problem of diffraction over-rated?
Post by: David Sutton on March 19, 2013, 01:11:32 am
Hello Wayne. I quite agree. I too will continue to focus stack when I can. I like to choose from the different layers what will be sharp and what will be blurred.
Though I have certainly lost detail at f/22, if I can't see it in print up to around 20 x 30 inches does that necessarily mean it doesn't matter? Does it matter I don't know what has been lost?
There are so many trade-offs about which we have to make personal decisions. After using the same camera for over three years I have a good idea what to expect from it visually, so I can usually tell, for example, if I've hand held.  As I am more interested in exploring my memory of what I saw, and not so much interested in what the camera recorded (often a lot of pixel mangling involved) I'll take the "it doesn't matter" approach.  This wouldn't suit lots of folks.
So if I was only displaying on the web I'd certainly go to f/32. The fine detail is total mush but once posted on the web I can't see it on a 1900x1200 screen.

Title: Re: Is the problem of diffraction over-rated?
Post by: Wayne Fox on March 19, 2013, 02:13:33 am
Though I have certainly lost detail at f/22, if I can't see it in print up to around 20 x 30 inches does that necessarily mean it doesn't matter? Does it matter I don't know what has been lost?

But it seems maybe you are "assuming" you would lose it anyway when it goes to print so it doesn't matter  .  I guess I'm not sure that's always the case.  Sometimes you may lose it and it doesn't matter, but it seems there certainly may be times it would improve the image if it was preserved and it might be more visible than one might think.

I think we're basically on the same page ... just a different perspective on it.
Title: Re: Is the problem of diffraction over-rated?
Post by: ErikKaffehr on March 19, 2013, 02:43:22 am
Hi,

I would say that there is always a risk that sharpening to much may create artifacts instead of real detail.

Look at the images on this link: http://echophoto.dnsalias.net/ekr/index.php/photoarticles/49-dof-in-digital-pictures?start=2

You can see that the sharpened images are very crisp, but they lack "finesse" in form of fine detail.

Another example is here: http://echophoto.dnsalias.net/ekr/index.php/photoarticles/68-effects-of-diffraction (end of page).
The fine detail within the read square is lost, although sharpening is extensive.

Best regards
Erik



But it seems maybe you are "assuming" you would lose it anyway when it goes to print so it doesn't matter  .  I guess I'm not sure that's always the case.  Sometimes you may lose it and it doesn't matter, but it seems there certainly may be times it would improve the image if it was preserved and it might be more visible than one might think.

I think we're basically on the same page ... just a different perspective on it.
Title: Re: Is the problem of diffraction over-rated?
Post by: Walt Roycraft on March 19, 2013, 07:07:20 am
Erik, I just wanted say thanks for your information. Very helpful and much appreciated.
Title: Re: Is the problem of diffraction over-rated?
Post by: Wayne Fox on March 19, 2013, 08:12:58 pm
But otherwise if I want DOF f/22 will do fine as long as it doesn't drive my shutter speed down to around the half second area where the "thump" of the shutter on the 5DII shows up worst.

Not sure if you are aware but the 1st curtain of the 5DII is not mechanical but electronic.  This means if you use live view in place of mirror up, there are no moving parts in the exposure process until the end of the exposure when the second curtain releases.
Title: Re: Is the problem of diffraction over-rated?
Post by: JohnCox123 on March 20, 2013, 12:37:50 am
I want to point out that with an optimally sharpened image you won't notice diffraction. Every image should be sharpened optimally.
Title: Re: Is the problem of diffraction over-rated?
Post by: David Sutton on March 20, 2013, 02:49:06 am
Not sure if you are aware but the 1st curtain of the 5DII is not mechanical but electronic.  This means if you use live view in place of mirror up, there are no moving parts in the exposure process until the end of the exposure when the second curtain releases.
No I didn't know that. I realise I was thinking of my 40D which wouldn't give a sharp image at around half a second even with mirror lock-up. With the noise the 5DII makes I assumed it would be the same!
Title: Re: Is the problem of diffraction over-rated?
Post by: Fine_Art on March 21, 2013, 08:23:59 pm
Here is the resolution of the D600 with 85 1.8 ISO100 no sharpening.

The way this siemens star chart works for those not familiar with it is a circle of 92 pixels is the angular equivalent of the linear nyquist limit of 2 pixels to resolve a line. The 92 is from Bart's pick of 144 cycles around. I am not painting the circles in for you because the re-jpg would squash some detail.



Title: Re: Is the problem of diffraction over-rated?
Post by: xpatUSA on March 22, 2013, 12:15:53 am
Here is the resolution of the D600 with 85 1.8 ISO100 no sharpening.
Interesting images, thank you for taking the time to show them to us.

You were smart to darken the central area where printer moire can appear. I tried to post a series of the same target to demonstrate a lens "sweet spot" but forgot to smudge out the middle bit. Therefore, it all got very confusing when the discussion veered sadly off-course into the area of sensor resolution.

Title: Re: Is the problem of diffraction over-rated?
Post by: Fine_Art on March 22, 2013, 01:56:29 am
Interesting images, thank you for taking the time to show them to us.

You were smart to darken the central area where printer moire can appear. I tried to post a series of the same target to demonstrate a lens "sweet spot" but forgot to smudge out the middle bit. Therefore, it all got very confusing when the discussion veered sadly off-course into the area of sensor resolution.



Yeah, I had to re-do it from before. When I first looked at the images I thought the pattern I was getting was from the camera/lens. When I took the target down I noticed the same pattern from the laser printer. I think it was inside the 92 pixel circle so it was rubish but I felt it might give people the wrong impression. I jifffy markered out past the strangeness to make very sure anything anyone saw in the image was from the camera/lens.

I did the test to make sure I want to keep this system. I might as well post it for others to see. It may be my imagination, or the weak low pass filter on the sonys but diffraction seems to jump in fast on the APS-C sonys. I almost never go past f11. On this setup I really cant see much difference going from 1.8 to f16 and anywhere in between. The contrast certaily goes up f5.6-f8. The actual finest detail seems to be about the same. The system has pixels to spare with monster high ISO capability which I have never had before so it is a keeper. FF D600 for wide angles and A55 for telephoto. Works for me.  I like my minolta/sony primes. DxO is on crack for the low ratings they give those lenses. This nikon lens is very nice but it is not a big gulf from the Sony macros. And it is easy to get tack sharp pictures with the SLTs due to no mirror slap. With a regular camera you need to put up with the MLU delay to get similar. Think about getting 10fps with MLU...

Title: Re: Is the problem of diffraction over-rated?
Post by: Bart_van_der_Wolf on March 22, 2013, 05:31:20 am
You were smart to darken the central area where printer moire can appear. I tried to post a series of the same target to demonstrate a lens "sweet spot" but forgot to smudge out the middle bit. Therefore, it all got very confusing when the discussion veered sadly off-course into the area of sensor resolution.

Hi Ted,

Discussions can go off-topic when people don't understand what they are looking at. The target itself is already a challenge for the printer (and can be used for that as well, using a loupe and calipers to measure the blur diameter), but that also gives information about whether the print is offering reasonably more detail than the lens can resolve, and how the Raw converter handles that aliasing detail within the 92 pixel Nyquist diameter. To keep that piece of additional information (as proof that the lens is limiting the resolution, not the print), I generally draw a 92 pixel diameter circle, but when it distracts, indeed one can cover the center with a disk.

What Arthur's shots also show is how contrast reduces as the detail approached its resolution limit, just like we can see in an MTF curve. With proper deconvolution sharpening the micro detail contrast will be increased and the resolution that is there will be easier to see, without exaggerating things. The resulting images will look more crisp, more realistic (because now our eyes will become the limiting factor, just like in real life).

Cheers,
Bart
Title: Re: Is the problem of diffraction over-rated?
Post by: Bart_van_der_Wolf on March 22, 2013, 05:58:47 am
I did the test to make sure I want to keep this system. I might as well post it for others to see. It may be my imagination, or the weak low pass filter on the sonys but diffraction seems to jump in fast on the APS-C sonys.

Hi Arthur,

It just means that the system is accurate enough to show that diffraction will progressively reduce contrast towards the Nyquist limit, which is exactly what happens. At the wide open side of the aperture range contrast may (additionally) be limited by residual lens aberrations that are overpowering the diffraction effects, but diffraction is always present.

When the input detail has lower contrast than this average target contrast of almost 100:1, then it will hit the zero contrast output level (which is beyond recovery) faster.

Cheers,
Bart
Title: Re: Is the problem of diffraction over-rated?
Post by: xpatUSA on March 22, 2013, 09:11:54 am
. . . also gives information about whether the print is offering reasonably more detail than the lens can resolve, and how the Raw converter handles that aliasing detail within the 92 pixel Nyquist diameter. To keep that piece of additional information (as proof that the lens is limiting the resolution, not the print), I generally draw a 92 pixel diameter circle, but when it distracts, indeed one can cover the center with a disk.

Thanks Bart,
Probably a regular question for you:
Is the physical diameter of that circle constant or does it depend on sensor spatial res. and taking distance? I understand that taking distance is not important within reasonable limits and why that is so but I'm thinking that the diameter of the 'Nyquist circle' in actual mm would vary?

[edit] I guess it all depends on what is meant by 'pixel', original target image (gif) pixels or the captured image (sensor) pixels. Now I'm getting muddled up . . any artifacts inside the said Nyquist circle are sensor (or printer) related are to do with the camera and not a lens under test. Which is why I find lens testing on my 9.12um sensor pixel pitch somewhat less than revealing . . . lots of moir[ay] though!

Quote
With proper deconvolution sharpening the micro detail contrast will be increased and the resolution that is there will be easier to see, without exaggerating things. The resulting images will look more crisp, more realistic (because now our eyes will become the limiting factor, just like in real life).

Interesting comment about our eyes, with which I agree. Makes you wonder how this gentleman, talking about Sigma Merrill shots, is viewing his images . . .

http://forums.dpreview.com/forums/thread/3450887

Well, he does mention "looking at an image at 100%" so that might be a clue  ;)

Ted
Title: Re: Is the problem of diffraction over-rated?
Post by: bjanes on March 22, 2013, 10:08:37 am
The target itself is already a challenge for the printer (and can be used for that as well, using a loupe and calipers to measure the blur diameter), but that also gives information about whether the print is offering reasonably more detail than the lens can resolve, and how the Raw converter handles that aliasing detail within the 92 pixel Nyquist diameter. To keep that piece of additional information (as proof that the lens is limiting the resolution, not the print), I generally draw a 92 pixel diameter circle, but when it distracts, indeed one can cover the center with a disk.

Bart,

Since your target is a challenge for the printer, wouldn't it make sense to provide a target at higher resolution so that one could print at a size larger than 130x130 mm when using the maximum resolution of the printer? For example, I am using the Epson 3880 which has a native resolution of 720 ppi, so if I could print at 260x260 mm at this resolution, I would effectively have twice the resolution. My printer does resolve the 3 resolutions at all contrast levels on your revised chart, but the lines are not that sharply defined.

Regards,

Bill
Title: Re: Is the problem of diffraction over-rated?
Post by: Bart_van_der_Wolf on March 22, 2013, 11:53:44 am
Thanks Bart,
Probably a regular question for you:
Is the physical diameter of that circle constant or does it depend on sensor spatial res. and taking distance? I understand that taking distance is not important within reasonable limits and why that is so but I'm thinking that the diameter of the 'Nyquist circle' in actual mm would vary?

[edit] I guess it depends on what is meant by 'pixels', original target image pixels or the captured image pixels.

Hi Ted,

The diameter of the observed/captured blur disc is set by the limiting resolution of the observer/capture device. When shot with a discrete sampling device such as an image sensor, it will be constant, a minimum of 91.67 pixels in diameter at the Nyquist frequency for a target with 144 cycles (144 / Pi x 2). The sensor resolution is limited at it's Nyquist frequency, no smaller detail then 2 pixels per cycle (= 0.5 cycles/pixel) can be reliably recorded, but detail is often reduced to zero contrast at slightly larger detail size. There may be some accidental alignment of finer detail with a single sensel, causing a local high or low, but that will produce aliasing (unless the contrast is reduced, e.g. by an AA-filter) detail will seem to grow in size which will look as a deviation in predictable patterns.

So it's the captured image pixels that will show a constant diameter limit. The printed target will exhibit (sinusoidal grating + a few 1/2/3 linewidth) features all the way down to the pixel limits of the printer (where it struggles and reveals the effects of diffusion and loss of color accuracy due to lack of sufficient ink colors for dithering).

As long as these real features are shot with a small enough magnification factor due to shooting distance (>25x focal length), those smallest features will be smaller than the Nyquist frequency can resolve. Unfortunately, the artifacts at the printing limits will be relatively easy to spot, because they became larger than the finest details, a bit similar like aliasing is larger than the finest detail, and darker due to ink diffusion. Ink diffusion will make shadow detail blend together, and the target will predict how serious it can be.

Cheers,
Bart
Title: Re: Is the problem of diffraction over-rated?
Post by: Bart_van_der_Wolf on March 22, 2013, 12:10:29 pm
Since your target is a challenge for the printer, wouldn't it make sense to provide a target at higher resolution so that one could print at a size larger than 130x130 mm when using the maximum resolution of the printer? For example, I am using the Epson 3880 which has a native resolution of 720 ppi, so if I could print at 260x260 mm at this resolution, I would effectively have twice the resolution. My printer does resolve the 3 resolutions at all contrast levels on your revised chart, but the lines are not that sharply defined.

Hi Bill,

No, the printer is doing its best at 720 PPI, and is (barely) resolving detail and correct gray levels at that level (lack of ink colors to dither intermediate tones plus ink diffusion filling the paper white areas). That's not a problem, it's just struggling with physics. More accurate droplet placement, good head alignment, accurate paper transport, and good dithering can reduce this effect to a minimum, but its inherent to the process.

To make life easier for the printer, one could print the full target at 360 PPI effective resolution to a size of 260mm square, but then the finest detail that is smaller than 360PPI will also become larger, and the target will need to be shot at twice the distance to make sure that the lens/sensor cannot resolve this larger detail. That may cause impractical indoor shooting distances for longer focal lengths.

The Nyquist frequency of the captured shot will remain at 92 pixels diameter, regardless of the shooting distance (assuming finer detail than can be resolved due to magnification factor).

Cheers,
Bart
Title: Re: Is the problem of diffraction over-rated?
Post by: Bart_van_der_Wolf on March 22, 2013, 12:23:16 pm
Interesting comment about our eyes, with which I agree. Makes you wonder how this gentleman, talking about Sigma Merrill shots, is viewing his images . . .

http://forums.dpreview.com/forums/thread/3450887

Well, what he is experiencing is that aliasing at the limiting resolution of a discrete sampling device (especially when viewed on a low resolution display) looks a bit different than what we normally see with our eyes. What we can achieve with deconvolution, is restoration of real detail (assuming a more or less anti-aliased image).

Cheers,
Bart
Title: Re: Is the problem of diffraction over-rated?
Post by: Fine_Art on March 22, 2013, 05:33:51 pm
Here is the same f16 file as above with RL sharpening, micro-contrast on, contrast by detail on. If I really needed to mess with it I would export to IPlus. I think the fine detail is as much as you can ask for like this. More would start to look unnatural.
Title: Re: Is the problem of diffraction over-rated?
Post by: Fine_Art on March 22, 2013, 05:51:32 pm
Hi Arthur,

It just means that the system is accurate enough to show that diffraction will progressively reduce contrast towards the Nyquist limit, which is exactly what happens. At the wide open side of the aperture range contrast may (additionally) be limited by residual lens aberrations that are overpowering the diffraction effects, but diffraction is always present.

When the input detail has lower contrast than this average target contrast of almost 100:1, then it will hit the zero contrast output level (which is beyond recovery) faster.

Cheers,
Bart

Correct me if my understanding is off here. The diffraction spot size is a function of aperture so using a given size like f8 will show the effects faster on the smaller pixels of an APS-C sensor than on an FF like this D600.
D600 5.95 micron pixels
A55   4.78 micron pixels

Airy disk at f8 is 10.7 microns so it spills over into adjacent pixels on both cameras. F16 is 21.5 microns. Diffraction should be an issue.

Looking at the unsharpened images they have a similar detail extinction radius. They will all sharpen to about the same amount.

Title: Re: Is the problem of diffraction over-rated?
Post by: Fine_Art on March 22, 2013, 06:46:26 pm
Of course I didn't go out to just do the resolution charts.  ;D  Here is a real shot with a 100% crop from the far left side. f16 with focus on the front tree.

Title: Re: Is the problem of diffraction over-rated?
Post by: Bart_van_der_Wolf on March 22, 2013, 08:35:38 pm
Correct me if my understanding is off here. The diffraction spot size is a function of aperture so using a given size like f8 will show the effects faster on the smaller pixels of an APS-C sensor than on an FF like this D600.
D600 5.95 micron pixels
A55   4.78 micron pixels

Airy disk at f8 is 10.7 microns so it spills over into adjacent pixels on both cameras. F16 is 21.5 microns. Diffraction should be an issue.

That's correct. The absolute size of the diffraction pattern (usually measured at the diameter of the first zero of the Airy disk pattern) has a given size, determined by the aperture number and wavelength. Sensor arrays with a smaller sensel pitch will therefore use more sensels to sample that same diffraction pattern. That also means that the center weighted sampling of the blur pattern will be more accurate (which is good for deconvolution), but since more samples get averaged per unit area the individual sensels will be blurrier.

Quote
Looking at the unsharpened images they have a similar detail extinction radius. They will all sharpen to about the same amount.

That is not exactly what I see. I see lower contrast signals near Nyquist, and a slightly larger bur diameter as the aperture gets narrower (although I don't understand the shot labeled as f/16). On your f/1.8 shot I'd call it a 93x93 pixel blur diameter, at f/8 it's a 94 or 95 pixel diameter, and at f/16 I'd give it a 91x93 pixel diameter (which is frankly a bit strange if it really is an f/16 shot, there is way too much aliasing for an f/16 aperture). RawTherapee is good, but it can't defy physics.

Since it is not the full resolution pattern (http://www.openphotographyforums.com/forums/showthread.php?t=13217) that I proposed, but only a square star area, I cannot verify if it was printed at a high enough resolution, or shot at a large enough distance. So maybe the features are larger than they should be and less challenging for the sensor, making the sensor set the limit and not the target?

Cheers,
Bart
Title: Re: Is the problem of diffraction over-rated?
Post by: Fine_Art on March 22, 2013, 10:07:35 pm
I did use your pattern with the edge blocks cropped off. This let me print it bigger on a 600dpi laser 8.5"x11" sheet. I up-rezed it in PS to generate the required number of pixels. In the top left corner of the screenshot you can see the full image where the piece of paper is a small rectangle. Anyone can calculate range with the focal length from that. I would estimate from memory 70 to 80 ft away. I am far enough away that the whole center is blurred out. I am sure that is not an issue. I can send the raws to you or someone at the site for people to convert with their own preferred software. The shots are of no artistic merit so copyright is not an issue! ;) I can also convert a raw image you send me to see if it is the software.

I get similar close to nyquist with my Sony macros or the G lens. the kit zoom or other zooms are mush in comparison. I believe all sonys have weak aa filters.

This is the reason i made the post as A - diffraction should be an issue, B - the pictures show not much difference, A and B should not both be correct unless diffraction is not the overriding issue.

In my past study of artifacts (the color noise thing) I went down the rathole of looking at a wide assortment of de-bayer methods. It quickly became a mess of academic papers that were beyond where I wanted to go. One useful thing did come out of that, an idea of the difficulties in de-bayering. I suspect the local area search of the best algorithms followed by rules of interpretation may be making the diffraction issue secondary. Especially when lines are involved predictions (fake detail) are pushed forward. This will tend to beat the standard ISO resolution type charts.

Here is a link to the version of RT I am using, feel free to try it.
http://www.rawtherapee.com/forum/viewtopic.php?f=15&t=4507 (http://www.rawtherapee.com/forum/viewtopic.php?f=15&t=4507)
Title: Re: Is the problem of diffraction over-rated?
Post by: Fine_Art on March 22, 2013, 10:20:44 pm
Here is one I re-found

http://scien.stanford.edu/pages/labsite/2007/psych221/projects/07/Dargahi&Deshpande.pdf (http://scien.stanford.edu/pages/labsite/2007/psych221/projects/07/Dargahi&Deshpande.pdf)

de-bayering is actually much tougher than the bilinear or bicubic we expect.
Title: Re: Is the problem of diffraction over-rated?
Post by: Fine_Art on March 22, 2013, 10:47:43 pm
Here is the f16 raw I uploaded. At the bottom of the page there is a blue link with the actual file. Sendspace is flogging their toolbar, they try to get you to click on their toolbar or add links.  MAKE SURE TO HOVER YOUR MOUSE OVER THE LINK TO MAKE SURE IT IS THE NEF FILE, NOT THEIR ADDS.

http://www.sendspace.com/file/pljyeg (http://www.sendspace.com/file/pljyeg)

I am also attaching the RT processing parameters sidecar file below. This will load the setting I used. Try it, move the sliders around, you may find something better.
Title: Re: Is the problem of diffraction over-rated?
Post by: Bart_van_der_Wolf on March 23, 2013, 06:57:57 am
I did use your pattern with the edge blocks cropped off. This let me print it bigger on a 600dpi laser 8.5"x11" sheet. I up-rezed it in PS to generate the required number of pixels.

Hi Arthur,

Frankly, I feel a bit uneasy when people start resampling a carefully designed pattern, and then get unexpected results ...

I have never, until now, seen an f/16 image that defies physics like your example. Even RawTherapee, which is very good (I like the Amaze algorithm very much), can't make a silk purse (aliasing, so detail beyond Nyquist) out of a sow's ear (no signal / only noise ratio due to diffraction).

Quote
In the top left corner of the screenshot you can see the full image where the piece of paper is a small rectangle. Anyone can calculate range with the focal length from that. I would estimate from memory 70 to 80 ft away. I am far enough away that the whole center is blurred out. I am sure that is not an issue. I can send the raws to you or someone at the site for people to convert with their own preferred software. The shots are of no artistic merit so copyright is not an issue! ;) I can also convert a raw image you send me to see if it is the software.

I suspect it's rather something with the target than the Raw conversion process. A laser printer is also not the best device to produce continuous tone images with subtle gradients. The slanted edges might be usable though, but they were cropped off.

Quote
This is the reason i made the post as A - diffraction should be an issue, B - the pictures show not much difference, A and B should not both be correct unless diffraction is not the overriding issue.


Exactly. Something is wrong, although it would be nice if there was no ill effect from diffraction, but unfortunately diffraction actually does hurt image quality.

Quote
In my past study of artifacts (the color noise thing) I went down the rathole of looking at a wide assortment of de-bayer methods. It quickly became a mess of academic papers that were beyond where I wanted to go. One useful thing did come out of that, an idea of the difficulties in de-bayering. I suspect the local area search of the best algorithms followed by rules of interpretation may be making the diffraction issue secondary. Especially when lines are involved predictions (fake detail) are pushed forward. This will tend to beat the standard ISO resolution type charts.

That would be the case, but algorithms like Amaze are (just like most others) optimizing a delicate balance between artifacts and detail. They do not invent detail where there is none to begin with. They are not in the business of doing single image super-resolution which uses resized samples of features that are located elsewhere in the image. All these demosaicing algorithms need some signal in a restricted local area (5x5 or 7x7 samples) to produce a luminosity estimate that differentiates features from surrounding background.

Quote
Here is a link to the version of RT I am using, feel free to try it.
http://www.rawtherapee.com/forum/viewtopic.php?f=15&t=4507 (http://www.rawtherapee.com/forum/viewtopic.php?f=15&t=4507)

Yes, I've been trying to get that 64bit Win 7 version for a while already, but the zip file is reported to be corrupt. I'll wait for an official build on the RawTherapee website (http://rawtherapee.com/downloads).

Again, it's not the Raw converter that is suspect.

Cheers,
Bart

P.S. I've received a reaction from the author of the website that's mentioned, and that he's checked his file and found it to be okay. So I'll try downloading with another browser. Anyway, the previous RT version I already have installed also gives very high resolution results, so I don't expect that to make much difference. 'Amaze' is amazing in how clean the high resolution conversion result is, and 3 FC suppression steps also takes care of most of the false color artifacting.

It does look like your camera and lens combination are a marriage made in heaven ...
Title: Re: Is the problem of diffraction over-rated?
Post by: Bart_van_der_Wolf on March 23, 2013, 07:11:31 am
Here is one I re-found

http://scien.stanford.edu/pages/labsite/2007/psych221/projects/07/Dargahi&Deshpande.pdf (http://scien.stanford.edu/pages/labsite/2007/psych221/projects/07/Dargahi&Deshpande.pdf)

de-bayering is actually much tougher than the bilinear or bicubic we expect.

Thanks for that link.

Cheers,
Bart
Title: Re: Is the problem of diffraction over-rated?
Post by: Fine_Art on March 23, 2013, 12:12:56 pm
Bart,

The lines in the printout are converging into finer detail. Nothing is going to change that. You see the lines get finer to the point they blur together. The target is distance invariant. The pixels are not going to resolve more by re-sampling. If you dont trust the laser printout look at the fine lines on the cracked paint. I put the target on that building in the park for exactly that reason, the random fine lines of the paint cracks.

You do understand that RT has built in deconvolution? In the unsharpened images it is not turned on. In the one sharpened image I attached later it is using R-L deconvolution. It is also using "micro-contrast" and "contrast by detail" which I assume is wavelets. You can see the difference at the edge of the target where the page is white. On the unsharpened files the image just goes white off the pattern. In the sharpened file you see the pattern bleed into the white area. That is invented detail. Any of these methods that use "variable gradients" are making predictions. Roger Clark talked about invented detail in digital years back when he did the comparison velvia drum scanned vs digital. He showed zooms of reeds where some of the apparent digital detail did not exist in the drum scan at higher res. I am not talking artifacts, I am talking a few ghost stalks of reeds.

I believe diffraction is no longer an issue. It obviously has not gone away, it is being predicted out by gradient type de-bayer along with all the sharpening type routines. By definition de-bayer has to figure out how to fill holes. The best routines are filling the diffraction blur hole.
Title: Re: Is the problem of diffraction over-rated?
Post by: Bart_van_der_Wolf on March 23, 2013, 04:43:19 pm
Bart,

The lines in the printout are converging into finer detail. Nothing is going to change that. You see the lines get finer to the point they blur together. The target is distance invariant. The pixels are not going to resolve more by re-sampling.

Hi Arthur,

While that is correct, upsampling does lose microdetail contrast. Therefore the target contrast may be a bit (can't tell if and how much) lower at the highest level of detail, making it easier for the camera to not develop aliasing. Because it's not possible to compare with the original, I can't judge if and how much of an influence it has. All I know (from other PM exchanges) is that a lower resolution target can influence the outcome of the Slanted edge score, which is of course much more sensitive than the visual star target. It was even possible to detect a difference in blur sigma between the left and right side of the target, because it was shot at a 1 degree angle off perpendicular.

Quote
If you dont trust the laser printout look at the fine lines on the cracked paint. I put the target on that building in the park for exactly that reason, the random fine lines of the paint cracks.

Well that's another issue that's often overlooked, diffraction first kills the lowest contrast microdetail, before it kills the higher contrast microdetail. One may be able to restore a certain level of detail, but some is already lost. From the looks of it, your camera+lens+Rawconverter combination seems to do a very good job, and strike a nice balance. Good for you.

Quote
You do understand that RT has built in deconvolution? In the unsharpened images it is not turned on. In the one sharpened image I attached later it is using R-L deconvolution.

Not only do I understand it, I pointed it out to a lot of folks who didn't know that.

Quote
It is also using "micro-contrast" and "contrast by detail" which I assume is wavelets. You can see the difference at the edge of the target where the page is white. On the unsharpened files the image just goes white off the pattern. In the sharpened file you see the pattern bleed into the white area. That is invented detail.

Yes, the amount of control is super useful, and effective. Not something for those who get intimidated easily by such features though.

Quote
Any of these methods that use "variable gradients" are making predictions. Roger Clark talked about invented detail in digital years back when he did the comparison velvia drum scanned vs digital. He showed zooms of reeds where some of the apparent digital detail did not exist in the drum scan at higher res. I am not talking artifacts, I am talking a few ghost stalks of reeds.

They are artifacts though, and Roger didn't say they weren't (it's mentioned at the bottom of this section (http://www.clarkvision.com/articles/scandetail/index.html#digi1camres1)). The demosaicing algorithms back then were not as advanced as what we have available today.

Quote
I believe diffraction is no longer an issue. It obviously has not gone away, it is being predicted out by gradient type de-bayer along with all the sharpening type routines. By definition de-bayer has to figure out how to fill holes. The best routines are filling the diffraction blur hole.

I wouldn't generalize a specific (specific very sharp lens / camera sensor with mild AA-filter and a not too small 5.97 micron sensel pitch / a very effective demosaicing algorithm / a specific level of contrast) situation, as if it were universally applicable.

What the star target learns us is that for this combination of components, apparently f/16 still produces good visual detail, approaching the Nyquist limit. Deconvolution sharpening with a relatively small radius can boost the signal to noise ratio to even less of a contrast loss near the limiting resolution, which can help e.g. with producing large output. Even the aliasing seems to be behaving quite nicely, thanks to the Amaze algorithm, so some of it may go unnoticed as false detail.

It looks like a very fortunate combination, congratulations. Diffraction is less of a consideration when you use this lens, so you can focus on other elements that make the shot.

Cheers,
Bart
Title: Re: Is the problem of diffraction over-rated?
Post by: Fine_Art on March 23, 2013, 09:13:32 pm
With all respect for the many things I have learned from your posts, I do not think it is this lens or lens/camera combo. Without question it is a fine lens and a fine camera. It was DxOs top rated lens in their article normal to short tele which is why I got it. However great it is, like you said before, it cannot overcome physics. It must be the software routines.

Here is the typical level of detail I get with the old Minolta 50 macro and this software. It's similar for the 100 or 300 f4G. At higher f ratios it needs more sharpening. Any good camera and sharp prime lens from any of the manufacturers will produce similar with a good tripod and remote release. Anyone getting much worse detail under those circumstances is using the wrong software.

Edit: Let me put it this way, I used to use other software then use IPlus to sharpen and remove noise. When I have tried that with this version of RT the image does not improve. Artifacts grow. When I try to remove noise it doesnt help. I get a more natural looking (fine noiseless film look) at the expense of a much softer image. There is no point in saving the result.
Title: Re: Is the problem of diffraction over-rated?
Post by: Vladimirovich on March 23, 2013, 09:52:29 pm
Yes, I've been trying to get that 64bit Win 7 version for a while already, but the zip file is reported to be corrupt. I'll wait for an official build on the RawTherapee website (http://rawtherapee.com/downloads).

actually the good place to download is here = http://www.visualbakery.com/RawTherapee/Downloads.aspx

Title: Re: Is the problem of diffraction over-rated?
Post by: Bart_van_der_Wolf on March 23, 2013, 10:41:00 pm
actually the good place to download is here = http://www.visualbakery.com/RawTherapee/Downloads.aspx

Yes, that's where the actual download is available. Oliver was very helpful in nailing my cause of failed zip files, I used a different browser, and all is fine now.

Cheers,
Bart
Title: Re: Is the problem of diffraction over-rated?
Post by: Bart_van_der_Wolf on March 23, 2013, 11:23:47 pm
With all respect for the many things I have learned from your posts, I do not think it is this lens or lens/camera combo. Without question it is a fine lens and a fine camera. It was DxOs top rated lens in their article normal to short tele which is why I got it. However great it is, like you said before, it cannot overcome physics. It must be the software routines.

Okay, here's (attached) a case in point (which I've maybe hinted at too cursory) to consider. Check out the wood grain structure at the patch in the approx. 8 o'clock position relative to the star target of the file you made available. It has lower contrast than the star target, and it has fine low micro-contrast detail, and it kind of fades in and out of being resolved and not resolved. I'd hate to deliver such (non-)detail to a commercial customer who's passion is in wood grain related materials ...

It would be interesting to compare to the 'better' apertures with the same Raw conversion settings.

Cheers,
Bart
Title: Re: Is the problem of diffraction over-rated?
Post by: Fine_Art on March 24, 2013, 01:56:52 am
You picked something that you know would be particularly hard for a de-bayer, low contrast, fine red lines. 3 pixels of 4 will be filled by the de-bayer from being filtered out. I will upload the raws for you. You have me curious too.
Title: Re: Is the problem of diffraction over-rated?
Post by: Fine_Art on March 24, 2013, 02:10:59 am
http://www.sendspace.com/file/95maip (http://www.sendspace.com/file/95maip)

Anyone using this, remember to hover your mouse over the link at the bottom of the page to make sure it is a .nef .
Title: Re: Is the problem of diffraction over-rated?
Post by: Bart_van_der_Wolf on March 24, 2013, 09:32:22 am
http://www.sendspace.com/file/95maip (http://www.sendspace.com/file/95maip)

Anyone using this, remember to hover your mouse over the link at the bottom of the page to make sure it is a .nef .

Thanks for making it available. The relative differences are subtle (when we disregard the precipitation) only a tiny bit in favor of f/8, as could be expected because the f/16 shot was still pretty good. The difference between these f/8 and f/16 shots would under most circumstances not be detectable in print.

Personally, to show diffraction effects, I draw a line at the aperture where 'diameter' of the diffraction pattern exceeds 1.5x the sensel pitch. In this camera's case that would be f/6.3, where the diffraction pattern hardly affects neighboring sensels when features are aligned with the sensel grid. If the residual aberrations are well corrected, then that's where I expect the lens' sweetspot to be, with this AA-filter and sensor. It would be close to producing the highest resolution possible per pixel, have a good correction of residual lens aberrations (also in the corners), and probably as little vignetting as possible.

Cheers,
Bart
Title: Re: Is the problem of diffraction over-rated?
Post by: bjanes on March 24, 2013, 11:45:55 am
Well that's another issue that's often overlooked, diffraction first kills the lowest contrast microdetail, before it kills the higher contrast microdetail. One may be able to restore a certain level of detail, but some is already lost. From the looks of it, your camera+lens+Rawconverter combination seems to do a very good job, and strike a nice balance. Good for you.

What the star target learns us is that for this combination of components, apparently f/16 still produces good visual detail, approaching the Nyquist limit. Deconvolution sharpening with a relatively small radius can boost the signal to noise ratio to even less of a contrast loss near the limiting resolution, which can help e.g. with producing large output. Even the aliasing seems to be behaving quite nicely, thanks to the Amaze algorithm, so some of it may go unnoticed as false detail.

Bart,

Your star chart is an excellent tool for quantitative analysis of resolution, but I remember from a previous discussion with you, it is high contrast and measures resolution near the Rayleigh limit (usually stated to represent ~10% MTF, but somewhat higher according to an analysis that you published). In real world photography, we often deal with lower contrast than is present in the star target, and the quantitative analysis should be supplemented by subjective analysis of the image for micro-contrast and "sparkle". DigLloyd (http://www.diglloyd.com/index.html) (a pay site, but well worth the modest cost) publishes extensive abundantly illustrated subjective studies using top lenses such as the Coastal Optics 60 mm f/4 APO and the Zeiss 135 mm f/2 APO on various cameras with and without low pass filters, including the Nikon D7100 which lacks a low pass filter and the pixel pitch corresponds to a 51 MP full frame sensor (allegedly coming from Nikon later this fall).

In general, he concludes that for a D800e type sensor, f/5.6 is the smallest aperture yielding maximal image quality, even after aggressive deconvolution sharpening. He does not state the deconvolution algorithm he uses or the settings. In his preview of the new Nikon 80-400 mm f/4.5-5.6, he states that for critical work, the lens is essentially a one aperture deal and it had better be good at f/5.6.

Okay, here's (attached) a case in point (which I've maybe hinted at too cursory) to consider. Check out the wood grain structure at the patch in the approx. 8 o'clock position relative to the star target of the file you made available. It has lower contrast than the star target, and it has fine low micro-contrast detail, and it kind of fades in and out of being resolved and not resolved. I'd hate to deliver such (non-)detail to a commercial customer who's passion is in wood grain related materials ...

It would be interesting to compare to the 'better' apertures with the same Raw conversion settings.


The above observation points out that subjective analysis does provide information beyond the resolution limit derived from your star chart. Deconvolution sharpening is very helpful in recovering contrast at smaller apertures (large f/stop numbers), but as you have pointed out, it can not recover low contrast high frequency data. For critical work, I suggest that an aperture of 5.6 is likely optimal with the D800e. What do you think?

Best Regards,

Bill
Title: Re: Is the problem of diffraction over-rated?
Post by: Jack Hogan on March 24, 2013, 04:12:55 pm
I suggest that an aperture of 5.6 is likely optimal with the D800e. What do you think?
That seems to be the sweet spot for most lenses to be used with the APS-C/FF formats.  In zooms it tends to move towards f/8 once focal length increases.

Jack
Title: Re: Is the problem of diffraction over-rated?
Post by: Fine_Art on March 24, 2013, 06:04:26 pm
Here are the others as promised,

f5.6
http://www.sendspace.com/file/xv3czj

f4
http://www.sendspace.com/file/6yi7uw

f3.2
http://www.sendspace.com/file/gqrsz6

f1.8
http://www.sendspace.com/file/sm7lmh

The differences are small. f4 to f5.6 does seem to be the best.

If you are opening these in RT you see the center gray blur circle shrink as you turn on each of the following features Sharpening RL, Micro-contrast, Detail by contrast. Also go back and forth between RL sharpening and USM. The RL makes the pattern go black and white high contrast with not much propagation of the lines in. The USM seems to leave the whole thing a bit gray with much stronger prediction of the lines, the faint blur circle moves in quite a bit more.
Title: Re: Is the problem of diffraction over-rated?
Post by: Bart_van_der_Wolf on March 24, 2013, 07:23:55 pm
The above observation points out that subjective analysis does provide information beyond the resolution limit derived from your star chart. Deconvolution sharpening is very helpful in recovering contrast at smaller apertures (large f/stop numbers), but as you have pointed out, it can not recover low contrast high frequency data. For critical work, I suggest that an aperture of 5.6 is likely optimal with the D800e. What do you think?

Hi Bill,

For a D800/D800E sensor array with an approx. 4.88 micron senselpitch, f/5.6 produces a diffraction pattern (green light) with a diameter of almost exactly 1.5x the sensel pitch. It's the magic number I've been mentioning, where the onset of visible diffraction starts (in the low contrast microdetail). That leads to a simple rule of thumb, sensel pitch in microns x 1.11 = F-number where visible diffraction (for green wavelengths) begins.

However, some lenses perform even better in the center at f/4 because they have very few residual lens aberrations even wide open, and f/4 creates even less diffraction than f/5.6. But still, the corners of the image may benefit from stopping down a bit further, so it ultimately boils down to a compromise based on the intended use of the lens. A portrait lens can perhaps get away with softer corners, but a lens for architecture or reproduction can not.

Cheers,
Bart
Title: Re: Is the problem of diffraction over-rated?
Post by: Jim Kasson on March 24, 2013, 08:00:04 pm
That leads to a simple rule of thumb, sensel pitch in microns x 1.11 = F-number where visible diffraction (for green wavelengths) begins.

Bart,

You got me thinking with that one. Can you turn the reasoning on its head?

Thinking ahead to sensors with really fine pitches, can you come up with a similar rule of thumb that takes the pixel pitch as the input and and spits out the f-stop beyond which you'll never (or almost never, you chose the contrast when to stop worrying) see any aliasing? By "beyond", I mean numerically larger, or of smaller diameter. Assume a Bayer CFA and no anti-aliasing filter.

Thanks,

Jim
Title: Re: Is the problem of diffraction over-rated?
Post by: Bart_van_der_Wolf on March 24, 2013, 10:41:39 pm
Bart,

You got me thinking with that one. Can you turn the reasoning on its head?

Hi Jim,

Sure. The diameter of the diffraction pattern (first zero) is usually defined as  2.44 x wavelength x F-number.

So e.g. 2.44 x 0.550 x 5.6 ~= 7.5152 micron, or ~1.54 x sensel pitch (4.88 micron for a D800(E), close enough to a 1.5x factor for my taste).

That equates roughly to:  F-number = sensel pitch x 1.5 / (2.44 x 0.550), or sensel pitch x 1.12 .
I tend to round the multiplier a tad down to 1.11 to increase the weight of the longer wavelengths (such as the earlier scrutinized woodgrain) a bit. Afterall, Luminance is roughly considered equal to 0.3 x R + 0.59 x G + 0.11 x B , so Green and Red account for 89% contribution to Luminance.

Quote
Thinking ahead to sensors with really fine pitches, can you come up with a similar rule of thumb that takes the pixel pitch as the input and and spits out the f-stop beyond which you'll never (or almost never, you chose the contrast when to stop worrying) see any aliasing? By "beyond", I mean numerically larger, or of smaller diameter. Assume a Bayer CFA and no anti-aliasing filter.

The totally diffraction limited resolution (all higher spatial frequency input will achieve zero modulation) is determined by, according to Jacobson, see here (http://photo.net/learn/optics/lensTutorial#part4):
Quote
Note that for lambda = 555 nm, the OTF is zero at spatial frequencies of 1801/N cycles per mm and beyond.

Which translates to 3.602 x sensel pitch in microns = narrowest aperture.

Do note that Jacobson's calculations do not account for the averaging effect of our area sampling sensels, which may take an additional toll of perhaps 1/3rd of an aperture stop. He also considers the additional effect of the aperture's diffraction (which is harder to generalize). Anyway, an absolute physical maximum of 3.6 x sensel pitch is also a useful figure to know (no aliasing possible when modulation of higher spatial frequencies is zero).

So, for diffraction affected resolution, it seems we have a range of 1.11 to 3.6 times the sensel pitch (in microns) as our physically limited creative playground. This all is of course hugely simplified, as if we shoot with monochromatic light, but it does provide useful benchmarks and reality checks.

Cheers,
Bart
Title: Re: Is the problem of diffraction over-rated?
Post by: xpatUSA on March 25, 2013, 04:37:18 am
Been following the discussion which I find very interesting.

A while back I wrote a spreadsheet to calculate diffraction for macro shots which was informative but slightly boring so I changed it to calculate MTF from wavelength, aperture (infinite focus) and pixel pitch. Much more fun.

I entered 4.88um pixel pitch, detail at 1/2 Nyquist frequency, f/6.3 and (just for grins) some seriously red light. Here's a screen capture:

(http://kronometric.org/phot/mtf/mtfScreenShot.gif)

It uses several published formulae, nothing original.

Title: Re: Is the problem of diffraction over-rated?
Post by: Jack Hogan on March 25, 2013, 09:22:03 am
One of my favorite images is this one, which is worth a look whenever one is getting too obsessive about detail. It predates DSLR's, I believe.

Hi Ted,

Yes, quite interesting.  And eye-opening.
Title: Re: Is the problem of diffraction over-rated?
Post by: Jack Hogan on March 25, 2013, 09:36:13 am
The important notion is that InFocus combines 2 operations, first a generic (or deblur) deconvolution, and second an optional sharpening operation. One should probably not try and solve the entire blur issue with only a single deconvolution (because you have no influence on the 'strength' of the effect), they are supposed to work in tandem (although I'd rather prefer more control over the deconvolution process). The optimal deconvolution radius setting seems to correspond reasonably well with the Sigma radius that my tools determines, maybe dialing in a tad smaller radius can help avoid the generation of excessive artifacts. It is not obvious which deconvolution method (generic/deblur) would be best. Deblur seems a bit more aggressive, it seems to be more than just a different PSF shape. Many unsatisfied reactions are caused by using too large a deconvolution radius. Assuming the two operations are executed in sequence, the sharpening radius to use should then be smaller than the deconvolution radius, unless one tries to achieve creative sharpening.

Hi Bart,

Thanks, interesting that.

I generally try 'Estimate' with a 2 radius, 0.3 softness and 0.2 artifact suppression first.  It typically works pretty well on my landscapes, but if I think it is being too aggressive I will switch to Generic and play with the radius, leaving artifact suppression at 0.2.  Radius usually ends up between 0.9 and 1.3.  I virtually never use the built-in optional 'classic' sharpener, preferring to do that elsewhere.   Maybe I should give it a second chance.  Does it work as well/better than what you can typically achieve later in the workflow?

I think I like InFocus a little better than the FM trial I've been playing with.
Title: Re: Is the problem of diffraction over-rated?
Post by: Bart_van_der_Wolf on March 25, 2013, 10:17:45 am
I generally try 'Estimate' with a 2 radius, 0.3 softness and 0.2 artifact suppression first.  It typically works pretty well on my landscapes, but if I think it is being too aggressive I will switch to Generic and play with the radius, leaving artifact suppression at 0.2.  Radius usually ends up between 0.9 and 1.3.

Hi Jack,

Yes, 'Estimate' sometimes works fine, sometimes not. It depends on the subject and the location of the focus plane. It's too bad one cannot import a custom made PSF kernel. Maybe something for the long overdue updated version.

Quote
I virtually never use the built-in optional 'classic' sharpener, preferring to do that elsewhere.   Maybe I should give it a second chance.  Does it work as well/better than what you can typically achieve later in the workflow?

No, not necessarily. It's just that one shouldn't try to do everything with only a single deconvolution pass. For very good control over subsequent 'Creative sharpening', their 'Topaz Labs Detail' plugin is very useful. It's like adding Clarity on steroids, to be used with restraint but very powerful.

Cheers,
Bart
Title: Re: Is the problem of diffraction over-rated?
Post by: Jack Hogan on March 25, 2013, 10:28:34 am
For critical work, I suggest that an aperture of 5.6 is likely optimal with the D800e. What do you think?

Hi Bill,

I was thinking about your comment above.  I hope I am not saying something too obvious but total system sharpness is fully dependent on the contributions of individual system components.  In other words total system MTF is the product of individual component MTFs (or the convolution of individual PSFs).  The way you maximize system MTF is by maximizing each component MTF - independently of the others.

All this to say that the best aperture with a given lens to maximize MTF is that best aperture period, independently of whether you are shooting with a D800e or a D40.  The difference is imo that with a 'e' you will notice much sooner when you are not in the sweet spot.

Cheers,
Jack
Title: Re: Is the problem of diffraction over-rated?
Post by: Jack Hogan on March 25, 2013, 10:34:55 am
No, not necessarily. It's just that one shouldn't try to do everything with only a single deconvolution pass. For very good control over subsequent 'Creative sharpening', their 'Topaz Labs Detail' plugin is very useful. It's like adding Clarity on steroids, to be used with restraint but very powerful.

Agreed, Bart.  Detail reminds me of my first motorcycle, a Suzuki GS650: way too powerful for my own good.  I stopped using it shortly after purchase as well, as a result of a couple of near fatal crash-and-burns.  Great fun though  :)