That famous AA filter...

[Slough]

Given how tiny artifacts are in the new 24-25MP cameras

Surely artifacts such as moiré will still be present, but they will appear for finer patterns in the subject matter. But if you enlarge the image more (which is surely the reason for such high pixel counts), then you will see them just as badly.

On a side issue, people are claiming the Nikon D3x has very sharp images at the 100% level. Presumably that means a weak AA filter. Presumably the argument is that only pros and knowlegeable amateurs will make big enlargements and these people will know how to handle artifacts in software. (That's a guess.)

[ErikKaffehr]

Hi,

Graeme suggest that the best solution is oversampling, that is more pixels. The industry is clearly moving in that direction possible for the wrong reasons, namely that pixels sell. Anyway in my eyes there seems to be an agreement on this forum that aliasing phenomena are bad and cannot really handled in software. There still seems to be a good reason for using those filters.

To my understanding, based on this discussion, color moiré can be reduced in postprocessing whereas the removal of BW moiré and other aliasing artifacts are not really feasible in software. On the "positive" side aliasing effect can give a spurious resolution giving the impression of more detail than actually resolved.

The discussion did not really answer my other question, why MFDBs don't have a AA-filter, with the sole exception of the the Mamya ZD which had it as an option. A suggestion was that the reason was a "raw" workflow is the rule on MFDBs while on DSLRs JPEG is at least an option.

My personal view would be that:

- The AA filters are there for a good reason
- AA-filters loose some acutance (by smearing an image point over multiple pixels) but this can partly be recovered using sharpening
- The softening by the AA-filter is just an addition to other softening effects, like residual aberrations in the lens and focusing errors

I have a small comment regarding resolution. If we have a very good lens which resolves very much higher than the sensor the resolution is dominated by the sensor. A well designed AA-filter would not reduce resolution, as it is given by the pixel pitch (or the Nyquist limit), but it would reduce contrast by "spilling" a controlled amount of light into the surrounding pixels. So contrast (or MTF) would be reduced and not resolution. If resolution is not dominated by the sensor the situation is much more complicated. This explanation may be simplistic and possibly not even correct.

Best regards
Erik

Do you have a reference for that?  I'd be interested.



There are two uses of the term "aliasing" in common usage.  One is the stairstepping of diagonal lines, also called "jaggies".  The other, which is what I believe Graeme is referring to, is the shifting of spatial frequency of a signal by a multiple of the sampling frequency k_max, due to the fact that the sampling cannot distinguish a signal of frequency k and k-k_max when k>k_max.  For some pretty pictures, see

http://en.wikipedia.org/wiki/Aliasing

I would think the main reason for the AA filter is to combat moire and color aliasing.  Thought the latter can be mitigated by post-processing techniques, the former is quite nasty and nearly impossible to to back out, since the information that would allow one to distinguish that an oscillation of luminance at frequency k-k_max really should have been frequency k is irretrievably lost.

On the other hand, there is substantial room for improvement in demosaicing algorithms.  Many of the ones I see break down strongly on texture data near the Nyquist frequency, and if they don't produce moire as a result, introduce maze artifacts and other structures that are just as bad.  AA filters help here, but I think there is progress to be had on the processing side.

[ErikKaffehr]

Hi,

The D3X and also the 10MP plus APS-C cameras are pretty close to diffraction limitation which start to set in around aperture f/8 some evidence points in the direction that the Canon 50D is limited by diffraction at f/8. Add to this residual aberrations, nonoptimal focusing, camera movement and other perils reducing sharpness and risk for aliasing is significantly reduced. Pixel pitch on the 3DX is about 6 microns, same league as 10 mpixel APS-C sensors.

Best regards
Erik

Surely artifacts such as moiré will still be present, but they will appear for finer patterns in the subject matter. But if you enlarge the image more (which is surely the reason for such high pixel counts), then you will see them just as badly.

On a side issue, people are claiming the Nikon D3x has very sharp images at the 100% level. Presumably that means a weak AA filter. Presumably the argument is that only pros and knowlegeable amateurs will make big enlargements and these people will know how to handle artifacts in software. (That's a guess.)

[Graeme Nattress]

There are two uses of the term "aliasing" in common usage.  One is the stairstepping of diagonal lines, also called "jaggies".  The other, which is what I believe Graeme is referring to, is the shifting of spatial frequency of a signal by a multiple of the sampling frequency k_max, due to the fact that the sampling cannot distinguish a signal of frequency k and k-k_max when k>k_max.  For some pretty pictures, see

http://en.wikipedia.org/wiki/Aliasing

I would think the main reason for the AA filter is to combat moire and color aliasing.  Thought the latter can be mitigated by post-processing techniques, the former is quite nasty and nearly impossible to to back out, since the information that would allow one to distinguish that an oscillation of luminance at frequency k-k_max really should have been frequency k is irretrievably lost.

On the other hand, there is substantial room for improvement in demosaicing algorithms.  Many of the ones I see break down strongly on texture data near the Nyquist frequency, and if they don't produce moire as a result, introduce maze artifacts and other structures that are just as bad.  AA filters help here, but I think there is progress to be had on the processing side.

Jaggies are just another aliasing artifact. Aliasing refers to jaggies, moire (both luma and chroma).

An OLPF is to either avoid, or limit aliasing, and hence the visual artifacts produced from aliasing.

Indeed chroma aliasing in a Bayer pattern colour filter array sensor can be reduced quite well with clever algorithms. It's much harder to do so though if there is also luma aliasing present and if there is no OLPF present.

Yes, often demosaicing algorithms break down with maze artifacts and bad colour artifacts if they're over-simplistic. Often it's the battle to eek out as much resolution as you can from the image that causes this, and is allowed because most natural images don't make such artifacts too visible. An OLPF can make the demosaic algorithm development easier, but does  not eliminate these difficulties.

OLPF has no impact on noise, as noise on a per pixel levels comes from the sensor, which is after the OLPF.

[ejmartin]

Jaggies are just another aliasing artifact. Aliasing refers to jaggies, moire (both luma and chroma).

How are jaggies related to frequency shifts by a multiple of Nyquist?

Indeed chroma aliasing in a Bayer pattern colour filter array sensor can be reduced quite well with clever algorithms.

What sort of algorithms?

[bjanes]

The problem of using deconvolution methods to retrieve information lost by low-pass bi-refringent optical filters has already been tackled, but I don't think it is on the mainstream of demosaicing research.

Do you have a reference for that?  I'd be interested.

Jonathan Wienke, among others, reports using Focus Magic (a deconvoluiton algorithm) for capture sharpening. This can be used as a first step of capture sharpening. Bruce Fraser used an unsharp mask for this purpose, with the radius determined by the megapixel count of the sensor and the amount determined by the strength of the blur filter.

The problem is to determine the Point Spread Function (PSP) to be used for the deconvolution. Jonathan uses an empiric PSP which produces good results for him. I too would be interested in some references in the scientific literature.

Bill

[walter.sk]

Jonathan Wienke, among others, reports using Focus Magic (a deconvoluiton algorithm) for capture sharpening. This can be used as a first step of capture sharpening. Bruce Fraser used an unsharp mask for this purpose, with the radius determined by the megapixel count of the sensor and the amount determined by the strength of the blur filter.

The problem is to determine the Point Spread Function (PSP) to be used for the deconvolution. Jonathan uses an empiric PSP which produces good results for him. I too would be interested in some references in the scientific literature.

Bill

Focus Magic has a mechanism that supposedly measures the blur width within a small rectangle that the user can move over various parts of an image and then pres the "detect" button.  On their website they give several examples of measuring the blur width, preferably of bright highlights that are supposed to be point sources of light.  The assumption is that the blur is uniform in all directions.  Focus Magic also has another routine available to "correct" directional blur from camera movement.  A circle with a rotatable diameter line is set either by measurement or trial and error to be parallel to the direction of the camera movement, and then the radius of the blur is set manually, by eye or after counting the pixels involved in the blur.  I suppose that they could provide references to the appropriate scientific literature.

Focus Magic

[bjanes]

Focus Magic has a mechanism that supposedly measures the blur width within a small rectangle that the user can move over various parts of an image and then pres the "detect" button.  On their website they give several examples of measuring the blur width, preferably of bright highlights that are supposed to be point sources of light.  The assumption is that the blur is uniform in all directions.  Focus Magic also has another routine available to "correct" directional blur from camera movement.  A circle with a rotatable diameter line is set either by measurement or trial and error to be parallel to the direction of the camera movement, and then the radius of the blur is set manually, by eye or after counting the pixels involved in the blur.  I suppose that they could provide references to the appropriate scientific literature.

Focus Magic

Deconvolution methods are widely used in astronomy, and the free ware Iris incorporates some of these methods, as shown in their documentation. In the example, they synthesize the PSP from a mathematical model. I don't know why they didn't determine the PSP from an image of a star as viewed by the telescope. It would be interesting to take an image of a star with ones DSLR and use this for the deconvolution. This would be more precise than using a point source from a specular highlight.

Bill

[joofa]

The problem is to determine the Point Spread Function (PSP) to be used for the deconvolution. Jonathan uses an empiric PSP which produces good results for him. I too would be interested in some references in the scientific literature.

There are tons of references in deconvolution theory (just Google this term), many of  them in channel equalization that is within the domain of communication systems. A straight forward implementation of deconvolution with a known kernel is not difficult in hardware/software as it is just filtering, however, the problem lies in determining the right deconvolution kernel, and that is the more compute intensive task.

Additionally, many algorithms assume Gaussian statistics and mean square error for objective functions for finding the right deconvolution kernel, which are not the best measures all of the times. More complicated algorithms exist in estimation theory literature, especially those using Bayesian statistics, but that is a whole separate field.

[Bob Smith]

- The AA filters are there for a good reason
- AA-filters loose some acutance (by smearing an image point over multiple pixels) but this can partly be recovered using sharpening
- The softening by the AA-filter is just an addition to other softening effects, like residual aberrations in the lens and focusing errors

AA filters do very good things.  The problem is that most subject matter in most images doesn't need them at all (especially as camera resolutions keep increasing) and your images are sharper without them.  The few areas that do need them can look horrible without them.  The trade off is how much time/effort are you willing to devote in software to fixing the problem areas versus accepting the softening of everything that the AA filter provides.  Various sharpening techniques and todays higher res cameras help a lot but you can't get all of what the AA filters suppresses back again.  For a portrait or fashion photographer that is constantly shooting a variety of fabrics, an AA filter is a no brainer.  For landscapes or most product photography I can very happily live without it.

Without an AA filter ALL of your images will be slightly sharper but a few will look horrible.  With an AA filter all of your images will be very slightly softer but none will look horrible.  If you're a marketing manger for a DSLR maker, which do you pick?

If I get a 5dII soon, I'll send my original 5D for a mod to remove the AA filter.

Bob Smith

[ejmartin]

There are tons of references in deconvolution theory (just Google this term), many of  them in channel equalization that is within the domain of communication systems. A straight forward implementation of deconvolution with a known kernel is not difficult in hardware/software as it is just filtering, however, the problem lies in determining the right deconvolution kernel, and that is the more compute intensive task.

Additionally, many algorithms assume Gaussian statistics and mean square error for objective functions for finding the right deconvolution kernel, which are not the best measures all of the times. More complicated algorithms exist in estimation theory literature, especially those using Bayesian statistics, but that is a whole separate field.

Yes, I have done that in the past.  The question I raised was in response to a post that seemed to suggest that there was some work done specifically on the use of deconvolution to back out the effects of the AA filter.  That, rather than some generic work on deconvolution sharpening, is what I was looking for a reference about.

[joofa]

Yes, I have done that in the past.  The question I raised was in response to a post that seemed to suggest that there was some work done specifically on the use of deconvolution to back out the effects of the AA filter.  That, rather than some generic work on deconvolution sharpening, is what I was looking for a reference about.

Hi,

There is some work on image restoration, including some that does not assume linear spatial invariance that is assumed many times in deconvolution, which may be in line of what you are looking for, though I am not entirely sure if that is what you need. Kindly PM me your email and I shall be glad to provide some references. (BTW, in your case I can figure out your email address, however, since I don't want to bombard people with emails without their approval, I ask them for sending a message to show interest.)

AA filters do very good things.  ...  The trade off is how much time/effort are you willing to devote in software to fixing the problem areas versus accepting the softening of everything that the AA filter provides.  ...

You might notice that since the sampling inherent in a digital imager is not point sampling, coarse anti-aliased-type smoothing is already provided by the sensor photosites, regardless of the presence of an AA filter.

[Slough]

I wonder if anyone has implemented an 'electronic' AA filter? Some materials' optical properties can be modified by the application of an electric field. So in principle this would allow the user to turn off the AA filter for 'normal' use, and turn it on when there are regular patterns in the subject.

[Mitchell Baum]

http://www.maxmax.com/hot_rod_visible.htm
You can get a few models also converted to non AA by this company.  Given how tiny artifacts are in the new 24-25MP cameras, I think it might be a good "upgrade" once they start to support these newer models.(wish they'd hurry up!)

Note - the Fuji S5Pro with this modification seems to be a great performer due to the fact that the sensor in it doesn't really need an AA filter since it brackets and blends the shot already.  Oh - speaking of which, using a program like the Zero Noise software  that's mentioned on this site in a couple of threads, you can bracket a shot on a camera without an AA filter and pretty much eliminate all of the ugly problems.

Has anyone here had Maxmax remove the AA filter? How was the result?

Best,

Mitchell

[ejmartin]

I wonder if anyone has implemented an 'electronic' AA filter? Some materials' optical properties can be modified by the application of an electric field. So in principle this would allow the user to turn off the AA filter for 'normal' use, and turn it on when there are regular patterns in the subject.

No I haven't heard of that.  However I have seen a suggestion that cameras with in-body IS might be modified so that a high frequency (at least several kHz) circular motion of small radius is applied.  That would have the effect of blurring the image over a quartet of pixels and would easily be turned off.  I don't know if the actuators for in-body IS can be adapted to apply this frequency motion, however; the motion they are compensating is much lower frequency.

[SeanFS]

[quote name='ErikKaffehr' date='Jan 2 2009, 06:25 AM' post='248745']
Hi,

Graeme suggest that the best solution is oversampling, that is more pixels. The industry is clearly moving in that direction possible for the wrong reasons, namely that pixels sell. Anyway in my eyes there seems to be an agreement on this forum that aliasing phenomena are bad and cannot really handled in software. There still seems to be a good reason for using those filters.

To my understanding, based on this discussion, color moiré can be reduced in postprocessing whereas the removal of BW moiré and other aliasing artifacts are not really feasible in software. On the "positive" side aliasing effect can give a spurious resolution giving the impression of more detail than actually resolved.

The discussion did not really answer my other question, why MFDBs don't have a AA-filter, with the sole exception of the the Mamya ZD which had it as an option. A suggestion was that the reason was a "raw" workflow is the rule on MFDBs while on DSLRs JPEG is at least an option.

My personal view would be that:

- The AA filters are there for a good reason
- AA-filters loose some acutance (by smearing an image point over multiple pixels) but this can partly be recovered using sharpening
- The softening by the AA-filter is just an addition to other softening effects, like residual aberrations in the lens and focusing errors



I remember reading somewhere the reason MFDB's don't have AA filters is they are RAW only devices. It was always anticipated they would have software to deal with any artifacts in Raw , and to a large extent this is true, and so it should be given their high prices. The DCS760 I have in the cupboard was built along the same principles but the software never lived up to the promises despite many upgrades. In contrast , the noise filter I have in Flexcolour does a superb job of controlling minor artifacts. Moire happens but is nowhere near the problem it was with the Kodak
The lack of AA filter does give an edge in detail and sharpness and probably means one less item that affects colour on the chip
Part of the reason ( in the same article)there has to be an AA filter in DSLR is they are used to shoot jpegs directly rather than RAW - no professionals don't want to deal with RAW files so much and so it is there to appease the mass market as much as anything .

[Daniel Browning]

The discussion did not really answer my other question, why MFDBs don't have a AA-filter...

I don't know, but I'll hazard a guess or two. First is momentum. Whatever reason there was in the beginning, changing it will upset some of their customers. Some of their customers know what an OLPF is and will be displeased with the addition of one. Other customers don't know what it is, but have accustomed to the microcontrast and aliasing, so would also be displeased. Some will appreciate the addition of an OLPF, but I think the former outweigh the latter by a wide margin, especially in that demographic. In any case, with their shrinking market they might be afraid to introduce changes.

Second: cost. Good OLPF are lab-grown, ground, and polished Lithium Niobate crystal. My understanding is that it is very expensive at low production volumes, and the cost scales exponentially with area. Canon/Nikon ship millions of units a year, so their economies of scale take care of the OLPF, not to mention the advantage of a much smaller area. MFDB are down to just 6,000 per year, and they buy off-the-shelf sensors that were designed for the aerospace industry. Those factors might combine so that the cost is nontrivial, even for something in the tens of thousands.

Those are my guesses.

[Wayne Fox]

I think the decision regarding DSLRs is simply marketing.  Prime example was Kodak's 14n.  All previous Kodak DSLRs were sold either without AA filters or with AA filters as an option.  Those were radically high priced systems compared to today's DSLRs.  Users were pretty much exclusively serious pros as they are with current medium format backs.  When Kodak brought out the 14n at a much lower price point than previous cameras they aimed at a completely different market but they kept the no AA filter approach.  The camera was blasted in early reviews because of aliasing and moire issues.  Those of us who had used many previous Kodaks knew how to deal with it an loved the camera.  New users hated it and panned it royally in various reviews.  It was the final nail in Kodak's pro camera coffin.  The majority of buyers of most current DSLRs are not pros... and even most of the pros using DSLRs would be put off by having to constantly deal with color moire issues if their routine subject matter was the sort that often caused it (bridal veils?).

The higher the resolution of the camera, the less an AA filter is needed.  My first Kodak, a DCS410 was 1.5 megapixel with no AA filter.  You got color noise on just about anything you pointed it at.  Training on that thing made the sort of noise a 14n produced child's play to deal with.  I still prefer the look of a no AA filter camera.  I can deal with the few instances where it poses a problem.  I'd like to see it offered as an option on more DSLRs but I'm probably an oddity in the market.  They're not designing cameras for me.

When Kodak introduced the 14n, they hyped up the super resolution of the sensor, and stated often that because of the high pixel density it did not need an anti aliasing filter - it wouldn't create moire.  They then changed their tune dramatically since most portrait and wedding photographers were bitten quickly (as were we) because of the moire from various fabrics and introduced moire reduction in their software (which didn't work super well).  We actually purchased over 70 of the cameras to replace our mixture of 520's and 560's, and managed the moire issue for the most part.  

 For landscapes it was ahead of its time in image quality, but it seems they didn't understand that high pixel density made moire more likely. (at least their sales and marketing people didn't get it).

From this discussion it sounds like the only way to not have this as a problem is a sensor than can resolve greater than any lens you can put in front of it?

I've heard of removing the AA filter from a 5D with reportedly great results for landscape ... would this work with a 21 or greater mp DSLR?

[JeffKohn]

I've heard of removing the AA filter from a 5D with reportedly great results for landscape ... would this work with a 21 or greater mp DSLR?

MaxMax.com offers the AA filter removal for a variety of cameras (more Nikon than Canon though, including D3 and D300). They don't have any of the 21+ mp models on the supported list yet. I think it usually takes some brave soul to offer their camera up as the guinea pig first so they can try it. Given that the D300 is on the supported list and the D3x sensor seems to have a lot in common with the D300 sensor I wouldn't be surprised if they offer D3x conversions at some point in the future.

[Bob Smith]

From this discussion it sounds like the only way to not have this as a problem is a sensor than can resolve greater than any lens you can put in front of it?

I believe that's correct... and why AA filters aren't needed on the cheap point & shoots that have relatively high resolutions with less than great optics.  For this same reason I don't see it as a problem that DSLR resolutions are getting to the point that they out resolve some lenses.  That's often put forth as an idea why we don't need that resolution.  I think just the opposite.  Resolution higher than the lens can resolve should remove the whole AA issue and is a good thing.

And yes I think the Kodak marketing folks massively underestimated the problems the no AA filter 14n would cause wedding shooters... one of their primary target markets.  The higher res did produce less instances of moire than previous no AA models (I owned 410, 330, 760 and 720x as well) but that was zero consolation to the new-to-digital wedding photographer that was suddenly seeing this new effect on piles of bridal veils.

Bob Smith