Interesting article on sample variations

[Ray]

So what you're telling Canon is that you prefer features despite the poorer QC.  And they're listening.  Tell Canon with your purchases that you value QC over features and I suspect that's what they'll provide.  Or provide both - but be prepared to pay more.

This is sounding like an opportunity!  How many would be willing to pay for a cherry-picking service?  Suppose an enterprising individual were to buy a dozen samples of a lens, test them all, and return all but the best.  Would you be willing to pay this person to do the QC the camera makers have skimped on?

Exactly! A lower quality lens with the right features is often preferrable to a first class lens without the features or with the wrong features.

A medium quality 150mm lens in a zoom will likely provide more satisfying results than the best 100mm prime ever made, if the FoV of the composition is that of a 150mm lens, making it necessary to crop the 100mm shot.

Likewise, a hand-held shot at the long end of a 100-400 medium quality zoom with the feature of IS, using say 1/200th sec exposure at ISO 200, will likely provide better results than a Zeiss 400mm prime without IS, at the same shutter speed and ISO.

Features can be important, particularly IS and accurate autofocus. I think Canon have demonstrated they are capable of manufacturing a lens with the IS feature without compromising the optical MTF response. For example, the 70-200/F4 IS is at least as good, and slightly better I believe, than the highly acclaimed previous non-IS version. Also, the recent 100/2.8 IS seems to be optically as excellent as the previous non-IS version, if not better.

I don't think an ad hoc cherry-picking service as a business would be allowed. There would be legal ramifications. What would happen to the returned lenses? It would be fraudulent to sell them without mentioning they are the rejects of cherry-picking.

Of course, in an informal manner, this is effectively what has been occurring for years. I once went to a lot of trouble comparing the Canon 400/5.6 prime with my 100-400 IS zoom which is noticeably soft at F5.6. I understood the 400 prime was supposed to be at least as good at F5.6 as the 100-400 at F8, its sharpest aperture.

Although the prime lacked IS, being sharp at full aperture would give it an advantage in certain circumstances when the subject is moving and a fast shutter speed is required irrespective of an IS feature.

I was surprised to find that the copy of the 400 prime I'd bought was not even as good as my 100-400 zoom. It was the last one in the warehouse, apparently. After my testing, it was returned to the Canon agent (by the retailer) for adjustment and tweaking, and I then repeated the whole series of tests on the same lens. I detected a very marginal improvement as a result of the adjustments, but not enough to make it sharper than the zoom. I got a refund.

This seems clearly a case of a Grade C zoom being compared with a Grade E prime. Whoever subsequently bought and accepted that lens that I'd rejected, would probably not be aware it was a Grade E within the QC manufacturing parameters for that particular model.

I'm advocating a system of honesty and transparency whereby the entire output of lenses is subjected to a rigorous MTF testing procedure along the lines of the old Photodo tests.

The additional cost might be 10% of the wholesale price of each lens. Such additional cost would be borne by the purchasers of Grade A and Grade B lenses plus a small discount that rightfully should be applied to the Grade D and Grade E lenses. Grade C lenses would cost the same as usual.

[telyt]

Exactly! A lower quality lens with the right features is often preferrable to a first class lens without the features or with the wrong features.

Perhaps but not if they're not working well- the AF needs micro-adjustment, the lens's performance is uneven in the corners, has to be stopped down for good performance, is weak at some focal lengths & sucks dust inside, the camera body's AF system is erratic and calibrated for no better than +- 1 DOF, the mirror box isn't aligned accurately, the IS goes fubar.  Without QC these features make the equipment more difficult to use.

Where did you get the 10% added cost figure for the QC'd lenses?

[Ray]

Perhaps but not if they're not working well- the AF needs micro-adjustment, the lens's performance is uneven in the corners, has to be stopped down for good performance, is weak at some focal lengths & sucks dust inside, the camera body's AF system is erratic and calibrated for no better than +- 1 DOF, the mirror box isn't aligned accurately, the IS goes fubar.  Without QC these features make the equipment more difficult to use.

Where did you get the 10% added cost figure for the QC'd lenses?

I'm addressing only lens variability here. The new features of AF micro-adjustment, and LiveView for extremely accurate manual focussing on the latest Canon DSLRs, are very welcome features.

The 10% figure is a guess based upon the use of properly designed, automated MTF testing eqipment and the use of low-cost Chinese labour. Such testing might be possible at an additional cost of less than 10%.

Here's an example of the sort of equipment I have in mind, although this equipment and software is designed to test P&S camera systems with fixed lenses.

http://kreysite.com/papers/SPIE2003.pdf

The chief point here is that a range of MTF measurements can be taken in just 6 seconds. To quote:

The draw back of MTF testing is that the proper measurement of the lens MTF is quite cumbersome and time consuming.

In the current investigation we designed, produced and tested a new semi-automated MTF set up that is able to measure the polychromatic lens system MTF at 6 or more field points at best focus in less than 6 seconds.

The computed MTF is a real diffraction MTF derived from a line spread function (not merely a contrast measurement). This enables lens manufactures to perform 100% MTF testing even in high volume applications.

[Ray]

.... the lens's performance is uneven in the corners, has to be stopped down for good performance, is weak at some focal lengths & sucks dust inside ......

Doug,
Are you referring to the Canon 100-400/5.6 IS ?  

I've looked at your site. You don't need further praise from me, but I sense you must be very concerned about lens performance and must appreciate the benefits of IS or VR. Animals and birds can frequently sit very still for their portrait, but not always for a sufficiently long time to enable one to set up tripod. In such circumstances, with a long telephoto, one wants to use the lowest ISO possible, consistent with a sufficiently fast shutter speed to freeze camera shake.

Most good lenses are sharpest at F5.6. The Canon 100-400 does not appear to be. My copy is sharpest at F8 and virtually as sharp at F11 as at F8. A 100-400 which is sharpest at F5.6, would be worth paying more for. Do they exist? Maybe they do.

A set of MTF charts would be the most certain way of informing the buyer.
 

[telyt]

The new features of AF micro-adjustment... are very welcome features.

Micro-adjustment is an admission of variability and poor QC.  Camera makers have convinced consumers that this is a 'feature' not a kludge.  The Leica S2 doesn't have this 'feature' because the QC work is done at the factory instead of shifting the cost to the purchaser.  The S2 is expensive - part of the high cost is QC.

You want lots of convenience features at low out-of-pocket cost?  Who wouldn't - but the purchaser is going to do the QC.  How much is your time worth?  I'm not trying to say this is right or wrong, good or bad... just please recognize that user QC is one of the drawbacks of low initial cost.

... I sense you must be very concerned about lens performance and must appreciate the benefits of IS or VR. Animals and birds can frequently sit very still for their portrait, but not always for a sufficiently long time to enable one to set up tripod.

I don't use a tripod.  I use a shoulder stock with monopod.  The weight is out of my hands, the stability is good enough that subject motion limits usably slow shutter speeds, mobility is nearly as good as a hand-held camera, and I avoid the flare and color quality loss of the added IS or VR glass.

Most good lenses are sharpest at F5.6.

That's why I avoid most lenses.  I want lenses that are sharp at full aperture.  In my experience with the Leica 280mm f/4 APO the aperture is used for DOF only, not to make the lens perform well.  Same applies with the Leica 1.4x APO-Extender on the 280.

[vandevanterSH]

I don't use a tripod. I use a shoulder stock with monopod.
**********
I stumbled on that combination by trial and error...A nice combination of stability with ease of use and mobility.

Steve

[JohnKoerner]

The typical purchaser isn't a 'thinking person'.  He (usually) doesn't think to ask if these features are useable, reliable, or accurate.  More useful features for the money spent, cool.  More crap-shoot features?  Why bother.

This may be true, but I would say that most people looking to buy mid-level to very nice equipment "think" enough to know the difference between what they need versus what they "want" but don't really need. I can't speak for others, but I myself do want (and look for) are the most useable features that I actually need, for the lowest price. I look first through competing companies, and then for lowest price/best rep amongst vendors. But, here again, I am not sure what this has to do with the subject, namely quality control in companies as well as the reality of manufacture-variance within product lines.

Spec sheets don't show manufacturing tolerances, MTBF, or in most cases, meaningful environmental tolerances.  45-point AF is pointless if it's not accurate, 'weather resistance' that fails in ordinary use is likewise pointless.  Yet there are consumers who live for spec sheet 'performance' and dump perfectly good lower-spec equipment for the latest stuff.  It's very common on this forum, and most others.

I personally do not go on just company-generated spec sheets, but wait until I read a few hundred examples of actual buyer feedback, as well as several examples of "web review" feedback and testing from qualified individuals. For example, I have been watching and waiting for several examples of feed back on both the 7D as well as the 100mm "L" macro lens before purchasing. I did not just run right out and get these products, and I still have not, even though I am very interested. After checking and re-checking the buyer/user feedback, the almost universal consensus is there are both truly superior upgrades, and not "the same thing with a new label."

It's great for me when I buy lightly-used, user-QC'd 'obsolete' equipment, great for the camera makers because these consumers don't think to ask if these features are any good, more of a crap shoot for someone who needs to use a reliable, accurate, durable camera.  More features without good QC invites Murphy's Law to play havoc with your 'features-per-dollar' equipment.

I agree 100% that this is a sensible way to upgrade, and as a matter of fact this is how I am considering upgrading into the aforementioned two relatively new Canon products, by obtaining them either used or after they have gathered a little bit of dust and are no longer a novelty. Already the 7D is down to $1599 and the 100mm L is down to $849, and I am sure they will be even less in another year. And I also agree that buying a used camera, tested and true from an honest person is, without question, the best way to get a proven-good product inexpensively.

Jack




.

[Ray]

Micro-adjustment is an admission of variability and poor QC.  Camera makers have convinced consumers that this is a 'feature' not a kludge.

Good! I'm all in favour of honesty. An honest admission and a brilliant, cost-effective solution. What more do you want! Furthermore, the micro-adjustment is not just a feature which allows a degree of user quality control as a kludge for camera body misalignment, but also compensates for lens misalignment.

If it works, it works. No point in despising it because it's cheap. This is progress.

....but the purchaser is going to do the QC.  How much is your time worth?

It's worth enough to favour an efficient MTF testing procedure to grade mass-produced lenses so I get a choice of paying in accordance with a well-defined quality instead of buying a lottery ticket.

[telyt]

Good! I'm all in favour of honesty. An honest admission and a brilliant, cost-effective solution. What more do you want! Furthermore, the micro-adjustment is not just a feature which allows a degree of user quality control as a kludge for camera body misalignment, but also compensates for lens misalignment.

If it works, it works. No point in despising it because it's cheap. This is progress.

Sorry, it's still a dumb kludge.  The real problem is that CaNikon have conditioned us to expect this shoddy level of QC.

[vandevanterSH]

The real problem is that CaNikon have conditioned us to expect this shoddy level of QC.
*********
As demonstrated by the general end user acceptance of the "problems" associated with the Nikkor 70-200 VR II.  "Pitted metal casting and silver particulates inside of the lens, doesn't affect function or image quality, why are you complaining???"  One good response, basically " it's your fault for looking inside the lens".  

Steve

[Playdo]

Very nice thread. I'm adamant that companies such as Canon would use this lack of QC to profit from lens calibration/repair, thus the low warranty. It amazes me how many posts I read where people send their equipment in at a high cost for calibration, yet it seems they find it perfectly acceptable. As long as there is financial gain from it, or preventative laws, nothing will change.

I believe folks were already comparing serial numbers on the "Antarctica" thread, which empowers the consumer with the ability to "know where they're at" on a much broader scale than ever would be possible only knowing their own situation.

Yes, I'd have thought that most returned goods would end up back on the shelf.  I was thinking a while back of having a site where anyone who has returned equipment could place the serial number and a description of the problem. Users could easily locate and check an item before or immediately after buying. Any thoughts?

[vandevanterSH]

Yes, I'd have thought that most returned goods would end up back on the shelf.
**********
I am sure that my returned 70-200 was resold for full price.  Nikon said that pitted metal and silver particulates didn't affect image quality and thus met standards for that lens.

Steve

[Ray]

Sorry, it's still a dumb kludge.  The real problem is that CaNikon have conditioned us to expect this shoddy level of QC.

Can you elaborate on that? What do you mean by a dumb kludge? Have you had a bad experience trying to use the micro-adjustment with lenses that don't autofocus accurately? Have you found this feature to be unsatisfactory?

If you want military grade products then you have to pay more than military grade prices due to the low demand and low quantities produced, much like MFDB systems. At least there's a large, well-defined market for military products, which tends to keep the price lower than it otherwise might be.

Perhaps you mean a kludge in the sense that some folks think that using a monopod is a kludge compared with a full tripod.

Your ideas would seem to lead to a situation whereby lots of consumers would not be able to afford cameras because the manufacturers insist on producing nothing which is not of military grade quality.

[stever]

i think the micro-adjust is a kludge, but not dumb.  it's a reasonable solution to make legacy lenses work with high-resolution SLRs -- and it works.

but Canon (and i believe Nikon) provide no help in it's practical application and left this to 3rd parties - fortunately we have Lensalign - but the camera manufacturers should provide the tools to make the bodies work with the lenses (and i'd much rather have them do that than provide worthless software disks with the cameras)

going forward, however, i don't see why bodies shouldn't be tested and firmwared with focus adjust parameters and likewise lenses - at least the Canon L lenses

i have no objection to software solutions to mechanical tolerance issues that are impossible or uneconomic to solve otherwise, they just need to be implemented effectively


this, however, is aside from the need to keep manufacturing under control, and i don't see any good excuses for Japanese companies who were early adopters of modern quality control practices and maintain manufacturing in Japan for reasons of product control  - i would not be surprised by statistical differences in lenses to be cherry picked, but the batch problems are not excusible

[telyt]

Can you elaborate on that? What do you mean by a dumb kludge? Have you had a bad experience trying to use the micro-adjustment with lenses that don't autofocus accurately? Have you found this feature to be unsatisfactory?

If you want military grade products then you have to pay more than military grade prices due to the low demand and low quantities produced, much like MFDB systems. At least there's a large, well-defined market for military products, which tends to keep the price lower than it otherwise might be.

Perhaps you mean a kludge in the sense that some folks think that using a monopod is a kludge compared with a full tripod.

Your ideas would seem to lead to a situation whereby lots of consumers would not be able to afford cameras because the manufacturers insist on producing nothing which is not of military grade quality.

You seem quite defensive of this technology.  With automated procedures and proper tools, do you think that correctly adjusting a lens at the factory to focus accurately would raise the retail cost to military-grade levels?  How did you arrive at this estimate?

[Ray]

You seem quite defensive of this technology.  With automated procedures and proper tools, do you think that correctly adjusting a lens at the factory to focus accurately would raise the retail cost to military-grade levels?  How did you arrive at this estimate?

Correctly adjusting a lens after fabrication to make it autofocus accurately may not always be possible. The lens may have to be junked, thus increasing manufacturing costs. The micro-adjustment feature is not simply a solution to compensate for a kludge of a camera body, but also a kludge of a lens which might otherwise never autofocus properly.

I confess I don't know this for certain, but I have experienced two situations of a lens that didn't auto-focus accurately on camera bodies that were quite satisfactory with all my other lenses.

I also once sent off a lens, still under warranty, to Canon for adjustment because it didn't autofocus properly. It still didn't autofocus accurately after adjustment, hence my purchase of the 50D with micro-adjustment.

By definition, military grade components are of the highest standard. Each component is manufactured using the highest QC procedures available.

Everything is a kludge in relation to the highest standards. Remember the Einstein quote about the Heisenberg theory of Quantum Mechanics? 'God does not play dice.' He was wrong on this point, wasn't he?  

[ErikKaffehr]

Hi,

I would recommend these two articles

http://www.josephholmes.com/news-sharpmediumformat.html

http://www.josephholmes.com/news-medformatprecision.html

It's obvious that modern photographic equipment can be very demanding. We perhaps need like 10 my (microns) tolerance between AF-sensor and sensor. The optical path involves two moving mirrors. There is also wear, thermal expansion and handling related issues. From the articles above it is quite obvious that it cannot be taken for granted that expensive pro equipment is properly in adjustment and it's also obvious that lens variations are large. The author also indicates that he has not observed similar problems with his Canons.

I'm a strong believer in mirror less designs, using live view and contrast detecting auto focus. The technology is not here yet. Contrast based AF is still slow, AFAIK, and I don't see live view working in darkness.

The mirror less design eliminates AF-problems, as the sensor itself is used to focus. The sensor lens still need to be aligned as the image plane and the sensor needs to be within a few microns.

Check also:
http://www.diglloyd.com/articles/Focus/focus-accuracy.html
http://diglloyd.com/articles/LensAndCamera...andNewBlur.html

My view is that pixel peeping with digital technology is easy and sensor resolution is going up, increasing the demands on the lenses and the focusing system.

Best regards
Erik

Can you elaborate on that? What do you mean by a dumb kludge? Have you had a bad experience trying to use the micro-adjustment with lenses that don't autofocus accurately? Have you found this feature to be unsatisfactory?

If you want military grade products then you have to pay more than military grade prices due to the low demand and low quantities produced, much like MFDB systems. At least there's a large, well-defined market for military products, which tends to keep the price lower than it otherwise might be.

Perhaps you mean a kludge in the sense that some folks think that using a monopod is a kludge compared with a full tripod.

Your ideas would seem to lead to a situation whereby lots of consumers would not be able to afford cameras because the manufacturers insist on producing nothing which is not of military grade quality.

[telyt]

Correctly adjusting a lens after fabrication to make it autofocus accurately may not always be possible.

Given the sorry state of most AF cameras' viewfinders for manual focus, how much would you be willing to pay for a lens that can't AF accurately?

[Ray]

Given the sorry state of most AF cameras' viewfinders for manual focus, how much would you be willing to pay for a lens that can't AF accurately?

Doug,
Canon have already addressed this difficulty you refer to. My 50D has a 920,000 pixel LiveView screen. My 40D also has a Liveview screen, but only 230,000 pixels.

When recently comparing resolution from the 40D and 50D (using the same lens), photographing a mounted banknote from a distance of a couple of metres, I was surprised to find that the higher resolution LCD screen of the 50D did allow for more accurate manual focussing. With the 40D, I was never quite sure if focussing was really spot on. Good enough for most practical applications, but not necessarily good enough for critical comparisons at the extreme pixel-peeping level.

So I resorted to the trick of using the appearance of moire effects on the LCD screen as an indication of absolutely accurate focus. Even when looking through an optical viewfinder, one can notice moire (or chromatic aberration, or aliasing) whenever a lens is perfectly focussed on a line chart (such as a Norman Koren Resolution Test Chart) at a particular distance from the chart and at a particular line spacing.

The fact is, autofocussing is never perfectly accurate (except perhaps by accident), no matter how expensive the lens. It's always a matter of the degree of accuracy that is acceptable for the application and/or the standards required. The more expensive lens (one would hope) has more accurate autofocussing.

As a person who is a bit obsessed with issues of resolution, I would not be interested in buying a lens which could not autofocus properly, according to my own standards. But many 'more normal' people might be quite satisfied with a lens that I would reject.

It's interesting that the P&S world is leading the way in this regard. There are a few P&S cameras, Ricoh in particular, that do autofocussing bracketing. My initial interest was perhaps that such bracketed images could be used to extend DoF using a program like Helicon Focus. However, it seems that the autofocus bracketing in such cameras is not designed for this purpose. The differences in focal plane are not great enough. It's purpose is to provide a choice from small differences in focal points in macro shots where DoF is extremely shallow, even with a P&S. Even in 'wide' mode, the focal points will not range from the insect's tail to the insect's head, but rather from the insect's nose to it's eye.

I would hope that autofussing bracketing will become a feature in future DSLRs.

[Rob C]

I would hope that autofussing bracketing will become a feature in future DSLRs.

I despair of me Ray; if such did become the norm it would be but one more device I would have to learn to override in my attempts at getting back to a digital F3.

Rob C