Anon and Mark,
Thanks for playing the role of Devil's Advocate. A good idea always needs testing . Let me now dispel some of your negativity (if not completely annihilate it ).
Has anyone here ever made an MTF curve for a lens? Do you know how long it takes? (Hint: 4 - 5 hours.) Do you know how much those machines cost? Then to place every lens on the machine and generate curves would be such an investment to the manufacturer that the cost of the lenses would shoot up. I seriously doubt anyone would want to pay for it.
Photodo is the only company I know of that has produced reliable MTF data for a variety of lenses. Their numerical rating for each lens may be a bit suspect because it's weighted. However, their graphs speak for themselves.
As to the cost in Sweden, I've come across figures of $700 per lens. It seems a believable figure and it's quite extrordinary that Photodo should provide this valuable resource completely free.
A $700 on-cost would be of course a major obstacle for any manufacturer trying to compete in a global market. However, Photodo is a small office testing relatively few lenses. MTF testing on a massive scale in an industrial context would involve industrial robots and streamlined processes. Add to that the fact that wages in China are considerably lower than in Sweden, you're probably going to come up with an additional cost of $50 per lens, a cost which could be borne by the premiums on Grade A and B lenses.
Then there is another headache for the manufacturer. Customers comparing their MTF results and demanding their unit match the best sample curve they can find. Not only would this make your lenses even more expensive, but it is also silly as the MTF curve can detect variations that cannot be seen in the image. Amateur MTF "experts" are not worth the manufacturer's time.
This is a complete red herring. In the early 1980's, I bought a pair of Celestion SL600 loudspeakers. They shipped with individual frequency response charts. I don't know for sure if this was a scam and the charts just theoretical, based on the design rather than the actual speaker, but the chart of the left speaker was in fact different from that of the right speaker, indicating they were real tests.
Did I buy myself a sound level meter and build myself an anechoic chamber so I could test whether the left chart was really different from the right chart? Of course I didn't. I've got better things to do with my time. And so have most people who buy lenses.
Providing individual MTF charts for lenses is really about transparency. Joe Bloggs is not likely to have the means or the inclination to question the veracity of the MTF charts,
but reviewers like Photodo, or even Popular Photography, do have the means. The current situation is that only Zeiss and Leica provide MTF charts for real lenses, on their websites. Whether or not individual Leica or Zeiss lenses have shipped with an MTF chart specific to the that lens is not clear to me. However, it does seem clear that Canon's published MTF charts for all their lenses listed in their Lens Work books, and on the USA website, are all theoretical.
Even if the cost of producing a set of real MTF charts for a particular lens is an astronomical $700 in Japan, one can't help wondering why Canon do not give us a set of MTF charts for even a 'typical' real lens.
The current system is good. If the average quality of manufactured lenses where really as random as this thread suggests, then there would be a lot more complaints than you see here. You have a better chance to get a good lens than you do a bad one.
No! Another red herring! There are many lenses out there that have sub-optimal performance but the owners are just not aware of it. It's only the few 'fanatics' on forums such as this who are aware that a problem exists. If you haven't got something else to campare with, or are not even inclined to do such a comparison, you're going to be blissfully ignorant.
If Mark's first copy of the 24-105 was the one without the flare problem, he wouldn't be aware that another copy
with the flare problem was actually sharper.