Accuracy of lens reviews

[Mark D Segal]

Bernard,

My own experience with Canon lenses indicates that there can be more significant quality variance between different samples of the same lens than one would ideally like to see. I bought a copy of the original series of the 24~105L which was so tack sharp that I wrote an article about it, comparing it with the 28-135 non-L for all those who may have been thinking of an up-grade. Michael liked my work on that comparison and published it on the website. The L of course is generally better. But then Canon did a recall of the original series 24~105 (because they said it produced more flare than it should - something I never noticed, frankly) and I was less impressed with the sharpness of the replacement they gave me, so I dashed back to Canon's service centre and retrieved my so-called "defective" lens before it went to lens heaven somewhere back in Japan. This is what sensitized me to the issue of what a quality standard should mean. I'm still absolutely thrilled with the quality of detail my original 24~105 L is giving me.

Mark

[Peter Jon White]

Reichman's tests are a great source of humor, but certainly not legal action. When I read his declaration that the 16-35L is more prone to flare than the 17-40L, using the images of the sun behind the power lines as proof, I almost fell off my chair. He doesn't seem to understand that the earth is rotating on its axis, and that the sun is blocked in the 17-40L image, but not completely blocked in the 16-35L image, resulting in the flare we see. He even tries to deflect criticism by stating that he hasn't moved the camera!

Well, given the inexorable rotation of the earth, and the impossibility of instantaneous lens changes, he of course would have to move the camera in order to maintain the same orientation of camera, power line tower, and sun between the two images.

So, OK Michael, the 16-35L is more prone to flare than the 17-40L. Sure, I believe you. ;-)

[michael]

Well, at least you aren't  overtly insulting here, the way you were when you wrote exactly the same observations on another forum recently.

Here's something to consider, along with your observation about my disregard for the earth's rotation – a rather thin retort, if you ask me.

Maybe I based my overall judgement on more than a single observation, and that the examples used were simply ones available for convenience sake. Has this occured to you?

Seems like a shallow argument to hang your assertion on. Me thinks more rude jibe than substantiating evidence.

Let me ask you this. Have you compared these lenses yourself? Have you evidence one way or the other, or are you simply basing your comments on other people's observations? If so, what experimental data are those based on?

If you have done such tests yourself, why not share the results with us to back up your counterclaim, rather than running around the net being a boor.

Michael

[jcarlin]

Peter,
    The sun moves at a rate of ~0.25 degrees/minute.  I have a hard time believing that it takes Michael more than a minute to change lenses, but lets assume that it takes him 2, well then those shots are off by no more than 0.5 degrees, say 0.6 at the most.  I have a hard time believing that this difference was that important given that both shots in question where shot more or less directly into the sun.  More to the point the advantage could have gone either way.  For the flare comparison later in the article.

In this instance the 17-40mm shows less internal reflections, but also lower contrast. It's possible that a minute change in the camera's position would change this, but I did enough similar comparisons to show that this is typically the case.

If you think Michaels review are so funny why don't you read them all the way through.  If you don't think they're informative then I can highly recommend Ken Rockwell's site

www.kenrockwell.com

Ken even occasionally uses the equipment he reviews.

[michael]

J,

Ahh, you spoiled all the fun.  

I was all ready with the math, and also reference to the EXIF data from the camera showing that the time between frames was less than 1 minute, and how with a lens that wide the apparent motion of the sun was so small as to totally negate his obnoxious comment.

My eclipse expedition to Romania in 1999 and to Death Valley in 2003 will show that I have just a slight familarity with the motion of the sun (and moon), and am well aware of the effect of this on photography.

Rats. I was really looking forward to having some fun.  

Michael

[Peter Jon White]

What nonsense. The sun can be completely blocked one moment and then not be completely blocked an instant later. All it takes is one little bit (like those technical terms?) of the sun to no longer be blocked by the tower in order for there to be a dramatic increase in flare in the image. It makes absolutely no difference what the rate of rotation is; the change from completely blocked to partly blocked is virtually instantaneous. All you need to do to understand that is to look at a shadow as it moves. It moves steadily. Once the edge of a shadow crosses a certain point, you've got flare folks!

This is just one more case (I can't mention the other or Reichmann will banish me) of Reichmann making absurd claims and then using his "authority" to back up the claim. He says there are other reasons for caiming the 16-35L has more flare but then he doesn't produce them, we just need to take his word for it. He's the "authority", after all. And the photos he does produce to prove the claim do nothing of the sort.

And I love the challenge that I produce contrary test results, as though I need to prove that the 16-35L DOESN'T have more flare! That's not the point, as anyone with half a brain would realize. I'm not claiming that one lens is or is not better than the other. I'm simply questioning the proof which Reichmann offers to support HIS claim that the 16-35L has more flare than the 17-40L.

Now, lest I be banished, I won't waste any more of Reichmann's precious bandwidth on this subject. I wouldn't want to be the reason the site goes under. How could I live with the guilt? ;-)

[DarkPenguin]

Actually, if you read your previous message you were clearly implying that you had reason to believe the contrary.  If it is something more than the rotation of the earth lets hear it.

As an aside I think I've determined that all lens reviews are bunk.  Ignoring sample variations and different end uses of the image if the person doesn't use the lens the way you would you won't get their results.

[Slough]

Peter:

I think you would do better to present a well argued case, rather than  post a few insults which sound like petulant whining.  

As far as I can see MR reviews equipment that he uses, or borrows for a short time, and then publishes his thoughts and opinions. He has experience of a wide range of products and quite often he has had long term use of the items in question. Sure the reviews are subjective. All reviews are subjective. Even ostensibly objective tests with detailed measurements are subjective, since the tester must decide what to include and how to interpret it.

On that last point, many people accept as gospel truth tests that are crammed to the gills with measurements. And yet they are unaware that the tests simply do not give a fair and balanced assessment of the product: all too often what is not measured is of more significance. In practice a supposedly subjective review can be more accurate.

As for the reliability of reviews, well each product is subject to sample variation, especially lenses, and of course each person has their own natural biases. IMO only a fool buys something because it came top in a test, or Joe Bloggs the famous photographer uses it. That is because their reasons for liking an item may be orthogonal to your needs. In practice it makes more sense to find as many reviews as possible, and preferably from sources that are respected.

I happen to consider reviews on this site to be interesting and informative and well worth what I pay for them!  

Curiously enough there has not been a stream of postings along the lines of "Blimey Michael, you ain't alf got that review wrong me old mucker, time to change careers, ever thought about road sweeping?". And that is despite numerous visitors to the forum who have clearly used some of the same equipment as MR. There has been some disagreement. That's to be expected.

Have a nice day y'all, as they on your side of the pond.

Leif

[michael]

I think that this thread has now run its course.

If anyone would like to contribute reasoned evidence one way or the other in this subject, then please start a new topic. I think we've had enough "opinion" though for now. Let's move on.

Michael