Well, this is indeed a refreshing change of pace. My work is usually criticized for being overly quantitative, and relying on numbers as graphs when a simple picture would make the point.
That is of course a valid point. Except for Roger Cicala’s excellent work, I don’t know where you’re going to go for larger sample sets. Right here on LuLa, the sample size is usually (always?) one; are you chastising the people who charge you to read their tests for that? I do look for unreasonable results, and sometimes obtain another sample if I get them. I also check lenses for decentering and focus plane tilt, two indicators of improper assembly.
My blog posts are not intended for peer-reviewed scientific publications. I don’t have the time of inclination to test to those standards, nor would my readers have the patience to deal with writings that met the standards of scientific publications. All I am doing is applying what I call “kitchen optics” – tests that almost any reader could perform for herself, given the time and a modicum of equipment – to cameras and lenses, hoping to get insights that go beyond the usual “here are the pictures I took with the NiCanOrama QRZ – 1066, and here’s what I think of them” that most everybody else is doing.
The reason Roger Cicala uses a larger sample set is so that one
can draw practically meaningful inferences from a set of data.
In the case of the graphs that Erik posted, the equipment required is a razor blade, a light source, a focusing rail, MTF Mapper and/or Imatest, and Excel. As in all my reports, I explain exactly how a reader who wishes to reproduce my results can go about it, either in the post itself, or by reference to an earlier post.
Measuring MTF50 in cycles/picture height has a long history in digital photography. Try the Imatest site for some background. If you want the paper that introduced most of us to slanted edge MTF testing, it’s here:
http://imagescienceassociates.com/mm5/pubs/26pics2000burns.pdf
If you want the Matlab demonstration code, it’s here:
http://losburns.com/imaging/software/SFRedge/index.htm
MTF50 is a well-known sharpness metric. For a discussion of it and why it’s appropriate, look at Jack Hogan’s explanation:
http://www.strollswithmydog.com/mtf50-perceived-sharpness/
Erik pulled the graphs from some of my blog posts. If you read the posts, the axes are explained. In the MTF50 vs subject distance tests, the units are cm, with 0 arbitrary.
Thanks, I'll check it out.
I don’t see a histogram in anything that Erik posted. Thank you for the statistics lesson, though.
The last two diagrams are histograms, which is why they require error bars to draw any statistically valid inference from them.
Again, my blog posts are not scientific papers. The results are not statistically significant, to be sure. However, I think they form a useful addendum to the pretty pictures that are the alternative. To my knowledge, no one, not even Roger, is testing cameras and lenses and reporting results to the general public in the way you want them tested and reported.
Correct, but, Jim, as you well know, this is is why we do science the way we do. So we know what the
truth is. The reason scientific papers require results to be statistically significant so that valid inferences can be made from a (limited) data set about accuracy (i.e., the truth). Without that, there is no way to determine that the results obtained are not due to noise, sampling or random error, or ascertainment bias. This is why, as I stated in my first post, while I find your results to be of interest, I find it difficult to draw practically significant conclusions.
