NEX-7 Rolling Review

[Mark D Segal]

Then why don't you simply open your eyes and take a look? That's all it takes ...

You've ended all discussion with me, bud. Hide behind your acronym and carry-on your keyboard warfare with someone else.I have better things to do with my life.

[Mark D Segal]

+1.

-1

because cropping simply normalizes the comparison of covered scene but leaves the resolution of the retained material intact and that is what matters here.

As for the focusing argument - another red-herring. I have a great deal of difficulty believing a published author with 4 decades of experience and knowing that every manufacturer who matters will be reading this stuff makes those kind of mistakes, and especially after the description of the care he put into this. Time to get real guys.

[Mark D Segal]

I think that there is no perfect methodology as simply sensor sizes are different.

Michael's test is interesting,
but it should also be done with a 35mm Leica on the NEX and a 50mm on the M9.
With if possible both lenses having about the same resolution.

Also, testing the Zeiss 24mm on the NEX VS a 35mm on the M9 would be interesting.
After all this would be comparing the best of what the NEX system has to offer VS the best of Leica.

I'm also curious about bokeh differences.

I agree, there is no perfect methodology when the sensor sizes and resolutions are different; so the methodology needs to be adapted as best as possible to the questions being asked, and that I continue to think Michael has done as well could be expected. I don't think using different lenses would be as good a solution, because that introduces another differentiating variable whose impact is not under the author's control and not well enough knowable.

[allenmacaulay]

because cropping simply normalizes the comparison of covered scene but leaves the resolution of the retained material intact and that is what matters here.

Hypothetical question:  Suppose we had the adaptors required to mount M-series lenses on a Pentax Q, which has the same sensor size as a typical point & shoot compact camera.  The crop factor is about 5.6 and it's a 12.4MP sensor.  We then compare it to an M9, both cameras using a 24mm M-series lens of your choice.  We take the pictures, then equalize the field of view by cropping out around 96% of the photo from the M9, leaving only a tiny central portion.  Afterwards, we upsize the Pentax Q to 18MP to match the resolution of the M9.  We then conclude that the Pentax Q has a higher resolution compared to the M9 with regards to making pictures as a whole.  Is this a valid conclusion?

[Mark D Segal]

Hi,

You may be right on the focusing issue.

I'm not convinced it's reasonable to think there is a "focusing issue". Firstly, the first set of images have depth. You can examine both columns at all f/stops and throughout the scene range, and you will see that the left column out-resolves the right column. There would have to be a case of severe short-focus on the Leica to produce a result like this as a result of a focusing fault. Given the brand and the owner of the camera, while nothing is impossible, it lacks credibility. Then at f/5.6, the images from the next day confirm the finding of the day before. Of course a major generic fault would repeat the same error no matter how many times it is tried, but the likelihood of that error is low and the repeat test is useful for eliminating pilot error, it being unlikely that the pilot errs the same way over and over again. All as much to suggest, while not out of the question, real hard to believe focusing is the issue here.

[Mark D Segal]

Hypothetical question:  Suppose we had the adaptors required to mount M-series lenses on a Pentax Q, which has the same sensor size as a typical point & shoot compact camera.  The crop factor is about 5.6 and it's a 12.4MP sensor.  We then compare it to an M9, both cameras using a 24mm M-series lens of your choice.  We take the pictures, then equalize the field of view by cropping out around 96% of the photo from the M9, leaving only a tiny central portion.  Afterwards, we upsize the Pentax Q to 18MP to match the resolution of the M9.  We then conclude that the Pentax Q has a higher resolution compared to the M9 with regards to making pictures as a whole.  Is this a valid conclusion?

A great many arguments can be demolished by taking them to extremes and introducing variables that are not part of the relevant discussion.

[pegelli]

We then conclude that the Pentax Q has a higher resolution compared to the M9 with regards to making pictures as a whole.  Is this a valid conclusion?

Nobody (including Michael) made conclusions on resolution of pictures "as a whole", the only valid conclusion from this test (and from an experiment you describe) is about resolution on the sensor.

[ErikKaffehr]

Hi,

Making two tests reduces the probability of "pilot error", especially with a good pilot that we are sure to have. But systematic errors cannot be ruled out.

Here is quotation from Erwin Puts test of the Leica M9 http://imx.nl/photo/leica/camera/page155/m9part2.html :

"The camera lens must have the exact distance from lens flange to sensor location, the rangefinder must be adjusted to this distance too and the focus cam must be precisely machined with the correct steepness over the whole focusing range. Leica sets the nominal distance between flange and sensor location to 27.80 mm. I checked three lenses (3.8/24, 1.4/35 and 1.4/50) and got these values: 27.84, 27.83 and 27.81! This is an outstandingly good result and a fine indication of the care of production at the Leica factory."

So he tested three lenses and they were 10, 30 and 40 microns off. At 40 microns of at f/1.4 the offset would cause a circle of confusion of 28 microns that is about 4 pixels!

And here is another one:

"With maximum aperture the result is amazing: the zero position delivers the best result. Note that both extremes are less good The plus range gives generally the better performance: so it is best to focus a bit farther than indicated. With the aperture stopped down to f/2.8 the best position is the +2 focus: here we see in critical situations the effect of a slight focus shift. But it is also evident that even at 2.8 there is hardly room for focusing errors if the optimum performance is required The accuracy of the rangefinder of the M9 is beyond reproach, but the optional magnifier (1.4x) is absolutely necessary."

So in this case the range finder was exactly correct for full aperture but stopping down to f/2.8 shifted best focus to +2 cm, at 1.3 m object distance.

It's quite clear from Mr Puts testing that the probability for systematic error is real.

It's the beauty of live view that focusing can be done from the sensor image at actual pixels. Focus shift can also fool live view, if focusing is not done at shooting aperture, and it is absolutely possible that sensor is not perpendicular to lens axis or that the lens is misaligned.

Best regards
Erik

I'm not convinced it's reasonable to think there is a "focusing issue". Firstly, the first set of images have depth. You can examine both columns at all f/stops and throughout the scene range, and you will see that the left column out-resolves the right column. There would have to be a case of severe short-focus on the Leica to produce a result like this as a result of a focusing fault. Given the brand and the owner of the camera, while nothing is impossible, it lacks credibility. Then at f/5.6, the images from the next day confirm the finding of the day before. Of course a major generic fault would repeat the same error no matter how many times it is tried, but the likelihood of that error is low and the repeat test is useful for eliminating pilot error, it being unlikely that the pilot errs the same way over and over again. All as much to suggest, while not out of the question, real hard to believe focusing is the issue here.

[dreed]

In retrospect, what I dislike about Michael saying that he likes the output from the NEX-7 over the M9 is that he doesn't articulate which particular sections of the photo he believes represent that.

In many ways, the "hand waving" about the NEX-7 being better than the M9 reminds me of reviews written for audio equipment. People say "the <newer and often more expensive thing> is warmer, more open", etc. After you've read one or two reviews like that, you quickly learn to dismiss such reviews - if you want to know how a pair of speakers sound, the only way is to know is to find a shop that can plug them in for you and sit down to listen. Rinse and repeat with audio equipment. With digital cameras, we're in the fortunate position where the reviewer is able to share the material for review with us. If you read the reviews of websites such as dpreview, when they compare images from specific cameras, they will mention what part of the image they believe highlights X as being better or worse than Y. I can completely understand that he doesn't want to turn LuLa into dpreview, but if someone is going to say image X is better than image Y and show crops from both images, then it is to everyone's benefit if examples are given about why X is better than Y from the material that is presented. Otherwise, the images may as well have not been posted - unless the objective is just to generate discussion.

In short a key part of what I think is missing from the review is the ability to understand Michael's conclusion through specific examples in the crops that he believes represents his findings.

Way back when, I seem to recall that mention was made of how well camera X rendered some remote building Y. If you search back 5 or 6 years, you'll find the reviews that I'm referring to. When discussion is at that level of detail, it is much easier (as a reader) to come to the same conclusion - or at least understand - the points the review(er) is making about image quality. When all that is presented is hand waving, you're left feeling as if the reviewer is not being genuine with their comparison and is just leading us on to some conclusion based on relative new-ness and cool-ness rather than specific qualities.

[01af]

I have a great deal of difficulty believing a published author with 4 decades of experience and knowing that every manufacturer who matters will be reading this stuff makes those kind of mistakes, and especially after the description of the care he put into this.

So you decide to keep your eyes closed. Well, each to his own ... those who are both able and willing to use their eyes can easily see that in the six-frame series of comparison shots, the NEX-7 was focused at the bush in the frame's center; the M9 was focused at a somewhat greater distance. At f/1.4, the center is (clearly) sharper with the NEX-7; the corner is (slightly) sharper with the M9. At f/5.6, the center sharpens up only marginally with the NEX-7; the center sharpens up considerably with the M9. Both facts are very clear indications that the M9's focus was not on the bush but beyond.

Which renders the entire comparison worthless. And sheds a bad light on the author's ability to interpret and understand his own test shots.

... the only valid conclusion from this test (and from an experiment you describe) is about resolution on the sensor.

If the only thing you're interested in is the resolution in the sensor then why bother making test shots? Why don't you simply take a look at the tech specs? Sony NEX-7—24 MP. Leica M9—18 MP. Twenty-four is more than eighteen, so Sony has higher resolution. Case closed. Any questions?

If however you're interested in a real-world comparison then you'll have to look at both cameras' output, not at one square millimeter of each sensor's output. I predict this: When doing the comparison properly—i. e. using equivalent lenses and focusing accurately with both cameras—then the difference in the final images will be so small that you'd be hard-pressed to tell them apart. The NEX-7 has a slightly higher resolution but the M9 will mostly compensate this through a significantly bigger image area. Heck, I wouldn't be surprised if some people are going to prefer the M9 pictures over the NEX-7 pictures, despite the lower resolution. The only real-world problem is—perfectly accurate focus is so much easier to achieve with the NEX-7 than with the M9 ... this will spoil at least some of the M9's real-world results.

[allenmacaulay]

A great many arguments can be demolished by taking them to extremes and introducing variables that are not part of the relevant discussion.

Right.  So where do you draw the line?  It seems you support the article's conclusions so throwing away half the pixels from the M9 is acceptable to you for the purposes of the comparison.  Ok, so what about micro 4/3?  Many people mount Leica lenses on micro 4/3 cameras, so let's run the test with a Panasonic GX1 (16MP sensor) and an M9, cropping out 75% of the M9 image to match the field of view.  Is that still acceptable?  What about a Nikon 1 with its 2.7X crop factor sensor where 86% of the M9's frame is removed to equalize the field, is that still fair?

[Mark D Segal]

Right.  So where do you draw the line?  

I don't draw lines. I evaluate the reasonableness of the actual methodology being used in the context of the question or questions being addressed, and that evaluation is based on what I know about the problems raised by such comparisons drawn from my own experience doing them.

[BernardLanguillier]

Agreed.  Leica's 35mm and 50mm lenses will both outresolve the sensors on the NEX-7 and M9.  It also gets the field of view pretty close so that cropping is not a factor and we can look at what the sensor as a whole can do.  As the test stands we're looking at lines/mm on the sensor and disregarding sensor size, taking it to a logical conclusion the camera in an iPhone will have higher resolution than the NEX-7 if we used the same test methodology.

Yep, that makes more sense to me also. It is, by the way, the comparison method that has been used by most testers until now when comparing APS and FF.

That enables also the testing of images in print at the same size/same crop.

Frankly, I am amazed by the conclusion if Michael's tests this time because it is radically opposite to what he has been claiming for years when comparing the M9 to cameras like the D3x that have very weak AA filters. He was very vocal about the fanboy behavior of those claiming the opposite.

Either his past conclusions were wrong, or his current test result is wrong. I would love to hear from him about this weird discrepancy.

Or shall we just agree that whatever carries a Sony logo is superior?

Cheers,
Bernard

[ErikKaffehr]

Hi,

To begin with I'm not absolutely sure that the NEX7 has an OLP filter. The A55 I have has a very weak one and I have seen some color Moiré on it. The NEX7 has smaller pixel pitch, so I guess sooner or later the AA-filter can be dropped.

Now, the NEX7 has pixels that are about half the size of the Leica M9 pixels. The way Michael tested the NEX image is downscaled to match the Leica's resolution. The downscaling would effectively hide any effect of AA-filter. The downscaling will also introduce aliasing artifacts which can enhance impression of sharpness. The downscaling method also includes some sharpening.

So I don't think that Michaels testing contradict his old claims. I don't share his view on the issue of OLP filtering.

By the way, I have checked out the Leica M9 on the DPReview site, they have an image in their comparator. I have downloaded that raw image and compared against Alpha 900 and Panasonics latest camera (I don't recall which). The Leica image was not very sharp, but according to EXIF data it was shot at f/22, that could explain the softness. I also found some color Moiré in the Leica image, I wouldn't expect it at f/22, so I guess the EXIF information could be false. I don't see how the Leica would record the shooting aperture.

Going back to OLP-filtering, it's destructive effects are overstated for sure. If we look at almost any lens tests the lenses peak at around f/8 or even f/5.6. So diffraction clearly dominates over OLP filtering at apertures < f/8. Also, I did shoot a test with carefully calibrated amount of defocus (by moving camera) and it was quite obvious that diffraction effects  were quite visible as soon as the airy ring diameter exceeded pixel pitch. That test is here: http://echophoto.dnsalias.net/ekr/index.php/photoarticles/49-dof-in-digital-pictures?start=1

Best regards
Erik

Frankly, I am amazed by the conclusion if Michael's tests this time because it is radically opposite to what he has been claiming for years when comparing the M9 to cameras like the D3x that have very weak AA filters. He was very vocal about the fanboy behavior of those claiming the opposite.

Either his past conclusions were wrong, or his current test result is wrong. I would love to hear from him about this weird discrepancy.

Or shall we just agree that whatever carries a Sony logo is superior?

Cheers,
Bernard

[Anadrol]

Right.  So where do you draw the line?  It seems you support the article's conclusions so throwing away half the pixels from the M9 is acceptable to you for the purposes of the comparison.  Ok, so what about micro 4/3?  Many people mount Leica lenses on micro 4/3 cameras, so let's run the test with a Panasonic GX1 (16MP sensor) and an M9, cropping out 75% of the M9 image to match the field of view.  Is that still acceptable?  What about a Nikon 1 with its 2.7X crop factor sensor where 86% of the M9's frame is removed to equalize the field, is that still fair?

I totally agree !

But one thing is not clear, did Michael upped the resolution of the M9 crop from 7.6 to 18 MP ?
Because if the NEX image was downsized to 18 MP, it could not have the display size of the 7.6 MP Leica original file.

[ErikKaffehr]

Hi,


Here is another comparison between NEX7 and M9: http://www.luminous-landscape.com/forum/index.php?topic=60329.msg486308#msg486308

The images are taken from DPReviews comparison tools. EXIF data on M9 indicates f/22. If the EXIF data is correct than DPReview should have not posted this image! Does M9 record aperture in EXIF?!

Best regards
Erik

[dreed]

Either his past conclusions were wrong, or his current test result is wrong. I would love to hear from him about this weird discrepancy.

Or shall we just agree that whatever carries a Sony logo is superior?

If you've read enough of his reviews you'll understand that once he starts to like something newer always seems to be better regardless of the brand. You just have to accept that as part of his reviewing style. IMHO, although he's presented crops from the pictures that he took, he's failed to articulate specific areas of those crops that demonstrate why the NEX-7 is better than the M9. That makes his conclusion rather hollow.

Some of the subjective aspects of the review (handling, ergonomics, etc) are very worthwhile but I'm increasingly of the opinion that unless you're going to shoot test charts and run computer analysis, there's no value in a "resolution test" of a digital camera if it does not have an objective statement regarding the results. In other reviews there is specific mention of what is better/worse with a specific lens/camera. I can't find any such commentary with the NEX7/M9.

In review of the last pair of crop'd images, looking at the bare twigs of the tree against the various surfaces, it's hard to tell whether the photo on the left is better because of resolution or better because of the different colour and hence the greater contrast to the twigs. In some cases there is clearly a preference for the one on the left and none that favour the picture on the right. The two photos are obviously close together in time (no difference in shadow) but the colour in them is quite different. In that last pairing, there's no labelling to say which is which, so I'm not going to assume one or ther other.

[ErikKaffehr]

Hi,

There are many fallacies in testing. Small differences in color and tonalities may affect judgement. I don't think that Michael says that the NEX-7 is better than the M9, what he says is that the NEX-7 crops are sharper.

Best regards
Erik

If you've read enough of his reviews you'll understand that once he starts to like something newer always seems to be better regardless of the brand. You just have to accept that as part of his reviewing style. IMHO, although he's presented crops from the pictures that he took, he's failed to articulate specific areas of those crops that demonstrate why the NEX-7 is better than the M9. That makes his conclusion rather hollow.

Some of the subjective aspects of the review (handling, ergonomics, etc) are very worthwhile but I'm increasingly of the opinion that unless you're going to shoot test charts and run computer analysis, there's no value in a "resolution test" of a digital camera if it does not have an objective statement regarding the results. In other reviews there is specific mention of what is better/worse with a specific lens/camera. I can't find any such commentary with the NEX7/M9.

In review of the last pair of crop'd images, looking at the bare twigs of the tree against the various surfaces, it's hard to tell whether the photo on the left is better because of resolution or better because of the different colour and hence the greater contrast to the twigs. In some cases there is clearly a preference for the one on the left and none that favour the picture on the right. The two photos are obviously close together in time (no difference in shadow) but the colour in them is quite different. In that last pairing, there's no labelling to say which is which, so I'm not going to assume one or ther other.

[EricV]

Eric,

Yes, this is true in theory based on pixel pitch alone, but wouldn't be in practice, because the IQ would be crap. The whole point of this was that DxO has already passed their judgement on the relative merits of the sensors – excluding resolution. That's why the DxO chart was at the top of the segment.

I was then trying to level the playing field on the resolution topic. Guess some people just don't get it.

Michael

Michael,
Your method of "leveling the playing field" on resolution deliberately throws out all the advantages of a large sensor compared to a small sensor.  Do you get that?

For example, if you compared the actual Leica to a hypothetical Leica where someone had deliberately masked off all but the center pixels, your conclusion would clearly be that those two cameras have the same resolution.  In fact, they would have exactly the same image quality under the ground rules of your test.  While strictly true, such a conclusion is not very useful in practice, and is likely to be misleading to many readers.  I know which of those two cameras I would prefer to own ....

[nucleonb]

The Rolling Review is a big compliment to M9. I did not know that M9 is so good! - its image after being cropped is as good as a full NEX-7, 24 MP image! However, I can afford NEX-5n or may be NEX-7 in the future :-)
Excellent review series! Thank you!
Leo