NEX-7 Rolling Review

[michael]

This will be my last comment on the subject.

I don't see why people are having such a hard time understanding what I was after. So, I will state it in different words, and hopefully more clearly than I have thus far.

Imagine that I am standing in the same place with both cameras and one lens. I know how they compare in terms of noise, dynamic range and other resolution independent factors, because I either believe in DxOMark, or I have discovered the answers for myself.

Now I want to know how they compare in terms of resolution. I know that one is 24 Megapixels and one is 18 Megapixels. The 24MP will always out-resolves the 18MP all other things being equal. That's not the point though, because that's a trivial case and not worth debating.

But, how does that translate to real-world prints of real-world images? I want to know which camera, taking a shot of the same subject, with the same angle of view, and making prints in which the coverage and size are the same, will look.

So I take a photograph with each camera, from the same spot, with the same lens. Now I make a print from each. But, because one covers a wider field of view I have to crop it to the same size as the one with narrower coverage. I also realize that the NEX-7 can make a larger print before running out of steam, because it has higher resolution. So, I make the biggest print that I can from the M9 (remaining within reasonable ppi parameters) and then I make a print from the higher resolution one at the same physical size, effectively giving up some resolution to equalize the playing field.

Come on folks. I've been doing this stuff, writing about it and teaching it for some 40 years! I understand the issues.

If you think that my methodology is wrong, fine. Do your own tests, and come to your own conclusions. Tell us about them.

But, I own both cameras (or at least I will when my NEX-7 is delivered and I return my test sample to Sony) and I wanted to know what the prints would like like under those conditions. Why those? Because they reflect the way I work.

I can't show you the actual prints I made and so I simulated them using the methodology described. My comparison prints and the online samples match quite closely, so I'm satisfied with the merits of what I published.

Michael

[Anadrol]

First I would like to tell you that I appreciate your site a lot, and this review as well.

Your Leica crop has only about 7.6 MP so you discarded 57% of the resolution.
An APS-C sensor has 2.36* less surface than a full frame sensor.

You ressed down the NEX file, but this is different than completely discarding the pixels !

Your test is interesting but it is only one side of the story.

"I asked Leica's chief lens designer if sensors were out-resolving lenses yet, and his answer was – no not yet."

So if lenses are better than sensors, just put a 35mm on the NEX, please,
I and many people would be very interested by the result.
The differences between the 35mm and 50mm lenses will be certainly less important than discarding 57% of the M9 resolution.

[dmerger]

Imagine that I am standing in the same place with both cameras and one lens.

Geez, Michael, asking me to imagine you with just one lens is like asking me to imagine a world with ten dimensions.  It just can't be done. 

[nucleonb]

Michael,
I have understood your test from the first reading. It is not complicated and see great value in your test results for both cameras and especially for me. No one, as I remember many posted comparricon tests, tried this simple and practical approach. It the value especially when changing non zoom lenses without changing position - not all of us have one excellent lens to share between two excellent bodies with different sensor sizes:-)

I am planning to purchase by end of the next week your Camera to Print & Screen videos after viewing just one of your posted video. Hope to find an answer for many questions.
>>> One question so far could not be answered by many asked: Canon rep, by Tim Grey (pro) and by many others including Red River paper technical support team.
How (or where) to turn off the Printer Managed Colors when printing with profiles using Adobe LR3.

My printer Canonon 9000 MK II, Computer Mac OSX 10.6, Adobe Lightroom 3 and Canon paper (or Red River paper) with profiles.
When I am printing with a profile (not manage by Printer) I would be always asked by LR3 not to forget to turn OFF the Printer managing colors. However, I could not find how or where and all who I have asked for help to find could not either. The prints with profiles are darker and little muddy. So, I am printing with the Printer managing the colors, which it not correct in many cases.

Again, your test is great! - simple and with no questionable settings or conclusions.
Leo

[ErikKaffehr]

Hi,

I may see the point, but than it depends on how large you print and,  to some extent, on which resolution you are using in the printer driver.

Best regards
Erik

This will be my last comment on the subject.

I don't see why people are having such a hard time understanding what I was after. So, I will state it in different words, and hopefully more clearly than I have thus far.

Imagine that I am standing in the same place with both cameras and one lens. I know how they compare in terms of noise, dynamic range and other resolution independent factors, because I either believe in DxOMark, or I have discovered the answers for myself.

Now I want to know how they compare in terms of resolution. I know that one is 24 Megapixels and one is 18 Megapixels. The 24MP will always out-resolves the 18MP all other things being equal. That's not the point though, because that's a trivial case and not worth debating.

But, how does that translate to real-world prints of real-world images? I want to know which camera, taking a shot of the same subject, with the same angle of view, and making prints in which the coverage and size are the same, will look.

So I take a photograph with each camera, from the same spot, with the same lens. Now I make a print from each. But, because one covers a wider field of view I have to crop it to the same size as the one with narrower coverage. I also realize that the NEX-7 can make a larger print before running out of steam, because it has higher resolution. So, I make the biggest print that I can from the M9 (remaining within reasonable ppi parameters) and then I make a print from the higher resolution one at the same physical size, effectively giving up some resolution to equalize the playing field.

Come on folks. I've been doing this stuff, writing about it and teaching it for some 40 years! I understand the issues.

If you think that my methodology is wrong, fine. Do your own tests, and come to your own conclusions. Tell us about them.

But, I own both cameras (or at least I will when my NEX-7 is delivered and I return my test sample to Sony) and I wanted to know what the prints would like like under those conditions. Why those? Because they reflect the way I work.

I can't show you the actual prints I made and so I simulated them using the methodology described. My comparison prints and the online samples match quite closely, so I'm satisfied with the merits of what I published.

Michael

[pegelli]

Your test is interesting but it is only one side of the story.

True, but Michael set out to only answer one of the many questions which is resolution on the sensor with one lens. One could theoretically conclude that the 24 MP APS-C should always be higher than a 18 MP FF but things like AA filter and other sensor characteristics could prove that theory wrong. So he tested and published the results to answer his question (which is indeed that the higher MP/smaller sensor outresolves the lower MP/bigger sensor).

I agree there are other interesting questions as well, for instance
1) putting a longer lens on the big sensor or shorter lens on the small sensor to compare the same image area
2) moving the camera with the smaller sensor back to cover the same image area
but the fact of the matter is that Michael gave full disclosure of his method and wasn't interested in the other two questions.

So if you're intersted in the answer to other questions there's two things you can do
1) ask Michael friendly to do more tests
2) do these tests yourself and post the results here

But saying he did the "wrong" test isn't going to solve anything, he did the right test to answer his question and it's an interesting result that I appreciate very much.

[ErikKaffehr]

Hi,

It's actually worse than discarding the pixels. Downsizing an image will cause a lot of artifacts and fake detail. It would not be very obvious on a subject like Michael's but the fake detail would still be there. Sometimes the effects will be subtle and often show up as fake detail.

Check this: http://bvdwolf.home.xs4all.nl/main/foto/down_sample/down_sample.htm

This page by the same author demonstrates the effects on a more everyday subject: http://bvdwolf.home.xs4all.nl/main/foto/down_sample/example1.htm

By the way, you can download Bart's test pattern here: http://bvdwolf.home.xs4all.nl/main/foto/down_sample/down_sample_files/Rings1.gif and test yourself.

Best regards
Erik

You ressed down the NEX file, but this is different than completely discarding the pixels !

[ErikKaffehr]

Hi,

I'm actually thankful that Michael does this and other tests. Unfortunately testers always face harsh and often undeserved critic.

Regarding the resolution issue it cannot be assumed that an 24 MP APS-C sensor would "outresolve" a 18 MP FF sensor. That does not only depend on AA filtering but at least as much on the lens and how much that lens is stopped down. The FF sensor has the advantage of the surface, making less demands on the lens. The smaller sensor has the advantage of having more pixels. Just stopping down from f/5.6 to f/8 has a perceivable effect on a 16 MP APS-C sensor.

Michael has a point regarding prints, but what resolution would he choose in the printer driver? The printer driver itself would use dithering, so it would not produce the same kind of artifacts as downsizing.

It is actually possible to scan crops of prints. I have done that on some occasions, because it demonstrates the whole processing chain.

Finally sharpening plays a very major role.

Attached images are from this article: http://echophoto.dnsalias.net/ekr/index.php/photoarticles/49-dof-in-digital-pictures?showall=1

Best regards
Erik

True, but Michael set out to only answer one of the many questions which is resolution on the sensor with one lens. One could theoretically conclude that the 24 MP APS-C should always be higher than a 18 MP FF but things like AA filter and other sensor characteristics could prove that theory wrong. So he tested and published the results to answer his question (which is indeed that the higher MP/smaller sensor outresolves the lower MP/bigger sensor).

But saying he did the "wrong" test isn't going to solve anything, he did the right test to answer his question and it's an interesting result that I appreciate very much.

[EricV]

Imagine that I am standing in the same place with both cameras and one lens.
Michael

Case A) The camera with the smaller sensor has such a small field of view that it does not capture the scene with the framing you want.  Result -- you go home without a picture.  This case was given no weight in your comparison.
Case B) Same as case A, but you reconsider and decide to capture the scene with the smaller sensor camera, stitching together multiple images to achieve the required field of view.  If you think about it, this is very close in spirit to the comparison you actually performed.  Result -- the stitched image wins.
Case C) You move on to a new subject or change your composition.  This time the camera with the smaller sensor has an adequate field of view.  Since you do not have the right lens for the other camera, you end up discarding most of its pixels.  Result -- the camera with the smaller sensor wins.  This is precisely the comparison you performed, and your result is perfectly valid and easy to understand, given this scenario.
Case D) You look through your camera bag again, and discover a longer focal length for the second camera.  The two cameras now have the same field of view and frame your image without wasted pixels.  Result -- the camera with the larger sensor wins!  This case was not considered in your comparison, but nearly any decent lens will provide this result, if the alternative is a large crop.

[ErikKaffehr]

Hi,

In case D you cannot know which camera wins. That depends on the lens, sensor and aperture chosen.

DPReview published comparison images from NEX-7 and Leica M9. The NEX-7 is sharper. See attached image.

Here the Leica image was upressed using bicubic to match resolution of the other two. It is not obvious to me that the FF cameras outperform the NEX7, although lens has same FOV. Leica left, NEX7 top right and Alpha 900 bottom right.

I'm not saying that DPReview's testing is fault less, nor that the way I compare images is the only correct way.

Note:

In this test the Sony Alpha 900 seems to outperform the Leica M9. That surprises me because I'd assume that the Leica would have the better lens. This also contradict testing made by Michael earlier where he found that the Leica outperformed the Alpha 900. On the other hand, the DPReview images show very similar differences to the test Erwin Puts has made here: http://imx.nl/photo/leica/camera/page176/s2part4.html (see the end of arcticle). Here are links to the images in Mr. Puts tests. By the way, if you don't know Mr. Puts he is the author of the Leica Compendium.

A900:


M9:


Nikon D3X is best of the bunch:


Leica S2 with bigger sensor and better lens is superior:


One explanation may be that the test images are central and shot at medium aperture where the lens may be diffraction limited. Most fixed focal normal lenses are very well corrected at medium apertures. If the images were shot at large aperture or crops were from edge or corner the superiority of the Leica lens may be visible.


Best regards
Erik

Case A) The camera with the smaller sensor has such a small field of view that it does not capture the scene with the framing you want.  Result -- you go home without a picture.  This case was given no weight in your comparison.
Case B) Same as case A, but you reconsider and decide to capture the scene with the smaller sensor camera, stitching together multiple images to achieve the required field of view.  If you think about it, this is very close in spirit to the comparison you actually performed.  Result -- the stitched image wins.
Case C) You move on to a new subject or change your composition.  This time the camera with the smaller sensor has an adequate field of view.  Since you do not have the right lens for the other camera, you end up discarding most of its pixels.  Result -- the camera with the smaller sensor wins.  This is precisely the comparison you performed, and your result is perfectly valid and easy to understand, given this scenario.
Case D) You look through your camera bag again, and discover a longer focal length for the second camera.  The two cameras now have the same field of view and frame your image without wasted pixels.  Result -- the camera with the larger sensor wins!  This case was not considered in your comparison, but nearly any decent lens will provide this result, if the alternative is a large crop.

[xxl_and]

From my point of view the explanation is clear and the review valid.

Michael, is it possible to also test the Minolta Rokkor-X 50mm F1.4 with the NEX-7 and compare it with the Leica 50mm you used?

[Alan Goldhammer]

My chances of owning either the Leica or the NEX-7 are remote.  From a scientific perspective the test was somewhat flawed as others have noted.  My only other observation is the rather large numbers of first time posters to this forum as a result of this test (plus the lack of real name identification though one of them indicated that they would be purchasing the most recent video tutorial which is a good thing for LuLa).

[Mark D Segal]

Hi,

DPReview published comparison images from NEX-7 and Leica M9. The NEX-7 is sharper. See attached image.


Best regards
Erik

Thanks for showing us all that Erik - and after all the hand-wringing about focus etc., it seems another reviewer approaching it in a slightly different way comes to roughly the same bottom line. One thing that would concern me about the DPReview approach is the up-sampling. It could also bias results - to what extent depending on how much and how done. As one discussant mentioned - there's no perfect methodology.

The one thing in all of this I would be careful of is to limit the differentiating variables to the strict minimum needed to answer the question being asked - and indeed to ask the questions in a manner allowing this, otherwise it becomes impossible to reliably associate causes with effects. So as usual, it all depends...............

[bjanes]

This will be my last comment on the subject.

But, how does that translate to real-world prints of real-world images? I want to know which camera, taking a shot of the same subject, with the same angle of view, and making prints in which the coverage and size are the same, will look.

So I take a photograph with each camera, from the same spot, with the same lens. Now I make a print from each. But, because one covers a wider field of view I have to crop it to the same size as the one with narrower coverage.
 
If you think that my methodology is wrong, fine. Do your own tests, and come to your own conclusions. Tell us about them.

But, I own both cameras (or at least I will when my NEX-7 is delivered and I return my test sample to Sony) and I wanted to know what the prints would like like under those conditions. Why those? Because they reflect the way I work.
Michael

This might have been Michael's last comment in this thread, but in cropping the Leica image, he has thrown away a great deal of its resolution. In comparing sensors of differing dimensions, it is customary to express resolution in terms of pixels per picture height. In the case of the M9, the picture height is 24 mm and the number of pixels is 3472, but in cropping the image, the effective picture height was 15.6 mm and 15.6/24*3472 or 2257 pixels. To the best of my knowledge, he has never explained this discrepancy.

I firmly believe that the methodology was incorrect and would be interested in a proper comparison, which I can not do since I have neither camera. So we will have to wait until someone does a proper comparison.

Regards,

Bill

[EricV]

One last post on this subject for me too.  For those of us old enough to remember film days, suppose Michael decided to compare a 4x5" view camera with an 8x10" view camera.  In addition to leveling the playing field by using the same lens, he could also level the playing field by using the same film type.  Then it is obvious that the conclusion of the test would be "the 4x5 has the same resolution as the 8x10".  Completely true, given the ground rules of the comparison, but in my opinion, completely missing the point of using the larger camera.

[PierreVandevenne]

And also completely true in general.

Is the point Michael makes so hard to get? The lens outresolves the sensor in the M9. That was the question, and that is the answer. Could be that it still outresolves the sensor in the Sony as well, we'll see with some future generation of camera. Stating that Michael misses the point/usefulness of a larger sensor is simply preposterous, and frankly, quite hard to understand given all the explanations he has given and his clear statements.

No one disputes the fact that the Sony, used from the same position as the Leica, would have an effective zero resolution on the field it does not cover. :-) Michael could have taken a shot closer or further to equalize FOVs, but he also explicitly explained why he didn't.

[pegelli]

I firmly believe that the methodology was incorrect and would be interested in a proper comparison, which I can not do since I have neither camera. So we will have to wait until someone does a proper comparison.

There's nothing "incorrect" about the methodology to answer Michael's question. You have a different question you think is more proper but that's beside the question (pun intended)

[ErikKaffehr]

Mark,

I absolutely agree that upsizing also has issues. One of the issues is that any resizing also requires some sharpening. Upsizing is also adding some artificial detail, but downscaling both discards real detail and creates fake detail. The upscaling was not DPReview methodology. I downloaded the raw images, converted using identical settings and rescaled. An issue with this is that we know little about the accuracy behind the DPReview images. For that reason I also looked at the images by Erwin Puts. Erwin is a real Leica expert, even if his expertise may mainly be in lenses. He described the methods shooting the images exactly. The pecking order is quite clear on those images Nikon D3X is best, Sony A900 not even close second and Leica M9 is honorable third. The real champ is Leica S2, however. All those images are at actual pixels.

This pretty much contradicts Michaels earlier tests, comparing Sigma SD1 to M9 and different Sony cameras. I don't know why. One point that is pretty clear that any decent fixed focal lens will be nearly diffraction limited at f/8 most often used in tests. Test shots used to be near the optical axis where aberrations normally are minimal. So the advantage that Leica lenses may have does not really come into play, because all decent lenses perform about the same on the optical axis at f/8.

Whatever method is used comparing images there will be some issues with focusing. Especially f/1.4 lenses do exhibit significant focus shift between f/1.4 and f/4. In Erwin Puts's test of the M9 the focus shift was 2 cm between f/1.4 and f/2.8 at 1.3m, a quite significant change. This affects "live view" more than AF as AF probably would use f/2.8.

What I have seen in previous tests I have made with my own stuff is that there is a big difference on image files from a 12 MP APS-C and a 24 MP full frame, but the difference between A2-prints may not be very obvious, much depending on the amount of fine detail. The difference show up for sure if you print large enough and look close enough. Looking at fine details in a large print can be quite attractive. Seeing fine detail is one of the reasons to print big. It is also the reason some folks buy Pentax 645D or even Phase One IQ180.

Best regards
Erik

Thanks for showing us all that Erik - and after all the hand-wringing about focus etc., it seems another reviewer approaching it in a slightly different way comes to roughly the same bottom line. One thing that would concern me about the DPReview approach is the up-sampling. It could also bias results - to what extent depending on how much and how done. As one discussant mentioned - there's no perfect methodology.

The one thing in all of this I would be careful of is to limit the differentiating variables to the strict minimum needed to answer the question being asked - and indeed to ask the questions in a manner allowing this, otherwise it becomes impossible to reliably associate causes with effects. So as usual, it all depends...............

[ErikKaffehr]

Hi,

The included screenshot demonstrates the amount of fake detail created by bicubic sharper.  The test target used shows these prominently.

You can download it here: http://bvdwolf.home.xs4all.nl/main/foto/down_sample/down_sample_files/Rings1.gif

Just open it in photoshop and resize the image to 66% using "Bicubic sharper". Note also that "uprezzing" would also give artifacts but significantly less than "downrezzing".

Note: The image has been made available by Bart van der Wolf, a frequent poster on this forums.

Best regards
Erik

[John Camp]

All this techie stuff on methodology seems to be missing one factor -- that what you want measured isn't what Michael was measuring. What he was doing is very clear. Nothing wrong with what he did -- in fact, that's what most photographers would do if out walking around shooting in the street. It's just not a lab test. Get a  life.