Nikon D3 v. Canon 1Ds3: more useful test I.M.O.

[203]

http://diglloyd.com/diglloyd/2007-12-blog....3Canon_vs_Nikon

[DaveCurtis]

This is a very good comparison. Different cameras same lens!

Still waiting for my 1Ds Mrk III. Being at the bottom of the world hopefully doesn't make me bottom of the list :-)


Dave
down in NZ

[bart alexander]

http://diglloyd.com/diglloyd/2007-12-blog....3Canon_vs_Nikon
[a href=\"index.php?act=findpost&pid=161093\"][{POST_SNAPBACK}][/a]

Too bad the comparison is limited by thesharpening and resizing techniques off the tester. The comparison would have been compeletely usefull if Lloyd would have added the untouched: unsharpened and not resized files to "play" with. This way anyone could have judged for themselves if the smaller would work or not. My bet is NO.

[macgyver]

http://diglloyd.com/diglloyd/2007-12-blog....3Canon_vs_Nikon
[a href=\"index.php?act=findpost&pid=161093\"][{POST_SNAPBACK}][/a]

Wow, quite a difference.

[Quentin]

Useful perhaps for resolution and high ISO junkies; not dissimilar results to the comparison I did between my Oly E-3 and Mamiya ZD


http://qdfb.smugmug.com/gallery/4002795#232900466

password: formatbattle

In short, 21mp beats 12mp for resolution - big surprise there    but the two cameras are aimed at different market segments.

Quentin

[John Sheehy]

Too bad the comparison is limited by thesharpening and resizing techniques off the tester. The comparison would have been compeletely usefull if Lloyd would have added the untouched: unsharpened and not resized files to "play" with. This way anyone could have judged for themselves if the smaller would work or not. My bet is NO.
[a href=\"index.php?act=findpost&pid=161685\"][{POST_SNAPBACK}][/a]

At the same time, some people would see the smaller version on the screen and declare it sharper.

I don't think that there's anything wrong with upsampling to compare images; in fact, it is often a necessity.  However, it is best done by upsampling both, both with ratios that are known to have equal softening power.  Then, stand back from the monitor.

To cry that it is unfair to upsample the D3 in the comparison is just sour grapes.  The camera has less resolution, and it should be reflected in the comparison, if the comparison is meant to be about images to be viewed/printed at the same size.  If the comparison is about viewing/printing larger with more pixels, then the only fair way to do that comparison is to print both in their entirety, at the same PPI.  Crops of such are pretty worthless for that purpose.

[John Camp]

I have a D3 and I have no problem with the test -- seemed fair to me. Anybody who expected a 12mp camera to stay with a 21mp camera (at low ISOs) with the same good lens is delusional. On the other hand, if the test had been shot at 1600 or 3200, the results would have been otherwise...

I fully expect a ~20mp D3x in 2008; that will shoot high ISO about as well as a 1DsIII (and not as well as a D3), and the resolution will be more or less the same as a 1DsIII.

Comparing these two cameras (the D3 and 1DsIII) in absolute ways seems sort of pointless. They are very different, and will probably have different end users. I doubt that the 1DsIII's extra resolution will be particularly visible in magazine or web uses, but may be quite visible in fine art or highly cropped shots.

JC

[NikosR]

Any detail resolving comparisons between two unequal resolution cameras, will involve introducing some other factor in the test, rendering the test scientifically questionable and questionably interesting for practical purposes. For example, upsampling the image of the lower resolution camera to the resolution of the larger one, will introduce interpolation artifacts and errors. Doing the reverse also has its problems.

The only, not unexpected, result I can deduce from diggloyd's test is that if you want to print big the higher resolution camera is better. Which, of course, we all knew that was true, and, in any case, 'big' is not quantified. Hence, I find these sorts of tests next to useless.

I would have found much more interesting a subjective comment about how large you can print with the lower resolution camera until the benefits of the higher resolution one start to become evident. The author would not have to give as a web example to peek at. I would have taken his word for it.

[John Sheehy]

I have a D3 and I have no problem with the test -- seemed fair to me. Anybody who expected a 12mp camera to stay with a 21mp camera (at low ISOs) with the same good lens is delusional. On the other hand, if the test had been shot at 1600 or 3200, the results would have been otherwise...
[a href=\"index.php?act=findpost&pid=161864\"][{POST_SNAPBACK}][/a]

Uh, no.  The 1Dsmk3 is the king of super-high-ISO, super low light.  All the declarations of the D3 having the lowest noise of any FF camera came about before the 1Dsmk3 was released.  The mk3 has less read noise per pixel, and almost twice as many of them.  The D3 can't touch it in extremely low light.

The only benefit the D3 may have over the 1Dsmk3 is slightly less image shot noise, but that may not even be true or significant.  I haven't measured the 1Dsmk3 shot noise yet.  The D3 has a 1/3 stop shot noise benefit over the 5D, but the 5D is not an especially efficient camera with photons.

[NikosR]

Uh, no.  The 1Dsmk3 is the king of super-high-ISO, super low light.  All the declarations of the D3 having the lowest noise of any FF camera came about before the 1Dsmk3 was released.  The mk3 has less read noise per pixel, and almost twice as many of them.  The D3 can't touch it in extremely low light.

So, any support or substantiation or even 3rd party references for your claims? Many of us would be interested in these.

[John Sheehy]

So, any support or substantiation or even 3rd party references for your claims? Many of us would be interested in these.
[a href=\"index.php?act=findpost&pid=162240\"][{POST_SNAPBACK}][/a]

I am the source; these are things I've measured myself in RAW files.  There are no other reliable sources out there that I know of.   The world of certified experts generally works in the dark as far as RAW capture is concerned; most have never even looked directly at RAW data, and look at cameras through conversions.

The D3 surpasses the 5D in both shot noise and read noise, so it is cleaner at all levels, from deep shadows to highlights.  The two Canon mk3 cameras, however, have slightly less read noise than the D3 at ISO 1600.  The 1Dmk3 has less pixels, and much less photon capturing area, so it is noisier than the D3 at all tonal levels.  The 1Dsmk3, however, has the same read noise at the pixel level at high ISOs as the 1Dmk3, something which was not true of the earlier 1D FF and 1.3x pairs.  That was a surprise to me, that Canon had managed it for so many pixels with a good burst speed.  Here is a 100% crop of the 1Dsmk3 at ISO 51,000 with absolutely no noise reduction or sharpening; just WB, full color interpolation, and a little saturation boost (my own manual conversion in IRIS):

[bjanes]

I am the source; these are things I've measured myself in RAW files.  There are no other reliable sources out there that I know of.   The world of certified experts generally works in the dark as far as RAW capture is concerned; most have never even looked directly at RAW data, and look at cameras through conversions.

The D3 surpasses the 5D in both shot noise and read noise, so it is cleaner at all levels, from deep shadows to highlights.  The two Canon mk3 cameras, however, have slightly less read noise than the D3 at ISO 1600.  The 1Dmk3 has less pixels, and much less photon capturing area, so it is noisier than the D3 at all tonal levels.  The 1Dsmk3, however, has the same read noise at the pixel level at high ISOs as the 1Dmk3, something which was not true of the earlier 1D FF and 1.3x pairs.  That was a surprise to me, that Canon had managed it for so many pixels with a good burst speed.  Here is a 100% crop of the 1Dsmk3 at ISO 51,000 with absolutely no noise reduction or sharpening; just WB, full color interpolation, and a little saturation boost (my own manual conversion in IRIS):

[a href=\"index.php?act=findpost&pid=162277\"][{POST_SNAPBACK}][/a]

Rather than a grainy 1DsM3 shot at high ISO with no comparison to the other camera and with no quantitative data, I would be interested in full well capacity and read noise at various ISOs so that one could construct a noise model similar to that shown below, which is taken from Roger Clark's analysis of the Canon 1DMII. The analysis shows data for the brightest f/stop (0) down to shadows (12). At a glance one can see the shot noise, read noise and total noise expressed in electrons on the left and Data Numbers (12 bit and 16 bit) on the right. Due to the peculiarities of that camera, I used well count at ISO 100 rather then the full well of around 75,000 electrons. Once the basic parameters are determined by experiment one can use a spread sheet for more detailed analysis.

One must still look at the picture after the analysis is done, since the noise spectrum is different for cameras of different resolutions when the results are viewed at the same picture size (the camera with the higher resolution will have a finer grained noise, and this is not taken into account by the standard deviations). Previous Kodak work with film grain might be applicable for quantitative analysis of the noise spectrum.

As is, your statments do not carry much weight and are contrary to Michael's observations, which were confirmed by a group of experienced photographers.

[juicy]

The D3 surpasses the 5D in both shot noise and read noise, so it is cleaner at all levels, from deep shadows to highlights.  The two Canon mk3 cameras, however, have slightly less read noise than the D3 at ISO 1600.  The 1Dmk3 has less pixels, and much less photon capturing area, so it is noisier than the D3 at all tonal levels.  The 1Dsmk3, however, has the same read noise at the pixel level at high ISOs as the 1Dmk3, something which was not true of the earlier 1D FF and 1.3x pairs.  That was a surprise to me, that Canon had managed it for so many pixels with a good burst speed.  Here is a 100% crop of the 1Dsmk3 at ISO 51,000 with absolutely no noise reduction or sharpening; just WB, full color interpolation, and a little saturation boost (my own manual conversion in IRIS):


[a href=\"index.php?act=findpost&pid=162277\"][{POST_SNAPBACK}][/a]

Hi!
It would be interesting to see the whole image also.

Cheers,
J

[John Sheehy]

Hi!
It would be interesting to see the whole image also.
[a href=\"index.php?act=findpost&pid=162303\"][{POST_SNAPBACK}][/a]

Well, most of it would be white.  This entire 100% crop is pretty much black in a normal conversion of the full image; it's a dark shadow area of the original.

[John Sheehy]

Rather than a grainy 1DsM3 shot at high ISO with no comparison to the other camera and with no quantitative data, I would be interested in full well capacity and read noise at various ISOs so that one could construct a noise model similar to that shown below, which is taken from Roger Clark's analysis of the Canon 1DMII. The analysis shows data for the brightest f/stop (0) down to shadows (12). At a glance one can see the shot noise, read noise and total noise expressed in electrons on the left and Data Numbers (12 bit and 16 bit) on the right. Due to the peculiarities of that camera, I used well count at ISO 100 rather then the full well of around 75,000 electrons. Once the basic parameters are determined by experiment one can use a spread sheet for more detailed analysis.

One must still look at the picture after the analysis is done, since the noise spectrum is different for cameras of different resolutions when the results are viewed at the same picture size (the camera with the higher resolution will have a finer grained noise, and this is not taken into account by the standard deviations). Previous Kodak work with film grain might be applicable for quantitative analysis of the noise spectrum.

As is, your statments do not carry much weight and are contrary to Michael's observations, which were confirmed by a group of experienced photographers.

[a href=\"index.php?act=findpost&pid=162290\"][{POST_SNAPBACK}][/a]

[Ray]

Here is a 100% crop of the 1Dsmk3 at ISO 51,000 with absolutely no noise reduction or sharpening; just WB, full color interpolation, and a little saturation boost (my own manual conversion in IRIS):

 

I'm making an assumption that the D3 ISO rating is accurate. It has been in the past. The latest Canons, such as the 40D, have now corrected the understatement of ISO rating. But dpreview definitely rates the 5D ISO 1600 as being ISO 2000 and ISO 3200 as being ISO 4000.

If test shot comparisons are made between the D3 and the 5D at equal ISO settings and equal aperture and shutter speed, the D3 sensor will receive approximately 33% more photons. Now, that's what I'd call significant at high ISO. At ISO 100, 1/3rd of a stop underexposure is nothing to worry about. At ISO 3200 and 6,400 it is.

You can't do a proper high ISO comparison between the 5D and the D3 without either overexposing the 5D shot by 1/3rd of a stop, or stopping down the aperture on the D3 lens by 1/3rd of a stop, or for that matter underexposing the D3 shot.

Are there any such comparisons available that take this factor into consideration?

I've recently been walking around this fascinating city of Chiang Mai at night taking photos without flash, at ISO 1600 and 3200 with my 20D and 5D. I have lots of underexposures. I think to myself I could really use a D3 in this situation, yet when I see D3 examples on the net of shots at ISO 12,800 and 25,600, they look worse than my 5D underexposed shots.

Maybe it's just wishful thinking on my part.

Forgot to mention. The 100% crops in the ACR window have no noise reduction and no sharpening. The conversions have luminance smoothing at a very modest amount of 25. These images are processed minimally on a laptop. I don't even have noise reduction programs installed.

[Jonathan Wienke]

You can't do a proper high ISO comparison between the 5D and the D3 without either overexposing the 5D shot by 1/3rd of a stop, or stopping down the aperture on the D3 lens by 1/3rd of a stop, or for that matter underexposing the D3 shot.

Are there any such comparisons available that take this factor into consideration?

Here's a practical use for my DR test methodology:

Run my Real-World Dynamic Range Test on both cameras at their highest ISO settings with identical aperture settings. Choose the frame from each camera that most closely matches this image with regard to the overall legibility of the smallest text around the center white square. The camera with the fastest shutter speed can get the cleanest shot with the lowest light level. Of course, the other factor to consider is the exposure difference between the shadow noise shots and the corresponding shots for each camera where the small text starts disappearing due to channel clipping. That will give you an idea of how well each camera will hold detail in the highlights. By factoring these two comparisons together, you can get an idea of which camera is really best suited for low-light use.

If the 1Ds-MkIII has a noise floor 4 stops lower than the D3 (which is a fairly big if), it will be able to match the D3 on the first test. And since its ISO is 1600 instead of 25,600, it will probably beat the D3 when it comes to highlight detail retention. Anyone have access to both cameras and feel like running the test?

[Quentin]

The lengths people go to to "prove" their favored dslr brand is better  

Quentin

[John Sheehy]

The lengths people go to to "prove" their favored dslr brand is better   

Quentin
[a href=\"index.php?act=findpost&pid=162792\"][{POST_SNAPBACK}][/a]

It's a known fact that Canon DSLRs actually meter at 1.2 to 1.25x their stated ISO values, while most other brands do not; so Canon's 1600 is actually about 1900 - 2000.  Telling factual truths is not "going to lengths".  Sometimes it just happens that things aren't even in real life, and describing those unevenesses is not necessarily a bias.

Canon has had absolute S/N ratios ahead of the other DSLR brands for some time now.  The D3 is the first real encroachment on Canon's lead.  Canon has traditionally applied very weak chromatic noise reduction to their conversion style, while other companies have tried to hide their noise with desaturation and luminance filtering of the shadows, seemingly closing the gap to the unwary viewer.  Then, just when Nikon comes out with the D3, Canon trumps it again with the 1Dsmk3, with slightly higher quantum efficiency, and similar read noise (at the pixel level), but with almost twice as many pixels.

[John Sheehy]

Rather than a grainy 1DsM3 shot at high ISO with no comparison to the other camera and with no quantitative data, I would be interested in full well capacity and read noise at various ISOs so that one could construct a noise model similar to that shown below, which is taken from Roger Clark's analysis of the Canon 1DMII.

That would be nice, but I'd need a better set of test shots to do something of that nature.  As it is, in perusing the few 1DSmk3 RAWs that I have, it seems that it captures about 60,000 photons at RAW saturation at ISO 100 (which is actually about ISO 120, if it follows Canon's tradition).  Even if it were a real ISO 100 this time, this means the 1DSmk3 captures as many total photons as the D3.  If it really is ISO 120 or so, then it captures more photons than the D3 with the same exposure.

No one seems to be shooting OOF color checkers with the same lens and Av and Tv settings on both cameras, to help the quest for IQ truth.

As is, your statments do not carry much weight and are contrary to Michael's observations, which were confirmed by a group of experienced photographers.

[a href=\"index.php?act=findpost&pid=162290\"][{POST_SNAPBACK}][/a]

Can you say "fooled by noise reduction and/or different image magnifications"?  That's what often happens.