DxO Mark

[DeeJay]

It makes out, to the general sheeple, that a Nikon 800 beats a Phase One IQ180 or even P65 or IQ140 for that matter.

One look at the high res samples that Nikon released tells straight away it's not even in the league.

It's a pile of rubbish and I wish people would stop referencing it as gospel.

[MrSmith]

at 1600asa it comprehensively trounces it. 

[ejmartin]

DxO Mark is silly, it adds five apples to four oranges to give a total score of nine to one camera; three apples to seven oranges to give a total score of ten to another.  Which camera is better?  Obviously the ten score         Well, not if the attribute that is important to your photography is apples...

On the other hand, the individual data are quite useful if you know how to interpret them.  Many do not.

[jeremypayne]

DxO Mark is silly, it adds five apples to four oranges to give a total score of nine to one camera; three apples to seven oranges to give a total score of ten to another.  Which camera is better?  Obviously the ten score         Well, not if the attribute that is important to your photography is apples...

On the other hand, the individual data are quite useful if you know how to interpret them.  Many do not.

What if one camera gets 5 apples and 5 oranges, and the other gets 4 apples and 4 oranges?? 

As you say, the overall score is a bit silly, but I find the individual measurements quite illuminating.

[michael]

I don't disagree that summing up a camera's sensor in a single number is silly. I also do agree that their individual analysis is among the most objective and comprehensive around.

Like most things – a mixed bag.

Michael

[ErikKaffehr]

Hi,

DxO-mark is about noise characteristics and not image quality in a broader sense. It does not take lens resolution, MTF or even sensor resolution into account. The noise analysis is probably quite good.

Best regards
Erik

It makes out, to the general sheeple, that a Nikon 800 beats a Phase One IQ180 or even P65 or IQ140 for that matter.

One look at the high res samples that Nikon released tells straight away it's not even in the league.

It's a pile of rubbish and I wish people would stop referencing it as gospel.

[theguywitha645d]

It really comes down to the fact that information is dangerous to those that don't understand it, either those that think it is definitive or those that think it is useless. DxO give us a fairly objective way to compare sensor output. As Erik pointed out, there is no spatial information, which confuses a great number of people.

[Christoph C. Feldhaim]

Question: What is information?
Answer: Information is a difference that makes a difference.

I took this (not word for word, but the meaning) from a lecture the famouse computer scientist Joseph Weizenbaum once held here in Germany.

So - does the single number score make a difference?

For me it raises curiosity and doubt in the same moment when I hear "A FF DSLR trumps MFDB backs" - so it makes a difference, but not a significant one. After all only personal testing can tell and be the base of a decision for a system where the sensor only is a part of.

[marcmccalmont]

In the past I have proposed the following adjustment to DxO Mark scores, it is a great tool to evaluate sensor characteristics but it does not account for resolution. It takes 4 times the pixels to double resolution so why not take the dxo mark score and multiply it by the sqrt of the sensors resolution to get a real world value? 5DII = 79 x sqrt 21 = 362, D800 = 95 x sqrt 36 = 570, IQ180 = 91 x sqrt 80 = 814, this correlates with what I see. I personally use DxO to evaluate whether an upgrade  is worth it or not, if I can't get a 10 point increase in DxO Mark score I don't spend the money. Using this metric it certainly made sense to upgrade my P45+ to a IQ180 and it makes more sense to upgrade my 5DII to a D800 than a 5DIII.
Marc

[douglasf13]

Hi,

DxO-mark is about noise characteristics and not image quality in a broader sense. It does not take lens resolution, MTF or even sensor resolution into account. The noise analysis is probably quite good.

Best regards
Erik

Exactly.  The measurements of DxO Mark are actually quite good, but it's the interpretation of those numbers that gets confused. 

[Ray]

After all only personal testing can tell and be the base of a decision for a system where the sensor only is a part of.

Crikey! I don't have the time to personally test all options and choices for camera equipment. I rely upon the test results from specialised organisations like DXO and Photozone, as well as more general opinions from this site, in order to make an informed decision. Such reviewers provide a tremendously useful service, provided one knows what one wants and provided one is able to understand the significance of the test data.

[Christoph C. Feldhaim]

Crikey! I don't have the time to personally test all options and choices for camera equipment. I rely upon the test results from specialised organisations like DXO and Photozone, as well as more general opinions from this site, in order to make an informed decision. Such reviewers provide a tremendously useful service, provided one knows what one wants and provided one is able to understand the significance of the test data.

Sure.
But would you base a big decision like about sensor format or a system change just on hearesay, even if from people who know what they do?
I don't say these numbers are worthless, especially not the numbers the overall score is based on.
From what is known about the new Nikons for someone already having bought into Nikon this camera most likely is a no-brainer.
But for someone like me who is still shooting film and who is pondering if and when to go digital?
No way to omit the self testing for various reasons.

[Ray]

Sure.
But would you base a big decision like about sensor format or a system change just on hearesay, even if from people who know what they do?
I don't say these numbers are worthless, especially not the numbers the overall score is based on.
From what is known about the new Nikons for someone already having bought into Nikon this camera most likely is a no-brainer.
But for someone like me who is still shooting film and who is pondering if and when to go digital?
No way to omit the self testing for various reasons.

Certainly one shouldn't make purchasing decisions based only upon hearsay. The great value of DXOMark's tests is that they employ a consistent methodology so that accurate and valid comparisons can be made regarding specific attributes of camera performance such as noise, dynamic range, tonal range and color sensitivity etc, at various ISO settings. The least useful test would be a resolution test because that would depend upon so many other factors such as lens, AA filter, the RAW converter used, default sharpening and default noise reduction etc.

It's generally safe to assume that the sensor with the greater number of pixels will deliver the greater resolution from the same lens, unless pixel-counts are fairly close, in which case the strength of the AA filter and the brand of converter used may tend to cancel any slight advantage that one sensor may appear to have as a result of its slightly greater pixel count, such as 15mp versus 12mp.

Speaking from my own experience, I would say it's likely to be rather difficult and hazardous for someone who is not familiar with digital cameras, attempting to test those attributes that DXO address. At the very least, I would recommend hiring the camera for a few days. How many cameras are you prepared to hire? If certain experienced testers have already done the work, why the need to reinvent the wheel?

A major consideration in choice of camera is the brand of lenses one already has, and/or the type and quality of lenses that are available for use with the particular brand of camera one is considering purchasing.

I bought into the Canon system about 14 years ago primarily because Canon had a number of innovative Image Stabilised lenses available. Fairly recently I bought into the Nikon system primarily because of the superb Nikkor 14-24/2.8 lens. But there were also othe factors that influenced that decision, such as the better DR of the D700 compared with my 5D and features such autobracketing of up to 9 exposures with a 1 stop interval, and autobracketing of ISO using a fixed aperture and shutter speed, which is/was very useful for quick shots of moving subjects in variable lighting conditions, although that advantage has now diminished in the case of the D7000 and D800 because there's little IQ advantage with those cameras in using a high ISO as opposed to underexposing at base ISO.

[jeremypayne]

hearesay

For this to be hearsay, you'd have to be taking Ray's word for what DXO Mark reported.

Using well-documented data that comes from a repeatable and verifiable process is not hearsay.

Just sayin'.

[Christoph C. Feldhaim]

For this to be hearsay, you'd have to be taking Ray's word for what DXO Mark reported.
Using well-documented data that comes from a repeatable and verifiable process is not hearsay.
Just sayin'.

Even if it is exactly done - and I believe DXO is doing a great job with their measurements - how do you want to translate these numbers into the aesthetics of your image rendering?
I don't intend to criticize the DXOmarks procedures - just the way how we might deal with the results.
There are excellent lenses from Nikon, Leica and Zeiss you might want to put on your new D800E.
But what do the numbers really tell you about the look of the image?

I remember well the review of the Zeiss ZM 50 mm F2 Sonnar together with other lenses here on this site ( http://www.luminous-landscape.com/reviews/lenses/zeiss-m-mount.shtml ) which appeared to be an intersting process which shed some light on how we are influenced by our expectations. What appeared to be a flaw or weakness at the beginning (the soft look) showed up as being intentional and actually a feature after contacting Zeiss.

I recently traded in my Mamiya Press system against a Mamiya 7 ii with 3 lenses.
But I already miss the 100 mm f2.8 Sekor from the Press.
The Mamiya 7 lenses are surely sharper and overall better, but the old 100 mm had a great softer look I'm missing now.
And thats not only because of the lower F-Stop or being not so sharp - it was super sharp for an old shard of glass.
Even at comparable F-Stops it was just different in a pleasant way.

Seeing for oneself is totally absolutely ever ever necessary, since resolution scores, MTF charts and all the other measurements to not at all tell the whole truth about a lens or camera.
One might use the charts to leave out bad apples, but nothing can ever replace seeing for oneself if there is an artistical purpose behind the choice (for technical purposes this might be different).

Yeah - and just for completeness I think I should also point to the MTF articles on the Zeiss blog, especially the second part in issue 31 of their news:
http://blogs.zeiss.com/photo/en/?page_id=18  (Issues 30 and 31 - click on the respective "Special" Link)

Cheers
~Chris

[jeremypayne]

I don't intend to criticize the DXOmarks procedures - just the way how we might deal with the results.
There are excellent lenses from Nikon, Leica and Zeiss you might want to put on your new D800E.
But what do the numbers really tell you about the look of the image?

A specific car that interests you has a specific engine.  Someone could test that engine and report its specs to you.

Knowing the specs of that engine won't tell you what it will be like to drive the car that uses that engine as there is more to driving a car than just the engine specs.

Doesn't mean those specs are hearsay or useless ... also doesn't mean you shouldn't drive the car before you buy it ...

[Christoph C. Feldhaim]

A specific car that interests you has a specific engine.  Someone could test that engine and report its specs to you.
Knowing the specs of that engine won't tell you what it will be like to drive the car that uses that engine as there is more to driving a car than just the engine specs.
Doesn't mean those specs are hearsay or useless ... also doesn't mean you shouldn't drive the car before you buy it ...

... But would you base a big decision like about sensor format or a system change just on hearesay, even if from people who know what they do? ...

Seems we agree more than we disagree.
What was that discussion about again?
 

[jeremypayne]

Seems we agree more than we disagree.
What was that discussion about again?
 

All I was doin' was pointing out that calling DXO's scientific test results "hearsay" was a bit off ... hearsay has negative connotations regarding its reliability.

[Christoph C. Feldhaim]

All I was doin' was pointing out that calling DXO's scientific test results "hearsay" was a bit off ... hearsay has negative connotations regarding its reliability.

Uhh ... umm ... yes ...
The "hearsay" aspect I was pointing to didn't mean DXOmark, but the lack of personal experience when just reading numbers.
Issue solved.
Seems we can close the case and move on.

Cheers
~Chris

[Ray]

Seems we can close the case and move on.

Cheers
~Chris

Hey! Chris, it seems you have great difficulty moving on. You're still shooting film.