Are the DXO Mark ratings reliable?

[Keith Reeder]

Absolutely right, Guillermo. I've always had the impression that those who criticise the validity and accuracy of DXO results are simply trying to subjectively defend their own choice of a camera model which doesn't score well in the DXO tests.

Arrant nonsense, Ray. People balk at DxO's outpourings because the evidence of their own eyes belies any real World legitimacy their nonsense is supposed to have.

Canon (because that's what you really mean) files are absolutely glorious at the image level, and nothing DxO and its fanboys have to say about it changes that fact.

Or do you seriously believe that Canon users don't care about image quality?

They get it in spades from Canon bodies. Despite what DxO might have to say about it.

[BernardLanguillier]

Canon (because that's what you really mean) files are absolutely glorious at the image level, and nothing DxO and its fanboys have to say about it changes that fact.

Or do you seriously believe that Canon users don't care about image quality?

Coming from VHS, I used to find my DVDs amazing until I saw a Bluray.

Cheers,
Bernard

[shadowblade]

Arrant nonsense, Ray. People balk at DxO's outpourings because the evidence of their own eyes belies any real World legitimacy their nonsense is supposed to have.

Canon (because that's what you really mean) files are absolutely glorious at the image level, and nothing DxO and its fanboys have to say about it changes that fact.

Or do you seriously believe that Canon users don't care about image quality?

They get it in spades from Canon bodies. Despite what DxO might have to say about it.

They have great image quality... right up until you hit the technical limitations of the sensor. Which are significantly narrower than the limitations of other sensors out there.

Shooting high-contrast, dramatic landscapes and cityscapes with a lot of shadow areas, I constantly ran into pattern noise and badly blocked-up shadows when trying to bring out shadow detail, even when the highlights were as bright as they could be without blowing out. Shooting with the 5D2 was as much about finding ways around the sensor's limitations as it was about finding great compositions. Lots of compositionally-good shots (which looked fantastic when I later went back and took them with the A7r) had to be simply thrown out because of these dynamic range limitations. And exposure blending or GND filters weren't always possible or practical. That's why I ditched Canon bodies as soon as the A7r allowed me to use Canon tilt-shifts in front of an Exmor sensor. I've encountered far fewer DR issues since then.

If you shoot in controlled lighting, or mainly shoot low-DR subjects, or don't mind blowing the highlights or shadows (e.g. if the highlights and shadows will be in out-of-focus areas anyway) then Canon sensors give very good image quality. If you can't control the lighting (e.g. landscapes) and shoot at low ISOs (Canon is comparable at high ISO) then you run into a lot of problems with Canon sensors which are much less of a problem shooting with other sensors.

Also, I like the greater resolution for larger prints.

[Slobodan Blagojevic]

... do you seriously believe that Canon users don't care about image quality?...

What a silly, silly question! Of course we do! That is precisely why we are so pissed off with Canon for not providing it at the state-of-the-art level. Instead, Canon presented yet another incarnation of the rotary phone model, very capable in its own right, though. It is just that the world has been using mobile phones for a while, you know

[Jim Kasson]

What a silly, silly question! Of course we do! That is precisely why we are so pissed off with Canon for not providing it at the state-of-the-art level. Instead, Canon presented yet another incarnation of the rotary phone model, very capable in its own right, though. It is just that the world has been using mobile phones for a while, you know

I've been there. I first started using Nikons with the S2, and continued on with the F series. I bought 'em all, except the F2. When Nikon came out with the D1, I bit, and bought every update. But it started to dawn on me that Canon made better digital cameras. I turned green with envy, and stayed that way for a few years. Then I bit the bullet, and bought a 1dsII, a 1dII, and five lenses. I kept my Nikon stuff. I enjoyed those Canons for more than a year. The Nikon announced the D3. I switched back.

It could happen again, only the reverse...

Jim

[Guillermo Luijk]

Canon (because that's what you really mean) files are absolutely glorious at the image level, and nothing DxO and its fanboys have to say about it changes that fact.

DxO is just il messagero. If your application doesn't involve high contrast scenes where strong shadow lifting is applied in pp, you'll be fine with Canon.

This doesn't mean their sensors are glorious nor that DxO lies.

[ErikKaffehr]

Hi,

The Canon images I have seen have ben quite OK! That said, I would believe that Canon is behind in some areas and read noise is part of that.

My take is that if you are shooting high ISO, Canon is just OK. But, if you are shooting low ISO and looking for shadow detail, Canon needs to improve.

Rumours say they will improve in a few months. Those rumours have been around for a few years.

Best regards
Erik

What a silly, silly question! Of course we do! That is precisely why we are so pissed off with Canon for not providing it at the state-of-the-art level. Instead, Canon presented yet another incarnation of the rotary phone model, very capable in its own right, though. It is just that the world has been using mobile phones for a while, you know

[spidermike]

Can anyone put a printed Canon image and a printed Nikon image side by side and say which is which?

But back to DxO - even they do not make any claims about the meaningfulness of their results. They simply provide the data and let people argue over them. For example what does a difference of 75 and 73 really mean in practice?

[shadowblade]

Can anyone put a printed Canon image and a printed Nikon image side by side and say which is which?

But back to DxO - even they do not make any claims about the meaningfulness of their results. They simply provide the data and let people argue over them. For example what does a difference of 75 and 73 really mean in practice?

Make it a 40x60" of a subject with fine detail and the resolution difference is apparent. If it's a scene with lots of deep shadows, the difference in shadow detail and pattern noise is also evident. Assuming, of course, no image blending or other HDR techniques have been used.

[ErikKaffehr]

Hi,

Personally, I don't like the DxO mark, it is just a figure of merit. But, DxO says that a difference below 5 points is not observable. So they clearly state that 75 or 73 do not matter.

Looking deeper in DxO figures says that a Nikon image at base ISO will give better shadow detail than a canon image at base ISO. Not necessarily better detail but less noise. If you shoot mostly at base ISO on tripod, as I do, this is highly relevant. If you shoot at high ISO it doesn't matter.

I would say that DxO-mark results are highly relevant within a limited scope, But there is a lot of information, that is quite useful.

On the other hand, todays cameras are very good. More often than not, the photographer is the limiting factor, and that is factor that DxO-mark dosn't take into account.

Best regards
Erik

Can anyone put a printed Canon image and a printed Nikon image side by side and say which is which?

But back to DxO - even they do not make any claims about the meaningfulness of their results. They simply provide the data and let people argue over them. For example what does a difference of 75 and 73 really mean in practice?

[Ray]

Then I bit the bullet, and bought a 1dsII, a 1dII, and five lenses. I kept my Nikon stuff. I enjoyed those Canons for more than a year. The Nikon announced the D3. I switched back.

Ouch! That was an expensive exercise! You bought a 1dsII, a 1dII, and five lenses, and used them for less than a couple of years? 

I made the switch to Canon in the days of film because they seemed to be a very innovative company and had a brilliant IS system built into some of their lenses. I had my eye on the new EF 100-400 IS zoom at the time, although I didn't buy that lens till later, when I got my first Canon DSLR, the D60.

As Canon began releasing new models of DSLRs, I felt very pleased I'd made the right decision in choosing Canon instead of Nikon and felt quite sorry for those Nikon owners who were stuck with their noisy, cropped-format models. 

When Nikon produced its first full-frame DSLR, the D3, I thought all the fuss was overrated. High ISO performance was marginally better than that of the 5D, by about 1/2 a stop of DR. The more significant improvement of the D3 was at its base ISO of 200. DR was over a stop better than that of the 5D at its lower base ISO of 100. That seemed to me to be a more useful performance feature, to be able to use a faster shutter speed whilst still gaining over one full stop of additional DR, compared with the 5D.

However, that was then, before DXOmark began publishing their sensor tests. If one compares the latest Canon and Nikon models, the gap in sensor performance seems to have widened in some respects. The D800E has a full 2 & 1/2 stops better DR, at base ISO, than the Canon 5D Mk3. Although to be fair, Canon has improved its high-ISO performance to the point where it's occasionally slight better than that of the Nikon D800E.

For example, the Canon 6D has about 1/2 a stop better DR at ISO 6400, although at base ISO the 6D is still 2 & 2/3rds EV behind the Nikon D810, regarding DR. That's a very significant advantage for the D810, as well as its higher resolution of 36mp, compared to the 20mp of the 6D. A half stop improvement in DR is the point where things begin to be noticeable. But 2 & 2/3rds of a stop! That's amazing! 

[Jim Kasson]

Ouch! That was an expensive exercise! You bought a 1dsII, a 1dII, and five lenses, and used them for less than a couple of years?  ;

It sure was expensive. However, I'm really happy that I saw the error of my ways early on, and didn't keep adding Canon lenses and bodies. One good thing did come out of it: I found a photographer who was committed to Canon, but didn't have any fast lenses. The project that I bought the Canons for needed all the light it could get, so I had a lot of fast lenses. I worked out a deal, now I have more hours of web programming services available to me than I could hope to use in this life.

In terms of money spent versus important pictures created, my little arabesque with Canon pales in comparison to my long , and possibly continuing, dance with the Hasselblad H system.

Jim

[shadowblade]

IMO they should get rid of the overall DxOMark score - it's based on subjective weightings and doesn't tell you anything useful about a sensor.

Stick to the actual data and the objective derived measurements (DR vs SO, col our depth vs ISO,  efficiency, SNR) which actually tell you how a sensor performs. If you're technically-minded and often push the limits of sensor capability, the numbers will speak for themselves. If you're not technically-minded and don't push the sensor's limits, then the sensor performance is meaningless to you anyway, since everything you shoot fits well inside the limits of any sensor.

[Manoli]

IMO they should get rid of the overall DxOMark score - it's based on subjective weightings and doesn't tell you anything useful about a sensor.

Stick to the actual data and the objective derived measurements

+1

[dwswager]

IMO they should get rid of the overall DxOMark score - it's based on subjective weightings and doesn't tell you anything useful about a sensor.

I don't mind the overall rating as long as you understand what it is.  You shouldn't be using it as a determing factor between one camera or another, but looking at the spread between them and trends.  For example, say you are looking at 2 cameras and 1 gets a 95 and the other a 77, then it should prompt you to go look at what is causing the spread. It may or may not be significant to you based on your intended use.  Another example right now would be that the 1st Canon comes up at #31 (Full Fram $6900 1Dx) just behind the $500 DX D3300 and most current Nikons are in the top 20.  This is about opposite of 5 years ago.  So it should prompt more investigation and deliberation on whether you feel the trend will continue or reverse again before choosing one brand or the other...or if it matters to you at all.

The 2 pictures below, though, illustrate exactly why you should look at the data.  From the 1st picture it looks like the D810 is absolutely better than the D750.  But when you get into the numbers, that advantage evaporates.  The 2nd picture shows the DR curve versus ISO and yep, the D810 can achieve better DR than the best the D750 can do.  But that is only because it has a base ISO of 64.  Once you get to ISO 100, the D750 beats it.  Which is better, depends where in the ISO range you are likely to be shooting.  Now most D810 shooters will shoot ALOT at 64ISO.  But not all the time.  And of course, there are other aspects to a camera besides just the quality of the data output of the sensor subsystem, especially when 2 cameras are this close!

[Manoli]

I don't mind the overall rating as long as you understand what it is. 

And to understand what it is ..

Stick to the actual data and the objective derived measurements (DR vs SO, col our depth vs ISO,  efficiency, SNR) which actually tell you how a sensor performs.

[dwswager]

And to understand what it is ..

No you don't need to look at the data to understand that number.  It is a statement about the best a camera can achieve!  Its a composite of it's best DR, Color Depth and ISO performance.  Basically, it is the ceiling on the camera sensor subsystem as determined under lab conditions. I don't need to look at the data to understand that.   In my example between the D810 (97) and D750 (93) it says that under the best conditions, the D750 will fall just short of the D810.  I need the data to understand how and when each will perform in that manner compared to how they perform overall and relative to other options so I can decide if it matters for me.

[Manoli]

Its a composite of it's best DR, Color Depth and ISO performance. 

Exactly, it's a composite - jack of all trades and master of none.

[Slobodan Blagojevic]

Exactly, it's a composite - jack of all trades and master of none.

Yet, we use measures like these all the time. Take GPA for example. A single number, encompassing numerous courses over several years, i.e, much more complex that the three-compent DXO one. It still gives you a valuable information in a single number.

[NancyP]

I guess that I am more interested in Bernard's images than in the DXOMark ratings. As it should be.