Pages: [1]   Go Down

Author Topic: Understanding DXO lens and camera ratings  (Read 3267 times)

wofsy

  • Newbie
  • *
  • Offline Offline
  • Posts: 38
Understanding DXO lens and camera ratings
« on: April 16, 2015, 08:36:34 am »


A quick look at DXO Mark tells you that the Nikon 810 has the best sensor - with the Canon 5d Mark 3 far behind - and the Zeiss lenses out perform most of the rest of the pack.

This is a strong message, so I would like to know exactly what these ratings mean.

For instance, it seems that for different camera bodies, the same lens can perform differently and all lenses seems to lose some of the pixel count that the camera theoretically provides. So a lenses on a 22 megapixel camera may only resolve 18 megapixels. Why do lenses lose information like this? Does a Canon 5d Mark 2 at a an overall rating of 82 fall far short of the Nikon 810 with a rating of 97 - or is this just 22 versus 36 megapixel difference?







Logged

spidermike

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 535
Re: Understanding DXO lens and camera ratings
« Reply #1 on: April 16, 2015, 08:56:09 am »

Quote
so I would like to know exactly what these ratings mean.

No-one knows, not even DxO. They are quite open about the fact that all they are doing is creating data and leaving the interpretation to others. IIRC they consider any difference in score of 5 or less to be negligible regards visible differences.
The main problem from my view is that brand-fans then spout DxO scores as a way to justify their own choice and the argument (or even abuse) that follows brings the whoel thing into question.

Quote
Does a Canon 5d Mark 2 at a an overall rating of 82 fall far short of the Nikon 810 with a rating of 97
The camera is more than the sensor (which is what DxO measures) and it depends what you want the camera for and what functionality you need. If you want it for fast-moving wildlife or sports then having a sesnor score of 97 is pretty meaningless if you don't get the shot but DxO does not measure this. If you are shooting landscape with a massive tonal range, or like to do a lot of recovery of heavy shadows then the D810 is superior and the DxO reflects that.
Logged

dwswager

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1375
Re: Understanding DXO lens and camera ratings
« Reply #2 on: April 16, 2015, 09:54:48 am »

A quick look at DXO Mark tells you that the Nikon 810 has the best sensor - with the Canon 5d Mark 3 far behind - and the Zeiss lenses out perform most of the rest of the pack.

This is a strong message, so I would like to know exactly what these ratings mean.

For instance, it seems that for different camera bodies, the same lens can perform differently and all lenses seems to lose some of the pixel count that the camera theoretically provides. So a lenses on a 22 megapixel camera may only resolve 18 megapixels. Why do lenses lose information like this? Does a Canon 5d Mark 2 at a an overall rating of 82 fall far short of the Nikon 810 with a rating of 97 - or is this just 22 versus 36 megapixel difference?


The overall scores are nothing more than an amalgamation of all the data to give a single numerical rating.  You need to understand how they amalgamate the date to make any sense.  The point is that a camera with score X and one with score X-5 probably don't have a whole lot of difference.  But 2 cameras with Scores X and 2X do.

The real key to using Dx0Mark is to look at the individual tests and the data behind them.  For example, the D810 performs really well with respect to DR.  But it really is at base ISO.  Once ISO starts going up it starts to pull back to the field, especially up around 1600.  For Landscape guys the D810 outperforms, in this aspect because they tend to shoot at low ISO.  For sports and other shooting at high ISO not so much.

Where people get mislead is by not understanding what the numbers mean.  For example, the overall score of the 7DmkII is around the same as the D300.  But  I can tell you I never shot my D300 over ISO 800.  The 7DmkII performs well over that ISO.  So what the numbers mean is in pedestrian shooting where you are not taxing the sensor, they will perform somewhat comparable, but when you need high ISO, the 7DmkII will outperform it. 

What you see today is that, in general, the Sony sensors outperform the Canon sensor.  That is just fact.  Five to six years ago, Canon was the sensor king.  That does not mean that the camera that houses a Sony sensor is a great camera or that a Canon camera is not.  What it means is that all else equal, a camera will perform better with a Sony sensor.  A Canon 5DmkIV with a 36MP Sony sensor would be killer.
Logged

DeanChriss

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 592
    • http://www.dmcphoto.com
Re: Understanding DXO lens and camera ratings
« Reply #3 on: April 16, 2015, 10:22:30 am »

A quick look at DXO Mark tells you that the Nikon 810 has the best sensor - with the Canon 5d Mark 3 far behind - and the Zeiss lenses out perform most of the rest of the pack.

This is a strong message, so I would like to know exactly what these ratings mean.

For instance, it seems that for different camera bodies, the same lens can perform differently and all lenses seems to lose some of the pixel count that the camera theoretically provides. So a lenses on a 22 megapixel camera may only resolve 18 megapixels. Why do lenses lose information like this? Does a Canon 5d Mark 2 at a an overall rating of 82 fall far short of the Nikon 810 with a rating of 97 - or is this just 22 versus 36 megapixel difference?

DXO looks at camera and lens pairs. For instance, the same lens on a high resolution body will always receive a higher resolution rating than the same lens on a lower resolution body because the higher resolution camera will always resolve more of what a given lens provides. The score will always be less than the resolution of the camera because anything less than a theoretically perfect lens and image sensor system will have losses. I don't know any details of exactly how they come up with the particular numbers, and I don't know if data interpolation from a Bayer type image sensor (which is most widely used) is factored in, but that is a large and unavoidable resolution loss right off the bat.

Regarding lenses I have noticed that there are important parameters that DXO does not consider, like flatness of field at real world distances. That can make or break lens performance, but you have to find out how a given lens performs elsewhere.   

With all of that said, DXO is a valuable resource, but it's only part of the story.
Logged
- Dean

digitaldog

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 20630
  • Andrew Rodney
    • http://www.digitaldog.net/
Re: Understanding DXO lens and camera ratings
« Reply #4 on: April 16, 2015, 01:51:25 pm »

Do they provide any kind of metric that defines what is and isn’t visible with the scores? In Colorimetry we know a report that tells us the average, min or max dE of less than 1 isn’t visible. So we can decide what is or isn’t pertinent with the so called scores. I’d hope DXO would do something similar as if we can’t see the difference, is it even meaningful?
Logged
http://www.digitaldog.net/
Author "Color Management for Photographers".

Some Guy

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 729
Re: Understanding DXO lens and camera ratings
« Reply #5 on: April 16, 2015, 03:26:53 pm »

Logged

ErikKaffehr

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 11311
    • Echophoto
Re: Understanding DXO lens and camera ratings
« Reply #6 on: April 16, 2015, 04:33:40 pm »

Hi,

DxO-mark sensor is about noise. How clean signal the sensor can deliver. DxO actually measures colour accuracy, using a standard called SMI, but that is not a part of DxO-mark.

Now, DxO-mark is not a bad thing. It is based on three sets of parameters:

- Dynamic range -- abiliity to reproduce shadows in a a correctly ETTR exposed image.
- Maximum ISO for acceptable reproduction
- Number of colours resolved at different ISO

Each of these metrics are well founded and may even be relevant.

BUT, essentially all cameras are good enough.


So, the differences that are measurable may matter little, as they are far beyond observable limits.

The reason that Nikon leads far beyond Canon is that it can extract far more detail in the shadows than Canon at base ISO. It may matter a lot if you shoot base ISO on tripod and high contrast subjects. But, whenever you go past say ISO 500 or so, Canon will be a good match for any Nikon or Sony. Look at DxO-s sports rating, and results will be different.

Lenses are quite different. Take two decent lenses at say f/8. They will be very close. If you shoot at f/8 a 300 $US lens will perform close to a $3000 $US lens. But, the 3000 $US lens may bring decent or even near perfect performance at f/1.4. That is what you are paying for with an Otus. Shoot at f/8 and any decent quality lens will be a decent match for an Otus.

Now, an Otus lens can deliver near perfect image quality at f/1.4. If you need that the Otus will deliver.

DxO-mark for lenses tries to take large aperture performance into account. If you want to have maximum image quality and shoot medium apertures, DxO-mark will present a pessimistic view of lenses. But, if you want to take very sharp pictures with a very thin zone of focus using large apertures, DxO-mark will make sense to you.

So, DxO mark gives a good information about the system, if you take it to the limits, but for normal work at medium ISO, medium aperture and so on most sensors and lenses will do an excellent job, leaving DxO-mark irrelevant.

Best regards
Erik




A quick look at DXO Mark tells you that the Nikon 810 has the best sensor - with the Canon 5d Mark 3 far behind - and the Zeiss lenses out perform most of the rest of the pack.

This is a strong message, so I would like to know exactly what these ratings mean.

For instance, it seems that for different camera bodies, the same lens can perform differently and all lenses seems to lose some of the pixel count that the camera theoretically provides. So a lenses on a 22 megapixel camera may only resolve 18 megapixels. Why do lenses lose information like this? Does a Canon 5d Mark 2 at a an overall rating of 82 fall far short of the Nikon 810 with a rating of 97 - or is this just 22 versus 36 megapixel difference?








Logged
Erik Kaffehr
 

dwswager

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1375
Re: Understanding DXO lens and camera ratings
« Reply #7 on: April 16, 2015, 05:31:49 pm »

BUT, essentially all cameras are good enough.

So, the differences that are measurable may matter little, as they are far beyond observable limits.

The reason that Nikon leads far beyond Canon is that it can extract far more detail in the shadows than Canon at base ISO. It may matter a lot if you shoot base ISO on tripod and high contrast subjects. But, whenever you go past say ISO 500 or so, Canon will be a good match for any Nikon or Sony. Look at DxO-s sports rating, and results will be different.

...

Best regards
Erik

Most cameras are good enough for a vast majority of shooting situations.  It is only at the margins where a performance difference becomes significant.  Since most shots don't require 13 stops of DR or 10fps, those types of performance features ONLY matter in the very rare cases where they are needed.

I will disagree somewhat on the performance only showing at base ISO however.  That is true for DR and color, but shoot both a D810 and 5DmkIII at ISO 6400 with 3 stops of underexposure (ISO 480,000 equivalent) and compare.  This is the worst case I have found on high school fields at 1/640th and f/2.8 so for me it is real world.  For example, most soccer fields have 2 light standards on each side which usually results in dead spots in the middle of the field and at the goals!
Logged

BernardLanguillier

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 13983
    • http://www.flickr.com/photos/bernardlanguillier/sets/
Re: Understanding DXO lens and camera ratings
« Reply #8 on: April 16, 2015, 05:32:44 pm »

The DxO single score puts a high emphasis on low ISO image quality, in particular DR. It makes it relevant for landscape, aritechture or studio photography but less so for non tripod/non flash work. This being said, most sensors perform in a similar fashion at high ISO, but there are important differences at low ISO.

For a sensor quality rating system less centered on low ISO image quality, you may want to refer to www.lenscore.org

The trend is pretty much the same though... and the superiority of Zeiss APO designs is even more clear I would say since they use a custom made 200mp slow CCD sensor to measure lenses on the exact same set up.

Cheers,
Bernard

wofsy

  • Newbie
  • *
  • Offline Offline
  • Posts: 38
Re: Understanding DXO lens and camera ratings
« Reply #9 on: April 18, 2015, 11:52:14 am »

 Thanks. This explains a lot. So let me see if I can summarize your points for practice. For street photography where medium f stops are common and fill flash is wide spread, pretty much any lens will take you home. Since iso's are low - around 200 - the Nikon may give you more shadow detail and truer color.

For landscapes higher rated sensors are preferred since they resolve the shadows better but this will really only make a difference in high contrast scenes.

Not sure I understand the differences for fashion photography.



Logged
Pages: [1]   Go Up