Pages: 1 2 3 [4]   Go Down

Author Topic: Are the DXO Mark ratings reliable?  (Read 27184 times)

synn

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1235
    • My fine art portfolio
Re: Are the DXO Mark ratings reliable?
« Reply #60 on: November 20, 2014, 07:38:15 pm »

Ratings and images are not mutually exclusive.
 Look at it as a yelp review of a restaurant. It will give you a ballpark idea what to expect, but you are free to make your own conclusions after first hand experience.
Logged
my portfolio: www.sandeepmurali.com

Ray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 10365
Re: Are the DXO Mark ratings reliable?
« Reply #61 on: November 20, 2014, 09:34:59 pm »

I guess that I am more interested in Bernard's images than in the DXOMark ratings. As it should be.

Nancy,
The ratings, and the more detailed graphs, are for people interested in technical performance.

Surely we all want the camera with the best technical performance, provided there are no downsides, such as excessive bulk, weight, cost and poor ergonomics.

When there are downsides, as there usually are, we have to weigh the pros and cons.

Viewing Bernard's pictures and viewing DXO graphs are two quite different activities with different purposes. One shouldn't confuse them.  ;D
Logged

shadowblade

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2839
Re: Are the DXO Mark ratings reliable?
« Reply #62 on: November 20, 2014, 09:42:47 pm »

I guess that I am more interested in Bernard's images than in the DXOMark ratings. As it should be.

I find the numbers a lot more useful.

Photos are dependent on the skill and resources of the photographer. Just because photographer A produces great (or terrible) photos with camera A doesn't mean that photographer B will produce great (or terrible) images with camera A.

In contrast, the numbers and data tell me what each camera is and isn't capable of, and which piece of equipment will give me better results given the same shooting style and the same photographer.

Early photographers produced great work using primitive bodies and mediocre lenses. Blow them up to the size of a large print and you can quickly see all the technical limitations - lack of resolution, detailless shadows and blown-out highlights. Give the same photographers an IQ280, a set of Rodenstock lenses and a copy of Photoshop and they'd probably produce much better work - same compositional quality, but with much better detail and technical quality.
Logged

dwswager

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1375
Re: Are the DXO Mark ratings reliable?
« Reply #63 on: November 20, 2014, 10:05:46 pm »

Exactly, it's a composite - jack of all trades and master of none.


Interestingly enough, a significant portion of the technical quality of the images we make are limited by a composite of those 3 numbers!  And given the testing over numerous samples, it appears the Sony Sensor and Nikon's implementation within their camera sensor subsystem seems to be master of all.

Considering my D810 has a 97 rating and my D7100 has a 83 rating, I would expect, because of those numbers, that the D810 will exhibit better DR, Color Depth and ISO performance than my D7100.  And guess what, it does!  Just like that composite number indicates it should.

Now, if it would help me take more visually interesting photos, I'd be all set.
Logged

ErikKaffehr

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 11311
    • Echophoto
Re: Are the DXO Mark ratings reliable?
« Reply #64 on: November 21, 2014, 12:30:48 am »

Hi,

There are a few problems with images:

  • You don't know the processing that was done on the image
  • Images are normally shown in small sizes
  • The images are normally shown as JPEGs which contain much less data than the original raw, 8-bits/channel

It is not easy to do proper tests, and it is easier to them under lab conditions than under real world conditions. AFAIK I know the DxO data are very accurate, but they essentially measure sensor noise.

DxO also does lens testing, and again they have a quite obscure rating. But they also have a lot of measurement data. My lens purchases are essentially almost always based on lens tests.

Best regards
Erik






I find the numbers a lot more useful.

Photos are dependent on the skill and resources of the photographer. Just because photographer A produces great (or terrible) photos with camera A doesn't mean that photographer B will produce great (or terrible) images with camera A.

In contrast, the numbers and data tell me what each camera is and isn't capable of, and which piece of equipment will give me better results given the same shooting style and the same photographer.

Early photographers produced great work using primitive bodies and mediocre lenses. Blow them up to the size of a large print and you can quickly see all the technical limitations - lack of resolution, detailless shadows and blown-out highlights. Give the same photographers an IQ280, a set of Rodenstock lenses and a copy of Photoshop and they'd probably produce much better work - same compositional quality, but with much better detail and technical quality.
Logged
Erik Kaffehr
 

Guillermo Luijk

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2005
    • http://www.guillermoluijk.com
Re: Re: Re: Are the DXO Mark ratings reliable?
« Reply #65 on: November 21, 2014, 03:23:39 am »

I guess that I am more interested in Bernard's images than in the DXOMark ratings. As it should be.

Do you mean you'll decide the purchase of one or another camera system based in how good are the pictures produced by their owners?. Or you simply meant that you enjoy more looking at pictures than looking at graphs and numbers?. If this is the case I'll tell you enjoying pictures is totally compatible with the usefulness of DxO data in deciding a purchase.

shadowblade

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2839
Re: Are the DXO Mark ratings reliable?
« Reply #66 on: November 21, 2014, 09:38:34 am »

Hi,

There are a few problems with images:

  • You don't know the processing that was done on the image
  • Images are normally shown in small sizes
  • The images are normally shown as JPEGs which contain much less data than the original raw, 8-bits/channel

It is not easy to do proper tests, and it is easier to them under lab conditions than under real world conditions. AFAIK I know the DxO data are very accurate, but they essentially measure sensor noise.

DxO also does lens testing, and again they have a quite obscure rating. But they also have a lot of measurement data. My lens purchases are essentially almost always based on lens tests.

That's essentially what I said - looking at photos taken by different people using different methods doesn't tell you much about the sensor.
Logged

dwswager

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1375
Re: Are the DXO Mark ratings reliable?
« Reply #67 on: November 21, 2014, 10:42:12 am »

That's essentially what I said - looking at photos taken by different people using different methods doesn't tell you much about the sensor.

That is what lab test like DxOMark bring to the table.  They give you clues as to performance ceiling.

However, looking at real world photos gives you clues as to what can be done with the image.  What it won't give you is the how to get there!  And, of course, you must look at pictures in the output mode you intend.  If you're target output is large scale printing, looking at 640 x 480 px screen images isn't gonna help much!  Image making today, even Ansel Adams showed this in his day, isn't solely about what comes out of the camera.  You want the best possible data out of the camera because it keeps more possibilities open, but the image making process extends through post processing and output.  Give an expert in after camera image handling a less than stellar image file and he is likely to produce a final image better than most can with a stellar image file.  Which is why he wants that stellar file to begin with as he can do more with it.

Erik mentioned using lens tests as the bases for lens purchases.  I do as well, but not for selection.  My method is to identify my need, Identify candidate options.  Then I look at lens tests to 1) see if there is just some stellar performer in the options, 2) See if something just way out performs it's price point, 3) Identify lenses that look good on paper, but just perform poorly.  From the remaining candidates, I then try to use test data to see how their lab performance might translate to my real world shooting style.  For example, if I'm using it for landscapes, then wide open performance won't matter as much, and I might trade off some other characteristics for wide open performance.  Finally, I look at ergonomics, functionality, durability even filter size.  I standardized on 77mm awhile back for ND and CPL so if I get 2 similar options that I'm trying to decide between and 1 is 77mm and the other isn't then it might tip the balance.  As a Nikon shooter, I am very pleased that both the 16-35mm f/4G VR and 18-35mm f/3.5-4.5G are both 77mm as are the 24-70mm f/2.8G and the 24-120 f/4G VR. 
Logged

ErikKaffehr

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 11311
    • Echophoto
Re: Are the DXO Mark ratings reliable?
« Reply #68 on: November 21, 2014, 02:57:27 pm »

Hi,

Let me give a few real world examples:

1) Bought a Sony 24-70/2.8. It has useless corners at 24/2.8 and bad edges at 70 mm. When used at f/8 it is decently sharp.

2) I did consider replacing it with a Sigma 24-105/4, but according the tests I have seen it is not really better. Still considering it.

3) I have a Sony 70-300/4.5-5.6G. A decent performer, but it had let me down a few times. According to tests the 70-400/4-5.6G is a better performer and I am quite happy with it.

4) I have a Macro Planar 120/4 for my Hasselblad. It is a lens beloved by many. According to MTF data it has useless corners at infinty. It has useless corners at infinity unless stopped down to f/11, corners improve at f/16.

So I feel the tests tell the truth, but life is a compromise. For instance, the Macro Planar is clearly not a great performer bit it is acceptable when stopped down to f/11 (or even f/8). But it is a lens with a focal length I need. I had a Sonnar 150/4 a much better lens at infinity, but I found that the Planar 120 was more useful. So, I traded the Sonnar 150/4 for a Sonnar 180/4 an even better lens. I can do this as I have a very good dealer on old Hasselblad stuff here in Sweden and old Hasselblad stuff is dirt cheap.

Would BC chime in and tell me that his Contax 120/4 Apo macro is an excellent lens, I would hope he is aware that it is a very different design from the Hasselblad lens. A few more lenses and floating element design in the Contax lens. Actually, the Hasselblad 120/4 is an excellent lens, too, when it is used for it's intended purpose, which is close up. Zeiss recommends the Planar 100/3.5 for longer distances, hopefully I get mine monday next week. A nice thing is that you can figure out all that from Hasselblad's published MTF tests!

Best regards
Erik

That is what lab test like DxOMark bring to the table.  They give you clues as to performance ceiling.

However, looking at real world photos gives you clues as to what can be done with the image.  What it won't give you is the how to get there!  And, of course, you must look at pictures in the output mode you intend.  If you're target output is large scale printing, looking at 640 x 480 px screen images isn't gonna help much!  Image making today, even Ansel Adams showed this in his day, isn't solely about what comes out of the camera.  You want the best possible data out of the camera because it keeps more possibilities open, but the image making process extends through post processing and output.  Give an expert in after camera image handling a less than stellar image file and he is likely to produce a final image better than most can with a stellar image file.  Which is why he wants that stellar file to begin with as he can do more with it.

Erik mentioned using lens tests as the bases for lens purchases.  I do as well, but not for selection.  My method is to identify my need, Identify candidate options.  Then I look at lens tests to 1) see if there is just some stellar performer in the options, 2) See if something just way out performs it's price point, 3) Identify lenses that look good on paper, but just perform poorly.  From the remaining candidates, I then try to use test data to see how their lab performance might translate to my real world shooting style.  For example, if I'm using it for landscapes, then wide open performance won't matter as much, and I might trade off some other characteristics for wide open performance.  Finally, I look at ergonomics, functionality, durability even filter size.  I standardized on 77mm awhile back for ND and CPL so if I get 2 similar options that I'm trying to decide between and 1 is 77mm and the other isn't then it might tip the balance.  As a Nikon shooter, I am very pleased that both the 16-35mm f/4G VR and 18-35mm f/3.5-4.5G are both 77mm as are the 24-70mm f/2.8G and the 24-120 f/4G VR.  
« Last Edit: November 21, 2014, 03:00:35 pm by ErikKaffehr »
Logged
Erik Kaffehr
 

ErikKaffehr

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 11311
    • Echophoto
Re: Re: Re: Are the DXO Mark ratings reliable?
« Reply #69 on: November 21, 2014, 03:03:53 pm »

Hi Guillermo,

Based on Nancy's postings I am pretty sure that she is a lady making good decisions based on the information that is available.

Best regards
Erik

Do you mean you'll decide the purchase of one or another camera system based in how good are the pictures produced by their owners?. Or you simply meant that you enjoy more looking at pictures than looking at graphs and numbers?. If this is the case I'll tell you enjoying pictures is totally compatible with the usefulness of DxO data in deciding a purchase.
Logged
Erik Kaffehr
 

Lightsmith

  • Full Member
  • ***
  • Offline Offline
  • Posts: 197
Re: Are the DXO Mark ratings reliable?
« Reply #70 on: November 25, 2014, 08:08:15 pm »

Simply the fact that they do the testing at only one aperture and one zoom setting and do a subjective IQ appraisal for their "score" says it all.
Logged

dwswager

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1375
Re: Are the DXO Mark ratings reliable?
« Reply #71 on: November 25, 2014, 10:13:38 pm »

Simply the fact that they do the testing at only one aperture and one zoom setting and do a subjective IQ appraisal for their "score" says it all.

Huh?  Not sure what you are saying.  Can you please explain?
Logged

Fine_Art

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1172
Re: Are the DXO Mark ratings reliable?
« Reply #72 on: November 25, 2014, 11:09:58 pm »

ou might want to actually explore the measurements tab before saying they only test at one aperture and one zoom setting. Clearly you have no idea.
Logged

ErikKaffehr

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 11311
    • Echophoto
Re: Are the DXO Mark ratings reliable?
« Reply #73 on: November 26, 2014, 12:02:18 am »

Hi,

They actually test at all apertures. There is an incredible lot of data.

Best regards
Erik



Simply the fact that they do the testing at only one aperture and one zoom setting and do a subjective IQ appraisal for their "score" says it all.
Logged
Erik Kaffehr
 
Pages: 1 2 3 [4]   Go Up