Pages: [1] 2 3 ... 5   Go Down

Author Topic: DXO tests  (Read 14297 times)

jerome_m

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 670
DXO tests
« on: November 22, 2014, 06:50:00 am »

I have seen a post about waiting for the DXO test for comparing one MF camera with a 24x36 camera using an Otus lens. While DXO may give some useful data, I would like to point out that the DXO test suite was designed to give results independent of the sensor size (one of their objectives was to compare P&S, APS-C and full-frame SLRs).

Therefore, by design, DXO tests will not show differences directly coming from using two cameras of different sensor sizes (e.g. different rendering of out of focus areas). They will show indirect differences (e.g. lower noise coming using larger sensels).

Whether these direct differences are important to one's practice is a different story, of course.
Logged

Theodoros

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2454
Re: DXO tests
« Reply #1 on: November 22, 2014, 08:45:42 am »

Actually, the man proposed that if Leica S will score better in DXO mark tests than the D810 he is currently using... He would buy it! Amazing... Surely DXO knows better on this guy needs than himself... But I'm sure I know better my needs than DXO ever will! So I replied to him that I'll be doing my own testing and add to my equipment whatever suits me...

By the way... IMO DXO testing is complete nonsense... Whenever I test my Nikon 16mp sensor against my other Nikon 36mp sensor which DXO claims has more DR... I end up with a picture with visibly more DR from my 16mp sensor.... I don't care how a sensor measures... I only care of what I can have after I process so that I can print the best possible image that my visualisation when capturing it created in my own mind... If one is to take DXO for granted... then, Canon users (the best selling camera maker) and MFDB users are fools...
Logged

bjanes

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3387
Re: DXO tests
« Reply #2 on: November 22, 2014, 08:50:13 am »

I have seen a post about waiting for the DXO test for comparing one MF camera with a 24x36 camera using an Otus lens. While DXO may give some useful data, I would like to point out that the DXO test suite was designed to give results independent of the sensor size (one of their objectives was to compare P&S, APS-C and full-frame SLRs).

Therefore, by design, DXO tests will not show differences directly coming from using two cameras of different sensor sizes (e.g. different rendering of out of focus areas). They will show indirect differences (e.g. lower noise coming using larger sensels).

Whether these direct differences are important to one's practice is a different story, of course.

This is not entirely true. DXO does publish a perceptual megapixel score for various camera/lens combinations as shown below.

With regard to pixel size, the SNR (signal to noise ratio) of large pixels is higher with large pixels when evaluated per pixel (Screen SNR/DR in DXO terminology), but when one makes a print, the shot with the smaller pixel camera is downsized to the same resolution as the image taken with the camera with large pixels, and the SNR advantage of the large pixel camera is lost (Print SNR/DR in DXO parlance). If read noise is well controlled, there is little difference between the print SNR between large and small pixels.

Regards,

Bill
Logged

BernardLanguillier

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 13983
    • http://www.flickr.com/photos/bernardlanguillier/sets/
Re: DXO tests
« Reply #3 on: November 22, 2014, 08:55:06 am »

Actually, the man proposed that if Leica S will score better in DXO mark tests than the D810 he is currently using... He would buy it! Amazing...

You've got to work on your reading skills...

Cheers,
Bernard
« Last Edit: November 22, 2014, 09:26:11 am by BernardLanguillier »
Logged

bjanes

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3387
Re: DXO tests
« Reply #4 on: November 22, 2014, 08:59:16 am »

You've to work on your reading skills...

Cheers,
Bernard


And also his testing skills.

Bill
Logged

Willow Photography

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 279
    • http://www.willow.no
Re: DXO tests
« Reply #5 on: November 22, 2014, 09:40:10 am »

And also his replying skills  :)
Logged
Willow Photography

ErikKaffehr

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 11311
    • Echophoto
Re: DXO tests
« Reply #6 on: November 22, 2014, 10:25:03 am »

Hi Bill,

I essentially agree with what you write. The only point I would make is that DxO hast not published lens tests MFD, AFAIK.

Best regards
Erik

This is not entirely true. DXO does publish a perceptual megapixel score for various camera/lens combinations as shown below.

With regard to pixel size, the SNR (signal to noise ratio) of large pixels is higher with large pixels when evaluated per pixel (Screen SNR/DR in DXO terminology), but when one makes a print, the shot with the smaller pixel camera is downsized to the same resolution as the image taken with the camera with large pixels, and the SNR advantage of the large pixel camera is lost (Print SNR/DR in DXO parlance). If read noise is well controlled, there is little difference between the print SNR between large and small pixels.

Regards,

Bill
Logged
Erik Kaffehr
 

jerome_m

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 670
Re: DXO tests
« Reply #7 on: November 22, 2014, 11:36:09 am »

This is not entirely true. DXO does publish a perceptual megapixel score for various camera/lens combinations as shown below.

With regard to pixel size, the SNR (signal to noise ratio) of large pixels is higher with large pixels when evaluated per pixel (Screen SNR/DR in DXO terminology), but when one makes a print, the shot with the smaller pixel camera is downsized to the same resolution as the image taken with the camera with large pixels, and the SNR advantage of the large pixel camera is lost (Print SNR/DR in DXO parlance). If read noise is well controlled, there is little difference between the print SNR between large and small pixels.

I know that DXO publishes two sets of measurements, one called "print" and one called "screen", but this is not what I was talking about. What I was saying is that both measurements are designed to ignore various factors that arise from the use of a bigger or smaller sensor. DXO tries to measure a sensor in isolation, but we are not using sensors in isolation to take actual pictures.
Logged

Doug Peterson

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 4210
    • http://www.doug-peterson.com
Re: DXO tests
« Reply #8 on: November 22, 2014, 11:47:04 am »

I've written many times about my technical issues with DXO. They very carefully (and I trust, honestly and accurately) measure a set of numbers which are of marginal use to most photographers.

bjanes

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3387
Re: DXO tests
« Reply #9 on: November 22, 2014, 12:06:35 pm »

I know that DXO publishes two sets of measurements, one called "print" and one called "screen", but this is not what I was talking about. What I was saying is that both measurements are designed to ignore various factors that arise from the use of a bigger or smaller sensor. DXO tries to measure a sensor in isolation, but we are not using sensors in isolation to take actual pictures.

The size of the sensor, other factors being equal, determines the number of photo-electrons captured and the signal to noise ratio varies as the square root of the number of photons captured. If the linear dimension of the sensor is doubled, the area is is increased by a factor of 4 and one will gain one stop of DR. The upshot of this is that the size of the sensor is taken into account.

Bill
Logged

bjanes

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3387
Re: DXO tests
« Reply #10 on: November 22, 2014, 12:11:46 pm »

I've written many times about my technical issues with DXO. They very carefully (and I trust, honestly and accurately) measure a set of numbers which are of marginal use to most photographers.

SNR and dynamic range are not of any importance to most photographers? When DXO gets around to testing the new CMOS MFDB sensors they will jump to the head of the sensor ratings and you might change your mind regarding the DXO tests.

Regards,

Bill
« Last Edit: November 22, 2014, 12:41:15 pm by bjanes »
Logged

jerome_m

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 670
Re: DXO tests
« Reply #11 on: November 22, 2014, 12:21:18 pm »

The size of the sensor, other factors being equal, determines the number of photo-electrons captured and the signal to noise ratio varies as the square root of the number of photons captured. If the linear dimension of the sensor is doubled, the area is is increased by a factor of 4 and one will gain one stop of DR.

That is true, mostly.

Quote
The upshot of this is that the size of the sensor is taken into account.

Up to a point, yes. But I said in my first message that the DXO tests "will show indirect differences (e.g. lower noise coming using larger sensels).".  Actually, they have to. There is no other way, unless one breaks the laws of physics.


But the tests do not show other effects. Therefore your last sentence should actually read: "The upshot of this is that the tests are designed to minimise the effects of the size of the sensor to the lowest point made possible by the laws of physics."


Anyway, I do not think that discussion is leading us anywhere. I was simply pointing out that DXO, whatever they are testing, do not tell the full story. Can we just simply agree that 2 camera-lens combination with the same DXO score may give very different results in practice?
« Last Edit: November 22, 2014, 12:25:40 pm by jerome_m »
Logged

Fine_Art

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1172
Re: DXO tests
« Reply #12 on: November 22, 2014, 12:25:07 pm »

The size of the sensor, other factors being equal, determines the number of photo-electrons captured and the signal to noise ratio varies as the square root of the number of photons captured. If the linear dimension of the sensor is doubled, the area is is increased by a factor of 4 and one will gain one stop of DR. The upshot of this is that the size of the sensor is taken into account.

Bill

I would say the size of the lens is the primary determinant of the number of photons captured. Following that, it is easier to project a large lens onto a large sensor really well, than squeezing the light onto a small sensor without wave front errors.

Which is why I chose full frame 35 for wide angles.
Logged

jerome_m

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 670
Re: DXO tests
« Reply #13 on: November 22, 2014, 12:43:17 pm »

I would say the size of the lens is the primary determinant of the number of photons captured.

You are right and it shows that the sentence you cited: "other factors being equal" is opened to interpretation. Most people would understand that f-stop should be equal. But for a larger sensor and the same apparent field of view the same f-stop will need a larger lens.

Quote
Following that, it is easier to project a large lens onto a large sensor really well, than squeezing the light onto a small sensor without wave front errors.

Which is why I chose full frame 35 for wide angles.

It is mostly the same for wide or narrow angles. The real difference is the f-stop, the geometrical aperture. Many optical aberrations increase like the square of the aperture, which is basically the same as saying "it is easier to project a large lens onto a large sensor really well, than squeezing the light onto a small sensor without wave front errors".
Logged

ErikKaffehr

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 11311
    • Echophoto
Re: DXO tests
« Reply #14 on: November 22, 2014, 12:56:52 pm »

Hi Jerome,

You can compare cameras using same generation Kodak sensors, like the Leica M9, the Leica S and the P45+.

Scores are 69, 76 and 77 larger wins. So the size differences are accounted for. Doubling sensor size gives about 10 points I guess.

Best regards
Erik




That is true, mostly.

Up to a point, yes. But I said in my first message that the DXO tests "will show indirect differences (e.g. lower noise coming using larger sensels).".  Actually, they have to. There is no other way, unless one breaks the laws of physics.


But the tests do not show other effects. Therefore your last sentence should actually read: "The upshot of this is that the tests are designed to minimise the effects of the size of the sensor to the lowest point made possible by the laws of physics."


Anyway, I do not think that discussion is leading us anywhere. I was simply pointing out that DXO, whatever they are testing, do not tell the full story. Can we just simply agree that 2 camera-lens combination with the same DXO score may give very different results in practice?
Logged
Erik Kaffehr
 

bjanes

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3387
Re: DXO tests
« Reply #15 on: November 22, 2014, 01:52:00 pm »

I would say the size of the lens is the primary determinant of the number of photons captured. Following that, it is easier to project a large lens onto a large sensor really well, than squeezing the light onto a small sensor without wave front errors.

Which is why I chose full frame 35 for wide angles.

That is quite correct. Roger Clark explains this in some detail using the concept of Etendue (sounds like a French word, maintaining the relationship with DXO). The larger sensor goes with a larger lens.

Bill
Logged

Theodoros

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2454
Re: DXO tests
« Reply #16 on: November 22, 2014, 04:20:54 pm »

SNR and dynamic range are not of any importance to most photographers? When DXO gets around to testing the new CMOS MFDB sensors they will jump to the head of the sensor ratings and you might change your mind regarding the DXO tests.

Regards,

Bill
They don't measure SNR or DR as relevant to PHOTO-graphy B ill... They measure it as unprocessed captured data... this is irrelevant to PHOTOgraphy or for many to understand.... They have first to understand what a photo-grapher is...
Logged

Theodoros

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2454
Re: DXO tests
« Reply #17 on: November 22, 2014, 04:37:26 pm »

in other words (do I have your attention B ill?)... to a photo-grapher, what SNR or DR a sensor captures is useless... It's what's left to him after processing that he cares (the latitude that affects the difference between capture and processing and what's left for the PHOTO-graph).... But that's a photo-grapher's way of thinking! ...understand B ill? You better, because there won't be anymore posts from me... No reason to argue on simple logic... that only concerns photo-graphers.
Logged

Fine_Art

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1172
Re: DXO tests
« Reply #18 on: November 22, 2014, 05:07:10 pm »

That is quite correct. Roger Clark explains this in some detail using the concept of Etendue (sounds like a French word, maintaining the relationship with DXO). The larger sensor goes with a larger lens.

Bill

Thanks for the link Bill, it is a good article as usual on that site. Roger is a gifted explainer.

That reminds me of a recent discussion in another thread about f ratio being the base constant vs angle of view with lens diameter, for determining a good pixel size.

Logged

Fine_Art

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1172
Re: DXO tests
« Reply #19 on: November 22, 2014, 05:14:22 pm »

in other words (do I have your attention B ill?)... to a photo-grapher, what SNR or DR a sensor captures is useless... It's what's left to him after processing that he cares (the latitude that affects the difference between capture and processing and what's left for the PHOTO-graph).... But that's a photo-grapher's way of thinking! ...understand B ill? You better, because there won't be anymore posts from me... No reason to argue on simple logic... that only concerns photo-graphers.


Maybe you should change your name to Animus.
Logged
Pages: [1] 2 3 ... 5   Go Up