Luminous Landscape Forum

Equipment & Techniques => Medium Format / Film / Digital Backs – and Large Sensor Photography => Topic started by: jerome_m on November 22, 2014, 06:50:00 am

Title: DXO tests
Post by: jerome_m on November 22, 2014, 06:50:00 am
I have seen a post about waiting for the DXO test for comparing one MF camera with a 24x36 camera using an Otus lens. While DXO may give some useful data, I would like to point out that the DXO test suite was designed to give results independent of the sensor size (one of their objectives was to compare P&S, APS-C and full-frame SLRs).

Therefore, by design, DXO tests will not show differences directly coming from using two cameras of different sensor sizes (e.g. different rendering of out of focus areas). They will show indirect differences (e.g. lower noise coming using larger sensels).

Whether these direct differences are important to one's practice is a different story, of course.
Title: Re: DXO tests
Post by: Theodoros on November 22, 2014, 08:45:42 am
Actually, the man proposed that if Leica S will score better in DXO mark tests than the D810 he is currently using... He would buy it! Amazing... Surely DXO knows better on this guy needs than himself... But I'm sure I know better my needs than DXO ever will! So I replied to him that I'll be doing my own testing and add to my equipment whatever suits me...

By the way... IMO DXO testing is complete nonsense... Whenever I test my Nikon 16mp sensor against my other Nikon 36mp sensor which DXO claims has more DR... I end up with a picture with visibly more DR from my 16mp sensor.... I don't care how a sensor measures... I only care of what I can have after I process so that I can print the best possible image that my visualisation when capturing it created in my own mind... If one is to take DXO for granted... then, Canon users (the best selling camera maker) and MFDB users are fools...
Title: Re: DXO tests
Post by: bjanes on November 22, 2014, 08:50:13 am
I have seen a post about waiting for the DXO test for comparing one MF camera with a 24x36 camera using an Otus lens. While DXO may give some useful data, I would like to point out that the DXO test suite was designed to give results independent of the sensor size (one of their objectives was to compare P&S, APS-C and full-frame SLRs).

Therefore, by design, DXO tests will not show differences directly coming from using two cameras of different sensor sizes (e.g. different rendering of out of focus areas). They will show indirect differences (e.g. lower noise coming using larger sensels).

Whether these direct differences are important to one's practice is a different story, of course.

This is not entirely true. DXO does publish a perceptual megapixel score (http://www.dxomark.com/Reviews/Looking-for-new-photo-gear-DxOMark-s-Perceptual-Megapixel-can-help-you) for various camera/lens combinations as shown below.

With regard to pixel size, the SNR (signal to noise ratio) of large pixels is higher with large pixels when evaluated per pixel (Screen SNR/DR in DXO terminology), but when one makes a print, the shot with the smaller pixel camera is downsized to the same resolution as the image taken with the camera with large pixels, and the SNR advantage of the large pixel camera is lost (Print SNR/DR in DXO parlance). If read noise is well controlled, there is little difference between the print SNR between large and small pixels.

Regards,

Bill
Title: Re: DXO tests
Post by: BernardLanguillier on November 22, 2014, 08:55:06 am
Actually, the man proposed that if Leica S will score better in DXO mark tests than the D810 he is currently using... He would buy it! Amazing...

You've got to work on your reading skills...

Cheers,
Bernard
Title: Re: DXO tests
Post by: bjanes on November 22, 2014, 08:59:16 am
You've to work on your reading skills...

Cheers,
Bernard


And also his testing skills.

Bill
Title: Re: DXO tests
Post by: Willow Photography on November 22, 2014, 09:40:10 am
And also his replying skills  :)
Title: Re: DXO tests
Post by: ErikKaffehr on November 22, 2014, 10:25:03 am
Hi Bill,

I essentially agree with what you write. The only point I would make is that DxO hast not published lens tests MFD, AFAIK.

Best regards
Erik

This is not entirely true. DXO does publish a perceptual megapixel score (http://www.dxomark.com/Reviews/Looking-for-new-photo-gear-DxOMark-s-Perceptual-Megapixel-can-help-you) for various camera/lens combinations as shown below.

With regard to pixel size, the SNR (signal to noise ratio) of large pixels is higher with large pixels when evaluated per pixel (Screen SNR/DR in DXO terminology), but when one makes a print, the shot with the smaller pixel camera is downsized to the same resolution as the image taken with the camera with large pixels, and the SNR advantage of the large pixel camera is lost (Print SNR/DR in DXO parlance). If read noise is well controlled, there is little difference between the print SNR between large and small pixels.

Regards,

Bill
Title: Re: DXO tests
Post by: jerome_m on November 22, 2014, 11:36:09 am
This is not entirely true. DXO does publish a perceptual megapixel score (http://www.dxomark.com/Reviews/Looking-for-new-photo-gear-DxOMark-s-Perceptual-Megapixel-can-help-you) for various camera/lens combinations as shown below.

With regard to pixel size, the SNR (signal to noise ratio) of large pixels is higher with large pixels when evaluated per pixel (Screen SNR/DR in DXO terminology), but when one makes a print, the shot with the smaller pixel camera is downsized to the same resolution as the image taken with the camera with large pixels, and the SNR advantage of the large pixel camera is lost (Print SNR/DR in DXO parlance). If read noise is well controlled, there is little difference between the print SNR between large and small pixels.

I know that DXO publishes two sets of measurements, one called "print" and one called "screen", but this is not what I was talking about. What I was saying is that both measurements are designed to ignore various factors that arise from the use of a bigger or smaller sensor. DXO tries to measure a sensor in isolation, but we are not using sensors in isolation to take actual pictures.
Title: Re: DXO tests
Post by: Doug Peterson on November 22, 2014, 11:47:04 am
I've written many times about my technical issues with DXO. They very carefully (and I trust, honestly and accurately) measure a set of numbers which are of marginal use to most photographers.
Title: Re: DXO tests
Post by: bjanes on November 22, 2014, 12:06:35 pm
I know that DXO publishes two sets of measurements, one called "print" and one called "screen", but this is not what I was talking about. What I was saying is that both measurements are designed to ignore various factors that arise from the use of a bigger or smaller sensor. DXO tries to measure a sensor in isolation, but we are not using sensors in isolation to take actual pictures.

The size of the sensor, other factors being equal, determines the number of photo-electrons captured and the signal to noise ratio varies as the square root of the number of photons captured. If the linear dimension of the sensor is doubled, the area is is increased by a factor of 4 and one will gain one stop of DR. The upshot of this is that the size of the sensor is taken into account.

Bill
Title: Re: DXO tests
Post by: bjanes on November 22, 2014, 12:11:46 pm
I've written many times about my technical issues with DXO. They very carefully (and I trust, honestly and accurately) measure a set of numbers which are of marginal use to most photographers.

SNR and dynamic range are not of any importance to most photographers? When DXO gets around to testing the new CMOS MFDB sensors they will jump to the head of the sensor ratings and you might change your mind regarding the DXO tests.

Regards,

Bill
Title: Re: DXO tests
Post by: jerome_m on November 22, 2014, 12:21:18 pm
The size of the sensor, other factors being equal, determines the number of photo-electrons captured and the signal to noise ratio varies as the square root of the number of photons captured. If the linear dimension of the sensor is doubled, the area is is increased by a factor of 4 and one will gain one stop of DR.

That is true, mostly.

Quote
The upshot of this is that the size of the sensor is taken into account.

Up to a point, yes. But I said in my first message that the DXO tests "will show indirect differences (e.g. lower noise coming using larger sensels).".  Actually, they have to. There is no other way, unless one breaks the laws of physics.


But the tests do not show other effects. Therefore your last sentence should actually read: "The upshot of this is that the tests are designed to minimise the effects of the size of the sensor to the lowest point made possible by the laws of physics."


Anyway, I do not think that discussion is leading us anywhere. I was simply pointing out that DXO, whatever they are testing, do not tell the full story. Can we just simply agree that 2 camera-lens combination with the same DXO score may give very different results in practice?
Title: Re: DXO tests
Post by: Fine_Art on November 22, 2014, 12:25:07 pm
The size of the sensor, other factors being equal, determines the number of photo-electrons captured and the signal to noise ratio varies as the square root of the number of photons captured. If the linear dimension of the sensor is doubled, the area is is increased by a factor of 4 and one will gain one stop of DR. The upshot of this is that the size of the sensor is taken into account.

Bill

I would say the size of the lens is the primary determinant of the number of photons captured. Following that, it is easier to project a large lens onto a large sensor really well, than squeezing the light onto a small sensor without wave front errors.

Which is why I chose full frame 35 for wide angles.
Title: Re: DXO tests
Post by: jerome_m on November 22, 2014, 12:43:17 pm
I would say the size of the lens is the primary determinant of the number of photons captured.

You are right and it shows that the sentence you cited: "other factors being equal" is opened to interpretation. Most people would understand that f-stop should be equal. But for a larger sensor and the same apparent field of view the same f-stop will need a larger lens.

Quote
Following that, it is easier to project a large lens onto a large sensor really well, than squeezing the light onto a small sensor without wave front errors.

Which is why I chose full frame 35 for wide angles.

It is mostly the same for wide or narrow angles. The real difference is the f-stop, the geometrical aperture. Many optical aberrations increase like the square of the aperture, which is basically the same as saying "it is easier to project a large lens onto a large sensor really well, than squeezing the light onto a small sensor without wave front errors".
Title: Re: DXO tests
Post by: ErikKaffehr on November 22, 2014, 12:56:52 pm
Hi Jerome,

You can compare cameras using same generation Kodak sensors, like the Leica M9, the Leica S and the P45+.

Scores are 69, 76 and 77 larger wins. So the size differences are accounted for. Doubling sensor size gives about 10 points I guess.

Best regards
Erik




That is true, mostly.

Up to a point, yes. But I said in my first message that the DXO tests "will show indirect differences (e.g. lower noise coming using larger sensels).".  Actually, they have to. There is no other way, unless one breaks the laws of physics.


But the tests do not show other effects. Therefore your last sentence should actually read: "The upshot of this is that the tests are designed to minimise the effects of the size of the sensor to the lowest point made possible by the laws of physics."


Anyway, I do not think that discussion is leading us anywhere. I was simply pointing out that DXO, whatever they are testing, do not tell the full story. Can we just simply agree that 2 camera-lens combination with the same DXO score may give very different results in practice?
Title: Re: DXO tests
Post by: bjanes on November 22, 2014, 01:52:00 pm
I would say the size of the lens is the primary determinant of the number of photons captured. Following that, it is easier to project a large lens onto a large sensor really well, than squeezing the light onto a small sensor without wave front errors.

Which is why I chose full frame 35 for wide angles.

That is quite correct. Roger Clark (http://www.clarkvision.com/articles/low.light.photography.and.f-ratios/index.html) explains this in some detail using the concept of Etendue (sounds like a French word, maintaining the relationship with DXO). The larger sensor goes with a larger lens.

Bill
Title: Re: DXO tests
Post by: Theodoros on November 22, 2014, 04:20:54 pm
SNR and dynamic range are not of any importance to most photographers? When DXO gets around to testing the new CMOS MFDB sensors they will jump to the head of the sensor ratings and you might change your mind regarding the DXO tests.

Regards,

Bill
They don't measure SNR or DR as relevant to PHOTO-graphy B ill... They measure it as unprocessed captured data... this is irrelevant to PHOTOgraphy or for many to understand.... They have first to understand what a photo-grapher is...
Title: Re: DXO tests
Post by: Theodoros on November 22, 2014, 04:37:26 pm
in other words (do I have your attention B ill?)... to a photo-grapher, what SNR or DR a sensor captures is useless... It's what's left to him after processing that he cares (the latitude that affects the difference between capture and processing and what's left for the PHOTO-graph).... But that's a photo-grapher's way of thinking! ...understand B ill? You better, because there won't be anymore posts from me... No reason to argue on simple logic... that only concerns photo-graphers.
Title: Re: DXO tests
Post by: Fine_Art on November 22, 2014, 05:07:10 pm
That is quite correct. Roger Clark (http://www.clarkvision.com/articles/low.light.photography.and.f-ratios/index.html) explains this in some detail using the concept of Etendue (sounds like a French word, maintaining the relationship with DXO). The larger sensor goes with a larger lens.

Bill

Thanks for the link Bill, it is a good article as usual on that site. Roger is a gifted explainer.

That reminds me of a recent discussion in another thread about f ratio being the base constant vs angle of view with lens diameter, for determining a good pixel size.

Title: Re: DXO tests
Post by: Fine_Art on November 22, 2014, 05:14:22 pm
in other words (do I have your attention B ill?)... to a photo-grapher, what SNR or DR a sensor captures is useless... It's what's left to him after processing that he cares (the latitude that affects the difference between capture and processing and what's left for the PHOTO-graph).... But that's a photo-grapher's way of thinking! ...understand B ill? You better, because there won't be anymore posts from me... No reason to argue on simple logic... that only concerns photo-graphers.


Maybe you should change your name to Animus.
Title: Re: DXO tests
Post by: BernardLanguillier on November 22, 2014, 08:18:51 pm
That is quite correct. Roger Clark (http://www.clarkvision.com/articles/low.light.photography.and.f-ratios/index.html) explains this in some detail using the concept of Etendue (sounds like a French word, maintaining the relationship with DXO). The larger sensor goes with a larger lens.

Indeed! What matters is the T stop of the lens reduced by a factor reflecting both the average angle of the light as it reaches the sensor (typically favoring smaller sensors and longer flange distances) and a technological factor representing the ability of the sensels to effectively capture photon energy at those angles (larger size of actual light collecting area should help here).

In other words, the sensor size alone is one factor among others, the lens is key as well.

Cheers,
Bernard
Title: Re: DXO tests
Post by: ErikKaffehr on November 24, 2014, 05:26:51 pm
Hi,

My interpretation is that the usual DxO-mark is essentially a sensor evaluation. What is measured essentially:

SNR 18% (Signal Noise Ratio) for 18% grey card . This is essentially shot noise. Large pixels have large full well capacity (FWC) so they have SNR. But, any image will be resized for printing so the electrons collected in the sensor wild be redistributed in printing. So, what matters is that a sensor may collect say 24 * 10^6 * 6* 10^4  = 1.44*10^12 photons (24 MP at 60000 FWC), would the sensor have twice the pixel size it would have an FWC of 120000 electron charges but the total number of photons collected would still be around 1.44*10^12.

Printing the two images on paper of any size, noise levels would be the same.

Doubling the size of he sensor would increase SNR by 41%, and that is totally independent of print size.

Dynamic Range is a bit different. The term hat DxO uses is the technical definition, DR when SNR=1. In practice, this figure is FWC/(readout noise). Now, readout noise is independent of pixel size. So doubling pixel size will double DR/pixel. But with half the pixel size we would have twice the DR. But noise adds in quadrature but readout noise is fixed. So doubling pixel size will have an 41% advantage in DR in print. That correspond by and large to half an EV.

The DxO-mark sensor doesn't care about resolution, just noise.

There is also a DxO-mark for lenses. One of their measures is effective MPixels. The description they give is very unclear. But it is very probably a variant of SQF, Subjective Quality Factor. The writing indicates that the contrast sensivity function of the eye is used and mention is made of industry cooperation.

SQF is based on research by Ed Granger (?) of Kodak and is widely used. A good definition is given here. There are some issues with SQF, mainly that is very dependent on sharpening, which of course relates to human vision being very much affected by sharpening. So, SQF is a figure for measuring perceived sharpness/rendition based on significant research but in no way fool proof.

This article by Dr. Hubert Nasse of Zeiss discusses SQF quite a bit (http://www.smt.zeiss.com/C12567A8003B8B6F/EmbedTitelIntern/CLN_31_MTF_en/$File/CLN_MTF_Kurven_2_en.pdf), but also gives some example(s) where visual impression differs from SQF.

To round of these remarks, let us look at a comparison of the Leica M9 and the Phase One P45+. Both use Kodak sensors wit similar pixel size.
For DR at pixel size DxO-mark gives 11.75 EV (at 48 ISO) for the P45+ and 11.04 EV (at 144 ISO)for the Leica M9. This difference is significant, but note that at 144 ISO the Leica gets a third of the exposure. So hat figure is pretty good.

Looking at "print mode", that is normalised, DR on the P45+ is 12.9 EV but on the Leica M9 11.63. So the 0.71 EV advantage of the P45+ has increased to 1.27 EV when the sensor size was taken into account.

Now, lets look at the Phase One P65+ instead. It has a technology called Sensor+ which bundles 4 pixels into one in hardware, reducing resolution to  fourth.

Looking at "screen resolution" in DxO-mark we can see the effect on SNR as a hump at 800 ISO. But "in the print version" that is normalised to print size we see no hump in SNR. This is expected as SNR essentially is only affected by sensor size.

Looking at "dynamic range" at print size we still see the effect of Sensor+ as a hump at 800 ISO, but at less amplitude compared to the pixel sized graph.

To sum it up, the normalisation that DxO makes takes sensor size very much into account.

Best regards
Erik

That is true, mostly.

Up to a point, yes. But I said in my first message that the DXO tests "will show indirect differences (e.g. lower noise coming using larger sensels).".  Actually, they have to. There is no other way, unless one breaks the laws of physics.


But the tests do not show other effects. Therefore your last sentence should actually read: "The upshot of this is that the tests are designed to minimise the effects of the size of the sensor to the lowest point made possible by the laws of physics."


Anyway, I do not think that discussion is leading us anywhere. I was simply pointing out that DXO, whatever they are testing, do not tell the full story. Can we just simply agree that 2 camera-lens combination with the same DXO score may give very different results in practice?
Title: Re: DXO tests (lens ratings)
Post by: ErikKaffehr on November 24, 2014, 05:40:43 pm
Unfortunately, DxO has not tested MF lenses. But, they have tested lenses on different cameras. The example below shows two different lenses on three different cameras.

The Nikon 50/1.4D performs almost as well on the D810 as the Otus does on the Canon 5DIII

If the Otus is used with the Nikon 7100 APS-C camera much performance is lost, but it still is about the same level as on the larger Canon 5DIII sensor.

This really indicates that DxO-mark lens takes both lens sharpness and sensor resolution into account.

Best regards
Erik
Title: Re: DXO tests
Post by: Doug Peterson on November 24, 2014, 05:42:06 pm
SNR and dynamic range are not of any importance to most photographers? When DXO gets around to testing the new CMOS MFDB sensors they will jump to the head of the sensor ratings and you might change your mind regarding the DXO tests.

Photographically usable dynamic range in a real-world workflow matters a LOT to many types of photographers. dXo does not test for this. They test DR in an engineering sense.

The effect of raw processing (quite important given the manufacturer software often gets the best result out of their file), aesthetics, tonal smoothness, quality (not just quantity) and look of grain/noise, and color accuracy in deep shadow transitions are all ignored.

If/when DXO publishes the IQ150 (https://digitaltransitions.com/blog/dt-blog/phase-one-iq150)/IQ250/Credo50 I'm quite sure it will score very well. That will not change my opinion that the best way to see how a camera handles in the real world, is to shoot it in the real world, and not in a lab setting.
Title: Re: DXO tests
Post by: ErikKaffehr on November 24, 2014, 07:07:50 pm
Hi,

DxO actually measures photographic DR, it is called "tonal range" in their tests. The DR figure in the DxO tests essentially say how far shadow detail can be pushed before noise becomes visible.

There is very little doubt the IQ-250 will make it to the top of marks, as doubling sensor size usually gives around 10 points.

DxO-mark measures data before raw processing, AFAIK.

DxO-mark is much influenced by the DR-figure which is shadow detail and high ISO capability as these are two of three ratings going into the evaluation. The third is colour depth, where MFD usually rank high.

Best regards
Erik

Photographically usable dynamic range in a real-world workflow matters a LOT to many types of photographers. dXo does not test for this. They test DR in an engineering sense.

The effect of raw processing (quite important given the manufacturer software often gets the best result out of their file), aesthetics, tonal smoothness, quality (not just quantity) and look of grain/noise, and color accuracy in deep shadow transitions are all ignored.

If/when DXO publishes the IQ150 (https://digitaltransitions.com/blog/dt-blog/phase-one-iq150)/IQ250/Credo50 I'm quite sure it will score very well. That will not change my opinion that the best way to see how a camera handles in the real world, is to shoot it in the real world, and not in a lab setting.
Title: Re: DXO tests
Post by: bjanes on November 24, 2014, 09:36:37 pm
Photographically usable dynamic range in a real-world workflow matters a LOT to many types of photographers. dXo does not test for this. They test DR in an engineering sense.

The effect of raw processing (quite important given the manufacturer software often gets the best result out of their file), aesthetics, tonal smoothness, quality (not just quantity) and look of grain/noise, and color accuracy in deep shadow transitions are all ignored.

If/when DXO publishes the IQ150 (https://digitaltransitions.com/blog/dt-blog/phase-one-iq150)/IQ250/Credo50 I'm quite sure it will score very well. That will not change my opinion that the best way to see how a camera handles in the real world, is to shoot it in the real world, and not in a lab setting.

From the DXO full SNR data one can calculate the DR at any desired noise floor as outlined by Emil Martinec (http://www.luminous-landscape.com/forum/index.php?topic=42158.0). The noise floor that DXO uses for DR is 1:1 or 0 dB. A SNR of 10:1 or 20 dB is more realistic for photographic DR as suggested by Jim Kasson (who posts here frequently and is very knowledgeable in these matters).

I calculated the DR at this noise floor at base ISO for the Nikon D800e and Phase One IQ180 and the results are shown below for both Screen DR and Print DR as used in DXO nomenclature. The DR for 0 dB is also calculated and corresponds closely to the results posted by DXO. Using Print DR as a criterion, the D800e has a 0.81 stop advantage at a 0 dB noise floor and at 20 dB it still has a 0.25 stop advantage. At 18% saturation shot noise predominates and the IQ 180 has better SNR. When CMOS MFDB backs with their large sensor area and presumably low read noise, the MFDB will come out on top.

Bill
Title: Re: DXO tests
Post by: Ken R on November 25, 2014, 06:06:30 am
From the DXO full SNR data one can calculate the DR at any desired noise floor as outlined by Emil Martinec (http://www.luminous-landscape.com/forum/index.php?topic=42158.0). The noise floor that DXO uses for DR is 1:1 or 0 dB. A SNR of 10:1 or 20 dB is more realistic for photographic DR as suggested by Jim Kasson (who posts here frequently and is very knowledgeable in these matters).

I calculated the DR at this noise floor at base ISO for the Nikon D800e and Phase One IQ180 and the results are shown below for both Screen DR and Print DR as used in DXO nomenclature. The DR for 0 dB is also calculated and corresponds closely to the results posted by DXO. Using Print DR as a criterion, the D800e has a 0.81 stop advantage at a 0 dB noise floor and at 20 dB it still has a 0.25 stop advantage. At 18% saturation shot noise predominates and the IQ 180 has better SNR. When CMOS MFDB backs with their large sensor area and presumably low read noise, the MFDB will come out on top.

Bill

Isn't the base iso of the IQ180 iso 35? In the DxO test it shows iso 100 as the lowest iso tested.
Title: Re: DXO tests (lens ratings)
Post by: bjanes on November 25, 2014, 06:28:28 am
Unfortunately, DxO has not tested MF lenses. But, they have tested lenses on different cameras. The example below shows two different lenses on three different cameras.

The Nikon 50/1.4D performs almost as well on the D810 as the Otus does on the Canon 5DIII

Erik,

Good work in pointing out these results. Although I have some reservations regarding the megapixel ratings that DXO assigns, the results that you cite confirm Jim Kasson's quiver plots previously alluded to and his and Bart's statements that, provided reasonably good lens performance, increasing sensor resolution is better route to improving overall system resolution. The Nikon 50/1.4 is not a particularly good lens, but using it on the D810 gives good results and the D810 will improve system resolution with all one's other lenses.

Bill
Title: Re: DXO tests (lens ratings)
Post by: Chris Livsey on November 25, 2014, 07:44:10 am
The Nikon 50/1.4 is not a particularly good lens, but using it on the D810 gives good results and the D810 will improve system resolution with all one's other lenses.
Bill

Would the A7R not be just as good at improving system resolution and with a wider range of one's other lenses?
The resolution is the same, do other sensor characteristics, internal processing etc take the A7R below the 810 by a significant margin?

(Noting the A7II announcement with no indication as yet if the RAW "tampering" has been changed would this change the equation?)

Title: Re: DXO tests (lens ratings)
Post by: ErikKaffehr on November 25, 2014, 08:14:20 am
Hi,

Regarding raw "tampering" I don't know what you mean. There is two kinds of raw compression on the A7r (and other Sony cameras), one is essentially a tone curve that is inverted in raw conversion. The itention is to put more smaples in the darks and fewer in the higlights. My view is it is quite OK.

The other is a delta type compression that can introduce artefacts. The only clears samples I have seen of that is artefacts on star tracs. I don't like it a bit!

There may be other "tampering", raw files are sometimes half cooked. No comments on that.

Neither of the above relates to resolution/sharpness. The D810 and the Sony sensor should be quite similar. The A7r has issues whit shutter cause vibrations in shutter speed range like 1/15 - 1/250, though.

Best regards
Erik



Would the A7R not be just as good at improving system resolution and with a wider range of one's other lenses?
The resolution is the same, do other sensor characteristics, internal processing etc take the A7R below the 810 by a significant margin?

(Noting the A7II announcement with no indication as yet if the RAW "tampering" has been changed would this change the equation?)


Title: Re: DXO tests (lens ratings)
Post by: Chris Livsey on November 25, 2014, 08:21:53 am

Regarding raw "tampering" I don't know what you mean.
The other is a delta type compression that can introduce artefacts.

Best regards
Erik
Thank you Erik, it was the compression I referred to, the word escaped me momentarily. Jim Kasson has demonstrated it http://blog.kasson.com/?p=4838 although as you say the effect on real world images may be limited at worst.
As I said real word usage with the shutter limitations on the Sony and the superior auto focus on the Nikon (negated if one is mounting manual focus lenses on each) and of course a roughly 2:1 price difference  ;D need to be factored in.
I will look if they have but surely given the Sony versatility DxO could test the same lens on both those bodies?
Title: Re: DXO tests
Post by: Doug Peterson on November 25, 2014, 09:28:00 am
Quantity of noise ≠ Quality of noise.
SNR ≠ Aesthetics of Shadows.

Using a DXO test to judge cameras is sort of like using a metronome to measure an orchestral performance of Mozart .
Title: Re: DXO tests
Post by: Bart_van_der_Wolf on November 25, 2014, 09:50:52 am
Quantity of noise ≠ Quality of noise.
SNR ≠ Aesthetics of Shadows.

Hi Doug,

While true, the noise characteristics tested by DxO are sensor noise, which is mostly (Poisson) shot noise and read-noise (and some mix of PRNU noise). How a Raw converter deals with that during demosaicing and noise reduction is a different matter. Not unimportant at all, but different.

DxO tests lens and camera physics, because they need that data for their type of Raw conversion and aberration/distortion correction. Others add their specific flavor of dealing with those issues.

Cheers,
Bart
Title: Re: DXO tests
Post by: bjanes on November 25, 2014, 10:34:36 am
Isn't the base iso of the IQ180 iso 35? In the DxO test it shows iso 100 as the lowest iso tested.

Ken,

This is where DXO testing is helpful. Manufacturers can rate the ISO of their sensors as they see fit using the ISO REI (recommended exposure index). See here (http://en.wikipedia.org/wiki/Film_speed#Digital_camera_ISO_speed_and_exposure_index) for details. They often rate the sensor lower than the saturation standard would indicate so as to provide highlight headroom and avoid blown highlights. 0.5 EV is often allowed for highlight headroom. As the DXO measurements show, when the IQ180 ISO is set to 100, the measured ISO is only 29, allowing 1.79 stops of highlight headroom. Effectively, this is underexposure.

At camera settings of 50 and 35 the measured ISO is 29, the same as with the camera setting of 100. This is relative overexposure as compared to ISO 100. This indicates that the amplifier gain is not changed when one goes from a camera setting of ISO 100 to 35.

Bill
Title: Re: DXO tests
Post by: ErikKaffehr on November 25, 2014, 11:35:32 am
Hi Doug,

DxO has not yet tested any of the 44x33/50 MP Sony sensor equipped cameras yet. But, we have a good comparison between the IQ-250 and the IQ-260 and IQ-280 due to your own library shots. Those shots were commendably good, by the way.

Paul Caldwell started a thread a thread on those shots and posted the pair of images below (http://www.luminous-landscape.com/forum/index.php?topic=87187.0), IQ-250 on the right:
(http://www.luminous-landscape.com/forum/index.php?action=dlattach;topic=87187.0;attach=97706;image)

These images are pretty consistent with what you would expect from a Sony sensor (similar to , but a bit larger than the one used in the Nikon D810 etc). It is also quite consistent with DxO data indicating significant readout noise in the shadows in the shadows of the older P and IQ-series backs. My own developments were similar, but Paul is a long time Phase One user, which I am not.

Paul has been quite impressed by IQ-250, enough to want to 'cross grade' from his 8 months old IQ-260 to the new IQ-250, in spite of the 1.3X crop factor. But he is not willing pay the  $21k "cross grade fee". The main reason is cleaner shadows on his Nikon compared to the IQ-260 (http://www.getdpi.com/forum/medium-format-systems-digital-backs/52479-iq250-iq260-residual-value.html#post612185).

Links here:
http://www.getdpi.com/forum/611862-post21.html
http://www.getdpi.com/forum/612068-post33.html
http://www.getdpi.com/forum/medium-format-systems-digital-backs/52479-iq250-iq260-residual-value.html#post612185

So, it seems that both your examples and Paul's experience indicates that there is quite a bit of reality in the DxO figures.

Best regards
Erik
Quantity of noise ≠ Quality of noise.
SNR ≠ Aesthetics of Shadows.

Using a DXO test to judge cameras is sort of like using a metronome to measure an orchestral performance of Mozart .
Title: Re: DXO tests
Post by: jerome_m on November 25, 2014, 03:05:55 pm
I am really losing interest in these discussion and actually regret to have started this thread.

I said in my first message that the DXO tests "will show indirect differences (e.g. lower noise coming using larger sensels)".  I am not arguing about that.

But the tests do not show other effects, because they are designed to ignore these effects. Erik, for example: you use both a MF camera and a 24x36 A99. Are you really sure that you never noticed any other effect beyond increased dynamic range and lower noise?
Title: Re: DXO tests
Post by: ErikKaffehr on November 25, 2014, 03:55:46 pm
Hi Jerome,

That is an interesting question to which I have no answer. You sometimes made the point that the way I work I equalise things, and that is quite true. I normally shoot medium apertures, and for limited DoF I rather use a long telephoto on the DSLR where I can focus using magnified live view. I see some disadvantage in DR to MFD but I don't see it as very limiting. Colour rendition is pretty close after I made my own DNG profiles. In general, what I see (based on my equipment) is that:


I do see a very significant difference in resolution/MTF stuff, on screen, with my 24 MP pixels uprezzed to the 39 MP of the P45+. In A2 prints I see no difference. I have done some limited experiments with different people and they are not conclusive. Perhaps I may have the impression that if we are comparing details, little difference is noticed in A2-size. Going up to A1 I would say the difference starts to be clear. Looking on screen there is a large difference. But, at least in one test folks easily detected which was which.

Something like this:

- The left one is better, is that with the blad? (Correct, Elderly observer partially color blind).
- That one is the MFD, it has higher contrast. (Correct, but I don't feel it is higher contrast, young guy)
- That one is MFD, it is a better crop, you have moved the camera. You don't think I notice. Correct, camera was moved 10 cm at bout 5 m distance to subject. Elderly observer.

On the other hand:

I test he very same image with a young guy, with perfectly good vision and he cannot see the difference. Most other test I have done with a trained observer, little difference was noted.

It is very hard to make to similar images, with two different equipments. Lab conditions, it is easy. But in real world lighting changes, graduated filters are used in postprocessing, white point may differ.

My finding after 17 months is that there is significant difference on my P45+ (39 MP) and Sony Alpha 99 (24 MP) in sharpness, but, I cannot see that  difference in A2 size prints. I have seen examples where other observers would easily detect differences. Far to few experiments to draw conclusions.

Here are some images I shot on a two week trip:

This is Hasselblad P45+, compound of two images.
(http://echophoto.smugmug.com/Themes/Berg-dal-och-vatten/i-XxmZfzs/0/X3/20140612-CF045214-Edit-X3.jpg)
While I was shooting the above image I made this free hand shot on the Sony SLT 99 (snapshot):
(http://echophoto.smugmug.com/Themes/Berg-dal-och-vatten/i-2r3h64K/0/X3/20140612-_DSC4611-X3.jpg)
Later in the evening we went for a walk, I just carried my Sony Alpha 77 (APS-C), two lenses in my pocket, tripod and an ND filter:
(http://echophoto.smugmug.com/Themes/Berg-dal-och-vatten/i-SPHjgJF/0/X3/20140604-_DSC6095-X3.jpg)
The next image was shot just after sunrise, using the Sony Alpha 99 (full frame 135). Waiting an hour for good light.
(http://echophoto.smugmug.com/Themes/Berg-dal-och-vatten/i-VxJWrkr/0/X3/20140604-_DSC4093-X3.jpg)
This one was shot on APS-C using a Sigma fisheye lens at f/16 for DoF (handheld):
(http://echophoto.smugmug.com/Themes/Berg-dal-och-vatten/i-VqG4Mjb/0/X3/20140607-_DSC6131-X3.jpg)
This was Sony Alpha 99 SLT (Full frame 135)
(http://echophoto.smugmug.com/Themes/Berg-dal-och-vatten/i-L999Mkt/0/X3/20140613-_DSC4663-X3.jpg)
And this one was Hasselblad P45+
(http://echophoto.smugmug.com/Themes/Berg-dal-och-vatten/i-qbgX2tt/0/X3/20140613-CF045257-X3.jpg)
I am perfectly sure all of these will make perfectly good A2 images.



Getting back to the DxO tests, what I see is that:

Best regards
Erik

I am really losing interest in these discussion and actually regret to have started this thread.

I said in my first message that the DXO tests "will show indirect differences (e.g. lower noise coming using larger sensels)".  I am not arguing about that.

But the tests do not show other effects, because they are designed to ignore these effects. Erik, for example: you use both a MF camera and a 24x36 A99. Are you really sure that you never noticed any other effect beyond increased dynamic range and lower noise?
Title: Re: DXO tests
Post by: Theodoros on November 25, 2014, 05:11:26 pm
From the DXO full SNR data one can calculate the DR at any desired noise floor as outlined by Emil Martinec (http://www.luminous-landscape.com/forum/index.php?topic=42158.0). The noise floor that DXO uses for DR is 1:1 or 0 dB. A SNR of 10:1 or 20 dB is more realistic for photographic DR as suggested by Jim Kasson (who posts here frequently and is very knowledgeable in these matters).

I calculated the DR at this noise floor at base ISO for the Nikon D800e and Phase One IQ180 and the results are shown below for both Screen DR and Print DR as used in DXO nomenclature. The DR for 0 dB is also calculated and corresponds closely to the results posted by DXO. Using Print DR as a criterion, the D800e has a 0.81 stop advantage at a 0 dB noise floor and at 20 dB it still has a 0.25 stop advantage. At 18% saturation shot noise predominates and the IQ 180 has better SNR. When CMOS MFDB backs with their large sensor area and presumably low read noise, the MFDB will come out on top.

Bill
Yeah... but what you post IS IRRELEVANT to what you Quote on... :D
Title: Re: DXO tests
Post by: Theodoros on November 25, 2014, 05:36:58 pm
Photographically usable dynamic range in a real-world workflow matters a LOT to many types of photographers. dXo does not test for this. They test DR in an engineering sense.

The effect of raw processing (quite important given the manufacturer software often gets the best result out of their file), aesthetics, tonal smoothness, quality (not just quantity) and look of grain/noise, and color accuracy in deep shadow transitions are all ignored.

If/when DXO publishes the IQ150 (https://digitaltransitions.com/blog/dt-blog/phase-one-iq150)/IQ250/Credo50 I'm quite sure it will score very well. That will not change my opinion that the best way to see how a camera handles in the real world, is to shoot it in the real world, and not in a lab setting.
In other words... When a sensor captures data... it is DATA it captures... not a PHOTO-GRAPH... A photo-graph though... is what a photo-grapher ONLY cares about (otherwise he's not a photo-grapher)... AND... what a sensor captures as DR or noise or colour or contrast,... IS IRRELEVANT to what the photo-graph looks like...

Additionally... there is NO GUARANTY whatsoever, that the processing latitude is relevant to the captured amount of latitude there are sensors that capture higher DR than other but print less and other sensors that seem to capture less... but print ALL they capture... and (of course) processing media... does make A LOT of difference in both the sensor's captured data presentation, as well as the data processing latitude...

In other words... B ill is out of subject (as usual) and Erik's thread, can only lead to some more ignorants added (since those that are not... won't bother...). In fact, I won't bother anymore either...
Title: Re: DXO tests
Post by: synn on November 25, 2014, 08:21:59 pm
Seriously you guys...
Title: Re: DXO tests
Post by: FMueller on November 25, 2014, 09:06:43 pm
Seriously you guys...

+1

Title: Re: DXO tests
Post by: Ken R on November 25, 2014, 09:17:32 pm
Seriously you guys...

+2

Usual suspects always appear and post the same images over and over and try to hammer in their same viewpoints over and over and argue with the same people...over and over... This happens in all forums but in this one it seems it is only a handful of individuals...

Maybe there should be a camera engineering forum...
Title: Re: DXO tests
Post by: Joe Towner on November 25, 2014, 10:54:09 pm
LensRentals got a couple of really fancy camera testers - http://www.lensrentals.com/blog/2014/05/introducing-the-optical-bench

I would love to see a number of MF lenses pass through this, as it takes the sensor out of the testing process (unlike DXO which is dependent on the camera).  I think given a proper sample size, we all could learn a lot about the optics and how well certain lenses work.
Title: Re: DXO tests
Post by: ErikKaffehr on November 26, 2014, 12:24:21 am
Hi,

I agree with what you say. Hasselblad and Rollei publishes MTF tests of all their lenses, Schneider and Rodenstock publishes MTF data, too.

Unfortunately, different firms use different presentation of data, so they can be hard to compare. Schneider often has 15, 30 and 60 lp/mm while Hasselblad uses 10, 20 and 40 lp/mm.

Best regards
Erik




LensRentals got a couple of really fancy camera testers - http://www.lensrentals.com/blog/2014/05/introducing-the-optical-bench

I would love to see a number of MF lenses pass through this, as it takes the sensor out of the testing process (unlike DXO which is dependent on the camera).  I think given a proper sample size, we all could learn a lot about the optics and how well certain lenses work.
Title: Re: DXO tests
Post by: jerome_m on November 26, 2014, 02:16:01 am
Quote
Erik, for example: you use both a MF camera and a 24x36 A99. Are you really sure that you never noticed any other effect beyond increased dynamic range and lower noise?

That is an interesting question to which I have no answer. You sometimes made the point that the way I work I equalise things, and that is quite true. I normally shoot medium apertures, and for limited DoF I rather use a long telephoto on the DSLR where I can focus using magnified live view. I see some disadvantage in DR to MFD but I don't see it as very limiting. Colour rendition is pretty close after I made my own DNG profiles.

So: what about trying to shoot differently for a month or so? It will neither kill you nor break the camera if you use different apertures, let the exposure wander a bit to the left or drop the color chart. You may learn a thing or two.
Title: Re: DXO tests
Post by: ErikKaffehr on November 26, 2014, 02:58:05 am
Hi,

You started a technical thread about DxO. I don't think my choice of subject or reluctance to intentional underexposure belong to the thread. You asked me some question, and I did answer those questions as correctly and politely as I could.

Best regards
Erik


So: what about trying to shoot differently for a month or so? It will neither kill you nor break the camera if you use different apertures, let the exposure wander a bit to the left or drop the color chart. You may learn a thing or two.

Title: Re: DXO tests
Post by: jerome_m on November 26, 2014, 03:18:02 am
You started a technical thread about DxO. I don't think my choice of subject or reluctance to intentional underexposure belong to the thread. You asked me some question, and I did answer those questions as correctly and politely as I could.

Are you implying that my last question is incorrect or impolite?
Title: Re: DXO tests
Post by: hjulenissen on November 26, 2014, 03:20:18 am
Photographically usable dynamic range in a real-world workflow matters a LOT to many types of photographers. dXo does not test for this. They test DR in an engineering sense.
Flash card manufacturers tests write speeds in an engineering sense (lets hope that it is not marketing guys). Thankfully, we photographers do not live in an alternate dimension, and the results of those engineering measurements can be directly related to concepts that matters to many photographers.

The idea that art and engineering live in unconnected spheres seems counterproductive to both art and engineering. It is also ignorant of the brilliant people and work that has resulted somewhere in-between.
Quote
The effect of raw processing (quite important given the manufacturer software often gets the best result out of their file), aesthetics, tonal smoothness, quality (not just quantity) and look of grain/noise, and color accuracy in deep shadow transitions are all ignored.
Many photographers use a 3rd party raw processor that will be remain constant*) no matter what camera they use.

Image quality cannot be totally described by 3 or 4 parameters. Thus, there may probably be visible differences (for some scenes) even for two cameras with similar dxo-type ratings. It seems that camera sensors are in some ways a lot simpler than some people make them out to be, especially if one leaves out the subjectivity present in raw development. Camera sensors (to my knowledge) does not have less accurate colors in the shadows. Raw converters may, if that is chosen by the implementer, possibly to work around some limitation in the sensor.
Quote
That will not change my opinion that the best way to see how a camera handles in the real world, is to shoot it in the real world, and not in a lab setting.
Of course. I would not spend $100.000 (or my career) on a piece of equipment without spending some time to know if the investement was good. That includes actually using the thing extensively.

Thing is, I have a finite amount of time and a finite ability to do precise comparisions. If there is a flaw in a camera that will only show up once every 10000 images, chances are that I won't figure out until I make the purchase. If someone are willing to do some relevant preliminary testing of a large number of cameras, I will try to comprehend their results and incorporate their conclusions into my own process. Sticking my fingers in my ears (eyes?) and singing "lala" does seem ignorant. Offering well thought-out critique of DXO (and everything else) is a good thing, the kind of thing that has helped us progress.

-h
Title: Re: DXO tests
Post by: ErikKaffehr on November 26, 2014, 04:19:19 am
Hi Jerome,

No, I don't imply that your question was impolite. What I said that I tried to answer your question (about my perception of the P45+ vs. the Sony) as accurately and politely as I could. I put some effort in both accuracy (finding and checking references) and wording.

On the other hand, I would also say that suggesting that I would spend time to different type shooting than I do, is quite a bit irrelevant on a thread discussing DxO tests, which incidentally was started by yourself.

Best regards
Erik

Are you implying that my last question is incorrect or impolite?
Title: Re: DXO tests
Post by: jerome_m on November 26, 2014, 05:29:16 am
On the other hand, I would also say that suggesting that I would spend time to different type shooting than I do, is quite a bit irrelevant on a thread discussing DxO tests, which incidentally was started by yourself.

I did not suggest that you change your practice for ever, I only suggested that as a test.

As to relevance, I can say the following: when I started this thread, I argued that some differences due to sensor size are not tested by DXO, because of the particular assumptions they made for their test. The way you work just happens to correspond to these assumptions and, consequently, you do not see differences that are equalised by both your practice and the DXO test suite. I suggested that you try a different practice a few times to be able to see these differences.

As a side note, the reason why these discussions go nowhere is that you and a few others present that particular way of measuring and testing as an absolute metric. It is not. I, for example, have little use for dynamic range (it is generally sufficient for me in modern cameras), shadow noise level or colour fidelity. I rarely have use for optical sharpness, as measured by flat resolution charts at 1-4m distance. I may have use for optical sharpness, as measured at infinite distance (and I presented some tests of mine for that).

For example, I have use for rendering of out of focus subjects, homogeneity of the rendering across the field, consistency of colour under different illuminants and plenty of other factors which are generally absent from DXO tests.
Title: Re: DXO tests
Post by: ErikKaffehr on November 26, 2014, 05:56:34 am
Hi Jerome,

DxO lens test show homogenity of rendering across the field, and they publish SMI for two illuminants, so they actually cover that ground quite a bit.

They don't show bokeh, the Photozone tests cover that area a bit more. DxO doesn't evaluate sensor colour reproduction i spiky spectra.

I will do some more lens testing as I added another Zeiss lens to my Hasselblad kit, the Planar 100/3.5 which is a famously sharp lens. Plan to do something similar to your shots. As a side note: i have posted sample images from all my Hasselblad (V-series) lenses here: http://echophoto.dnsalias.net/ekr/Articles/Shoots/BernardSamples/

The screenshots below are from DxO reports on one of the Hasselbalds (50 MP back) and comparing Zeiss 24-70/2.8 and Sigma 24-105/4 Art. I am a bit interested in buying the Sigma 24-105/4 as the Zeiss 24-70/2.8 is lacking a bit at the long end.

Best regards
Erik



For example, I have use for rendering of out of focus subjects, homogeneity of the rendering across the field, consistency of colour under different illuminants and plenty of other factors which are generally absent from DXO tests.
Title: Re: DXO tests
Post by: jerome_m on November 26, 2014, 08:17:02 am
DxO lens test show homogeneity of rendering across the field

That data would only be useful for me if I reproduced paintings. I am not interested in homogeneity a 1-4m, which is what you get from test charts. I am interested about much larger distances, and the results are different. I am also interested in homogeneity of the bokeh, which they do not test at all (e.g. swirly bokeh or cateye effect).


Quote
They publish SMI for two illuminants, so they actually cover that ground quite a bit.

Not at all. All cameras work reasonably well under CIE-D50 and CIE-A. I would like to know what happens under non-standard yet common illuminants: overcast daylight, fluorescent or led lightning for example. I know the results will be poor for fluorescent, but I also know that some camera fare much better than others.


Quote
I am a bit interested in buying the Sigma 24-105/4 as the Zeiss 24-70/2.8 is lacking a bit at the long end.

In practical use, I find the SAL-2470F28Z to be an excellent lens at 70mm. I am more irritated by its propensity to show two well delimited unsharp triangles in the lowest corners when photographying people standing on grass at 24mm. Note that what irritates me is that these triangles are well delimited: the Nikon 24-70 has corners which are just as poor, but the change is more gradual. That is less noticeable.
Title: Re: DXO tests
Post by: Ken R on November 26, 2014, 09:56:05 am
Two things regarding Dxo (and other reviewers/testers):

#1: Tests/Reviews should disclose at which focus distance the lenses were tested. Preferably it would be nice to have tests that cover at least two points, one at infinity and other at a medium distance typical or one akin to what that particular lens most common use is. (One reason why Lens Rentals have invested in several lens testing systems is to do just that)

#2: The DxO sensor testing data and overall rating number give a really good idea of how good a sensor is compared to others specially when the differences are considerable. A few points here and there are almost imperceptible in real world use and when comparing two sensors or cameras with close enough rating I would not use the rating as the determining factor in choosing between them. But having used a lot of cameras that are in the DxO database of tests and looking at how DxO has rated them and compared their real world performance by using them I somewhat agree with the DxO ratings. Not saying the ratings tell the whole story but they are valuable. Specially for comparing the dynamic range of cameras. Color is much more difficult since it is extremely dependent on processing and a bit more subjective. Again, IMHO.

Title: Re: DXO tests
Post by: ErikKaffehr on November 27, 2014, 02:06:49 am
Hi,

Some good points from Ken R!

Here are two articles from Tim Ashly who used to have a very nice blog on the equipment he uses, and that was a wide selection of stuff he has owned.

The first one is on the DxO-mark in general:

http://tashley1.zenfolio.com/blog/2013/3/whats-your-doh-mark-score

The other one on the lens tests:

http://tashley1.zenfolio.com/blog/2013/3/dxo-mark-lenses-on-d800---my-real-world-response

With regard to the lens tests he makes several very good points. I would like to comment on two of them field curvature and focus shift.

Field curvature varies with focusing distance. A good practice is to test lenses at 50 times focal length. A large target is needed for that. Modern lenses often have floating elements (or groups) that compensate field curvature at different distances.

Field curvature would show up in the DxO test as loss of sharpness across the field, but it cannot be told apart from other aberration from the benchmark data.

Field curvature is interesting, because a lens with curved field can have very good sharpness at the point of focus but still perform bad on flat targets. The Zeiss macro Planar 120/4 (Hasselblad version) is a good example of that. See MTF data below, pretty ugly at infinity but really good at close up, which is it's intended range. But, would it be focused at a point of axis at infinity that point would still be in good sharpness, but centrum would be out focus. Stopping down to say f/11 DoF hides curvature and the lens looks pretty good.

Zeiss did produce a more complex design for the Contax, Apo Macro Planar 120/4, that lens had a floating group keeping field curvature at bay. For the Hasselblad V, Zeiss has three different Planar lenses. For long distance work Hasselblad recommends the 100/3.5 while for short distance they recommend the Macro Planar 120/4. They say that if the subject area is larger than 1 sqm (square meter) use the 100/3.5 else the 120/4, at least for reproduction type photography.

The other issue is focus shift, and generally a problem with large aperture lenses. When stopping down the point of focus moves. So if focusing is at full aperture slight misfocus will arise when stopped down. Photozone has a very good test of this effect.

Best regards
Erik

Title: Re: DXO tests
Post by: voidshatter on December 16, 2014, 07:27:43 am
I've written many times about my technical issues with DXO. They very carefully (and I trust, honestly and accurately) measure a set of numbers which are of marginal use to most photographers.

I am the minority of photographers (not pro working in the studio) so I can see how the DXO scores matter in my real world usage for pixel peeping. For this I agree with Bill.

http://www.getdpi.com/forum/medium-format-systems-digital-backs/52844-backlight-landscape-photography-realized-say-no-silhouette.html
Title: Re: DXO tests
Post by: Theodoros on December 16, 2014, 11:58:27 am
I am the minority of photographers (not pro working in the studio) so I can see how the DXO scores matter in my real world usage for pixel peeping. For this I agree with Bill.

http://www.getdpi.com/forum/medium-format-systems-digital-backs/52844-backlight-landscape-photography-realized-say-no-silhouette.html

Why would you disagree with this http://nikonrumors.com/2014/12/03/the-best-nikon-cameras-and-lenses-according-to-senscore-and-lenscore.aspx/#more-84608 (that disagrees with DXO) then?
Title: Re: DXO tests
Post by: ErikKaffehr on December 16, 2014, 12:35:32 pm
Hi,

Your posting illustrates the weak points of the MFDBs quite clearly. Doug himself posted a comparison of the IQ-250, IQ-260 and the IQ-280 which showed a great advantage of the IQ-250 over the IQ-260/IQ-280 in shadow detail.

Very clearly, DxO measures just some aspects of performance, but I would suggest that what they measure, they measure well.

Little doubt, when (or if) DxO tests the IQ-250 it will go to the top. The sensor size would give it an advantage of about 10 points on the DxO-mark scale.

Best regards
Erik

I am the minority of photographers (not pro working in the studio) so I can see how the DXO scores matter in my real world usage for pixel peeping. For this I agree with Bill.

http://www.getdpi.com/forum/medium-format-systems-digital-backs/52844-backlight-landscape-photography-realized-say-no-silhouette.html
Title: Re: DXO tests
Post by: voidshatter on December 16, 2014, 02:22:50 pm
Why would you disagree with this http://nikonrumors.com/2014/12/03/the-best-nikon-cameras-and-lenses-according-to-senscore-and-lenscore.aspx/#more-84608 (that disagrees with DXO) then?
I do not disagree with this. This does not contradict dxomark. They just have different weights from different aspects. If you shoot wedding where you are have limited light condition with a bound of shutter speed then of course senscore makes more sense than dxomark overall score.
Title: Re: DXO tests
Post by: Theodoros on December 16, 2014, 03:28:22 pm
Soooo, they're both right! ....no? ...and therefore so are other "weights", ...right?
Title: Re: DXO tests
Post by: ErikKaffehr on December 16, 2014, 04:23:42 pm
Hi,

According to the information at Senscore the major difference to DxO-mark that DxO-mark is to a large extent based on base ISO, with "low light" part taking high ISO noise into account, while the Senscore is covering a large ISO range.

On the other hand, DxO actually publishes data for all ISOs and even includes a few MFD systems.

Best regards
Erik


I do not disagree with this. This does not contradict dxomark. They just have different weights from different aspects. If you shoot wedding where you are have limited light condition with a bound of shutter speed then of course senscore makes more sense than dxomark overall score.
Title: Re: DXO tests
Post by: bcooter on December 16, 2014, 04:37:12 pm
Hi,

According to the information at Senscore the major difference to DxO-mark that DxO-mark is to a large extent based on base ISO, with "low light" part taking high ISO noise into account, while the Senscore is covering a large ISO range.

On the other hand, DxO actually publishes data for all ISOs and even includes a few MFD systems.

Best regards
Erik



I stay away from these threads (obviously I didn't today), because this type of stuff is killing this section of the forum.


IMO

BC
Title: Re: DXO tests
Post by: eronald on December 16, 2014, 07:17:56 pm
I stay away from these threads (obviously I didn't today), because this type of stuff is killing this section of the forum.


IMO

BC

J,

 There is such a thing as overtesting. However if you want to know at one glance how a new camera will fare wrt DR and ISO, take your current model, read the DxO number, take the new one, add the stop difference. And there is *nothing* more important in PRACTICE than knowing the ISO/DR is sufficient -

 BTW, the daylight video from the GH4 looks like MF shots. The stills look like sh*t. I don't get it. And in fact I don't care :)

Edmund
Title: Re: DXO tests
Post by: JV on December 16, 2014, 08:28:38 pm
BTW, the daylight video from the GH4 looks like MF shots. The stills look like sh*t. I don't get it. And in fact I don't care :)

I never bonded with the Panasonic for stills.  I had the GF1 and the GH2. Sold them earlier this year,together with the 3 lenses I had. 

I always wondered whether the GH4 would have been any better but apparently not.
Title: Re: DXO tests
Post by: eronald on December 16, 2014, 09:39:24 pm
I never bonded with the Panasonic for stills.  I had the GF1 and the GH2. Sold them earlier this year,together with the 3 lenses I had. 

I always wondered whether the GH4 would have been any better but apparently not.

Yes, it is strange. The video quality out of camera  is breathtaking. There is an interesting twist in that the camera can extract 8MP stills from video clips ...

As a camera, it's interesting, everything is economical but sufficient. The still AFS is superfast, but it still hunts a bit during video in AFC which is annoying and forces me to prefocus; that is the only real annoyance I see; I guess a real video camera would have built-in ND filters and a smooth zoom.

Edmund
Title: Re: DXO tests
Post by: bcooter on December 17, 2014, 12:43:18 am
J,

 There is such a thing as overtesting. However if you want to know at one glance how a new camera will fare wrt DR and ISO, take your current model, read the DxO number, take the new one, add the stop difference. And there is *nothing* more important in PRACTICE than knowing the ISO/DR is sufficient -

 BTW, the daylight video from the GH4 looks like MF shots. The stills look like sh*t. I don't get it. And in fact I don't care :)

Edmund


No.  

If you want to understand a camera, buy, rent, borrow and and shoot something you've never done before.  Something that takes waiting until the light is pretty, or that is slick, or gritty, compelling or humorous, dangerous or safe.  

Something you're proud of when you see it, stumble upon it, make it,  construct it in front of the lens, not after so it can be lab put in an electron microscope.

Every camera I've owned, someone said was "not right". REDS, my old Phase backs, the leicas, micro 43, Canons, (wow do the interweb techies love to beat up on Canon).

The first serious digital camera I bought was a dcs/760.  I read and read and read.  Every review said the Nikon was better, or a kodak proback, every one.

i loved that camera, it made me bank and I learned more from using it than any review, lab test, test chart, color checker, micro detail blowup would have ever shown me.

Images from that camera are in the new opening of our website, so it's not dxo that matters . . . well I guess I should be slightly open minded and say it doesn't matter to me.

I ONLY went to higher rez cameras becasue at 6mpx it was on the verge of a few issues and moved to the 1ds, which I absolutely promise you I could use today for every still session I do and no client, pundit, or electronic magazine review would change that.

IMO

BC

P.S.  The GH4 will shoot a great still.  Just work it in post and find it's sweet spot.  All cameras have one.
Title: Re: DXO tests
Post by: Chris Livsey on December 17, 2014, 02:48:37 am

IMO

BC


The voice of sanity.
Title: Re: DXO tests
Post by: amsp on December 17, 2014, 06:49:28 am
Certain individuals on these boards are the kind of people that would buy a paint brush with Kolinsky Sable hair, count and measure the hairs, and then conclude that a roller is superior because it covers more surface in less time when painting your kitchen. They would then proceed to spend years on a fine art forum telling everyone they should be using rollers instead, posting pictures of their kitchen wall every chance they get.

Could it be that some people who have spent a lifetime being the "expert" of something start defining their self worth by how much others value their opinion? Then when they retire and nobody cares what they think anymore feel a void that they need to fill by buying things they don't actually need, test it to death, and then spend all their time writing about it instead of using it.

...nah, I'm sure I'm way off base  ::)
Title: Re: DXO tests
Post by: ErikKaffehr on December 17, 2014, 11:42:40 am
Hi,

I would say that you a bit of base, yes…

If you look at the title of the thread it says:

Luminous Landscape Forum > Equipment & Techniques > Medium Format / Film / Digital Backs – and Large Sensor Photography > DXO tests

And it is simply a discussion what DxO measures and what they don't.

Now, the way I see this is:


Now, it may be a bit unfortunate that this discussion is on the
"Luminous Landscape Forum > Equipment & Techniques > Medium Format / Film / Digital Backs – and Large Sensor Photography" subforums as it is a bit general topic, but it is the forum the OP has started it.

Best regards
Erik

Certain individuals on these boards are the kind of people that would buy a paint brush with Kolinsky Sable hair, count and measure the hairs, and then conclude that a roller is superior because it covers more surface in less time when painting your kitchen. They would then proceed to spend years on a fine art forum telling everyone they should be using rollers instead, posting pictures of their kitchen wall every chance they get.

Could it be that some people who have spent a lifetime being the "expert" of something start defining their self worth by how much others value their opinion? Then when they retire and nobody cares what they think anymore feel a void that they need to fill by buying things they don't actually need, test it to death, and then spend all their time writing about it instead of using it.

...nah, I'm sure I'm way off base  ::)
Title: Re: DXO tests
Post by: sgilbert on December 17, 2014, 04:45:36 pm
Erik,

I suspect that the post you found off base wasn't referring to DxO sharing information. 

SG
Title: Re: DXO tests
Post by: eronald on December 17, 2014, 07:58:41 pm
Certain individuals on these boards are the kind of people that would buy a paint brush with Kolinsky Sable hair, count and measure the hairs, and then conclude that a roller is superior because it covers more surface in less time when painting your kitchen. They would then proceed to spend years on a fine art forum telling everyone they should be using rollers instead, posting pictures of their kitchen wall every chance they get.

Could it be that some people who have spent a lifetime being the "expert" of something start defining their self worth by how much others value their opinion? Then when they retire and nobody cares what they think anymore feel a void that they need to fill by buying things they don't actually need, test it to death, and then spend all their time writing about it instead of using it.

...nah, I'm sure I'm way off base  ::)


I think most of us already have wives to list our faults :)

By the way, may I wish all of you merry end-of-year festivities, and a Happy New Year?

Edmund
Title: Re: DXO tests
Post by: JoeKitchen on December 17, 2014, 08:03:20 pm

I think most of us already have wives to list our faults :)


 :D
Title: Re: DXO tests
Post by: BernardLanguillier on December 17, 2014, 10:56:46 pm
Certain individuals on these boards are the kind of people that would buy a paint brush with Kolinsky Sable hair, count and measure the hairs, and then conclude that a roller is superior because it covers more surface in less time when painting your kitchen. They would then proceed to spend years on a fine art forum telling everyone they should be using rollers instead, posting pictures of their kitchen wall every chance they get.

...nah, I'm sure I'm way off base  ::)

If your assumption is that people cannot possibly have both artistic abilities and an interest in technology, then you aren't any more off base that those thinking that athletes can't read books. ;)

Cheers,
Bernard
Title: Re: DXO tests
Post by: eronald on December 18, 2014, 03:11:51 am
If your assumption is that people cannot possibly have both artistic abilities and an interest in technology, then you aren't any more off base that those thinking that athletes can't read books. ;)

Cheers,
Bernard

Bernard,

Don't waste your time. There's an old american proverb I just made up which says "wrestle a pig, you'll get covered in mud".

Edmund.
Title: Re: DXO tests
Post by: Manoli on December 18, 2014, 04:21:45 am
There's an old american proverb I just made up which says "wrestle a pig, you'll get covered in mud".

Will rank among your best posts - a classic.

M
Title: Re: DXO tests
Post by: ondebanks on December 18, 2014, 10:52:26 am
I stay away from these threads (obviously I didn't today), because this type of stuff is killing this section of the forum.


IMO

BC

BC,

Are you the kind of person who shows up at parties, just to tell them that you have no intention of coming in to their party?

Look: the title of the thread is "DXO tests". There's a man walking slowly ahead of that thread, waving a red flag, and calling through a hailer: "Tech alert! Here be Geeks! They're talking about DxOmark tests! Stay indoors if you are of a tech-sensitive disposition! And whatever you do, don't click into the thread! The techies are coming! THE TECHIES ARE COMING!!".

And yet, here you are...  ???

Ray
Title: Re: DXO tests
Post by: bjanes on December 18, 2014, 12:08:52 pm
Bernard,

Don't waste your time. There's an old american proverb I just made up which says "wrestle a pig, you'll get covered in mud".

Edmund.

Edmund,

Well said, but sometimes it is necessary to wrestle with pigs in order to castrate them. Ask Joni Ernst ;D

Regards,

Bill
Title: Re: DXO tests
Post by: amsp on December 18, 2014, 12:59:15 pm
So now I'm a pig AND I should be castrated, I must have touched some nerves around here. Stay classy guys!  ::)
Title: Re: DXO tests
Post by: bcooter on December 18, 2014, 01:36:26 pm
BC,

Are you the kind of person who shows up at parties, just to tell them that you have no intention of coming in to their party?


No.

I just thought I was going to a party at the Sky Bar only to find out it's been taken over by Denny's.

Edmund,

Since you like quotes based on even-toed ungulates . . . a final saying for you.    

Not my pig, not my farm.


IMO

BC
Title: Re: DXO tests
Post by: Ken R on December 18, 2014, 01:51:55 pm
Technocrats always have the last word around here.

I mean, some people have well 8,000 posts on this forum, that alone should tell you something.

Thank god they do not determine my gear choices.

Title: Re: DXO tests
Post by: Slobodan Blagojevic on December 18, 2014, 06:41:19 pm
That is quite correct. Roger Clark (http://www.clarkvision.com/articles/low.light.photography.and.f-ratios/index.html) explains this in some detail using the concept of Etendue (sounds like a French word, maintaining the relationship with DXO). The larger sensor goes with a larger lens.

Bill,

Pardon my ignorance and impatience to read every word of that interesting article. I am not sure I understood the concept of "larger lens" or "wider aperture lens." Are you saying:

1. That a lens with a wider max aperture (say f/2) is always going to result in better image quality than a, say, f/4 lens, regardless of the actual aperture used or...

2. That a lens with a wider max aperture (say f/2) will only result in better image quality at apertures wider than the other lens?
Title: Re: DXO tests
Post by: ErikKaffehr on December 18, 2014, 11:27:10 pm
Hi Slobodan,

My take on the issue is that a larger sensor can collect more photons. A larger sensor will also go with a larger lens,as you would need a longer focal length for the same angle of view. Photons are important, because much of the noise is just statistical variation of the number of photons reaching the pixels.

Let's take a simple example. You go from say 16 MP APS C to say 20 MP "full frame". The full frame sensor will have 1.6^2 = 2.56 the area of the APS/C sensor. So it can collect 2.56 times the number photons (all other factors kept constant).

Now, let's assume that you use a decent quality 35 mm lens on the APS-C camera and use f/5.6, which probably is pretty optimal. That means the area of the aperture will be 31 mm^2.

Now, on the full frame a 55 mm lens would give you the same angle of view. You could still shoot at f/5.6 the area of the aperture would than be 76 mm^2. So using same exposure time 76/31 = 2.5 times more photons would leak trough that aperture. So you would get the same number of photons per mm^2 on both sensors.

Now, if you needed that f/5.6 to achieve a given DoF on the APS-C and would need similar DoF full frame, you would need to stop down to 5.6 * 1.6 = 8.9 (if my memory don't fails me). If you shoot on tripod, you can increase exposure time to compensate, and still get the same amount of photons. Or yo can increase ISO, which means collecting less photons.

As a side note, astronomers like diffraction limited systems, and they don't need things like aperture. More often than not, they are looking at stars and just want to collect light from those stars. Astronomers also seldom have problems with DoF :-), so for astronomers a big lens is a big advantage. A larger lens collects more photons, but a larger lens has also less diffraction, so the spot image of the star will be brighter. As far as I recall, in film times a telescope having twice the diameter would have sixteen times the detection capability. With digital sensors the case may be different as not so much is gained once the star image is smaller that the pixel size of the sensor. Ray perhaps has some interesting input on the issue.

Best regards
Erik



 


Bill,

Pardon my ignorance and impatience to read every word of that interesting article. I am not sure I understood the concept of "larger lens" or "wider aperture lens." Are you saying:

1. That a lens with a wider max aperture (say f/2) is always going to result in better image quality than a, say, f/4 lens, regardless of the actual aperture used or...

2. That a lens with a wider max aperture (say f/2) will only result in better image quality at apertures wider than the other lens?